mjpt777

8 ms·

1.
by mjpt777 10y ago
The driver does not allocate in the main message flow so GC does not factor in. Also the driver can run out of process and not be impacted by client processes.
2.
by mjpt777 11y ago
Thanks for the feedback. I'll adjust my terminology to be more precise as it is the right thing to do. Doing things in the open is a great way to learn. I'm not shy of airing my laundry :-)
3.
by mjpt777 11y ago
While one thread is in the action of rotation other producers return right away from the offer. The possibility of starvation occurs if the same thread each time it retried, that buffer has advanced to the next rotation. If adding 100 byte
4.
by mjpt777 11y ago
When I re-read the definitions I can see what you take from it. I think it comes down to what you consider as systemic progress. With respect to the preciseness of the definitions I have likely misinterpreted this. Is the system the algorit
5.
by mjpt777 11y ago
I've read the _The Art of Multiprocessor Programming_ and am happy to read it again. Maybe I have misinterpreted. So if "killed mid-operation" must be supported then I don't see how many algorithms can be said to make pr
6.
by mjpt777 11y ago
Writer two is not blocked, it is not waiting, it is not being obstructed. It is wait-free. It returns and can do whatever work it wishes. However the data structure would be corrupt if writer one never completed. This is corruption and not
7.
by mjpt777 11y ago
You keep saying you are not an expert but keep pushing the point :-) Try this. A process could have a rogue thread scribbling on memory and thus corrupting it. Such a process then that has this thread, by your definition, cannot be wait-fre
8.
by mjpt777 11y ago
If you look at the code the reader does not read the tail. It skips forward reading the length fields. It would never see the writer two message and it never blocks. It just thinks there are no new messages.
9.
by mjpt777 11y ago
Please read the algorithm in code. No threads ever wait. :-)
10.
by mjpt777 11y ago
I understand the point that we need to make software robust. No single algorithm can cope with every possible failure mode. That is why we need multiple instances run on multiple nodes to be resilient. This is independent from if this is a
11.
by mjpt777 11y ago
Thanks for the link. If you look at the implementation no thread is ever stopped from completing. If a producing thread was killed mid operation by another malicious thread then the stream is broken. Other threads are never blocked. Other w
12.
by mjpt777 11y ago
Can you point to a respected publication that states that is the definition of a wait-free algorithm, i.e. that other threads must help and it must cope with killing the thread externally? By kill a thread I assume you mean use the OS to in
13.
by mjpt777 11y ago
Can you expand on how you see the writer can "fail"? Also please relate this to other algorithms you see as wait-free but don't have your failure issue. The code is very simple and does not have external dependencies. It is a
14.
by mjpt777 12y ago
I asked the question if this is based on conjecture or experimental evidence. I'm happy to base my view on experimental evidence from my own research and testing.
15.
by mjpt777 12y ago
I'm sorry but this analysis is completely wrong. I don't work for Azul but have used C4 and spent time studying it. C4 does not use transactional memory, it does however employ a very elegant lock-free algorithm which allows conti
16.
by mjpt777 14y ago
x86 is a total store order memory model so any ASM MOV instruction that writes to a memory address will eventually be seen when the store buffer drains. In code we must ensure the write is not register allocated. This is achieved by the u
17.
by mjpt777 14y ago
You are absolutely right that volatile is inadequate for ordering C/C++ concurrent algorithms, and memory barriers/fences are additionally required. I tried to focus on the hardware in this article.
18.
by mjpt777 14y ago
I like to have only one writer thread for any data. This can have its own flyweight. Other threads can be readers each having their own read only flyweight implementation.
19.
by mjpt777 14y ago
In finance you may be re-evaluating a whole portfolio of assets, or doing a value at risk (VAR) calculation across everything. More often you want low-latency access to the entire dataset without going to disk. For this the entire data mu
20.
by mjpt777 14y ago
Interesting. It sounds like your issues are IO dominant since you do not mind the JVM startup cost from Hadoop for each query on each node. I'm more often looking at large data that is all memory resident which tends to drive the design t
21.
by mjpt777 14y ago
How would it work with object pooling if I wanted to query a large table of data? This is often needed in real big data applications.
22.
by mjpt777 14y ago
Most of those frameworks you reference are much greater than 1 microsecond round trip which is the same ballpark as good network IO. For IPC this is much less. I operate in the financial low-latency space where this matters.
23.
by mjpt777 14y ago
State of the art network hop is less than 3us so this is relevant. For on box IPC which can be as low as 100ns then very very relevant.
24.
by mjpt777 14y ago
The byte[] buffer is deliberate because that is what will typically be passed to the network or IO device, so easier integration. If you check the code you will see the buffer is only allocated once so the cost is not an issue.
25.
by mjpt777 15y ago
A good posting showing the effect of setting thread affinity for version 1.0 of the Disruptor. http://java.dzone.com/articles/java-threads-steroids
26.
by mjpt777 15y ago
Single nodes can die in the system without issue. They often do! Since we use IP multicast the network failure is transparent as a replica takes up the primary role. The one issue to be managed with this type of system is exceptions in th
27.
by mjpt777 15y ago
The business logic simply outputs state changes that other systems subscribe to. If the subscribing system misses the update our reliable message delivery will ensure the message gets to the subscribing system and processed. Our system
28.
by mjpt777 15y ago
All transactions processed by the LMAX system are ACID. The application is modelled such that each input event represents one transaction. Where business transaction span multiple nodes, rather than handle the complexity of retaining ACI
29.
by mjpt777 15y ago
The LMAX system is more about latency than throughput. When designing for very low-latency the result can be a system that achieves very great throughput if the appropriate techniques are employed. Single threaded applications are very s