6 ms·

Well, consistency and availability are the problem here. While enforcing one of the property, you lose on the other. Not sure how would you achieve those opposi

Well, consistency and availability are the problem here. While enforcing one of the property, you lose on the other. Not sure how would you achieve those opposing properties without introducing huge amount of complexity. In this case, replication consists of communication between 2 instances, and when one them fails you need to pick consistency or the availability. Paxos and Raft introduce consensus using multiple instances communicating with each other to mitigate those problems. Implementing correctly communication with 2 instances is hard enough, implementing it correctly for n instances is for sure harder. In Memgraph, we actually tried to implement Raft and we did manage to some degree, but the effort was too much, especially when you look at the results we got (protocol that didn't work for many edge cases).

macintux 6y ago
Any chance you’ll have aphyr take it out for a spin with Jepsen?
- mbuda 6y ago
  We'll probably put the replication feature in the Memgraph Community soon. At that point, we'll make a PR to the Jepsen repo with all our tests included + yea, additional testing will be possible. That's always good. As they say, once you wrote a bunch of tests for your code, please leave it to the others to test it as well :D
cpard 6y ago
thanks for the detailed response. Did you try to implement Raft yourself or use something like etcd for coordination? It would be great to write about this journey too in the future.
- mbuda 6y ago
  We implemented Raft as a C++ library from scratch. We can probably publish that as s standalone implementation (open-source it). The main problem was the write performance and network communication overhead, even for read queries. There are some known solutions for all that. None of the Memgraph users required Raft type of guarantees, so we decided to go with a more straightforward and faster solution. It doesn't mean we won't get back to Raft in the future if required :D
  - cpard 6y ago
    open sourcing such projects is always a good thing :)

macintux 6y ago

Any chance you’ll have aphyr take it out for a spin with Jepsen?

mbuda 6y ago
We'll probably put the replication feature in the Memgraph Community soon. At that point, we'll make a PR to the Jepsen repo with all our tests included + yea, additional testing will be possible. That's always good. As they say, once you wrote a bunch of tests for your code, please leave it to the others to test it as well :D

cpard 6y ago

thanks for the detailed response. Did you try to implement Raft yourself or use something like etcd for coordination? It would be great to write about this journey too in the future.

mbuda 6y ago
We implemented Raft as a C++ library from scratch. We can probably publish that as s standalone implementation (open-source it). The main problem was the write performance and network communication overhead, even for read queries. There are some known solutions for all that. None of the Memgraph users required Raft type of guarantees, so we decided to go with a more straightforward and faster solution. It doesn't mean we won't get back to Raft in the future if required :D
- cpard 6y ago
  open sourcing such projects is always a good thing :)