8 ms·
Why is "written in C++" part of the headline? As engineers we focus too much on the implementation details and not the benefits to the user. How about: - Zoo
by lambda_garden 3y ago
Why is "written in C++" part of the headline?
As engineers we focus too much on the implementation details and not the benefits to the user.
How about:
- ZooKeeper alternative with lower latency
- ZooKeeper alternative with lower memory use
- ZooKeeper alternative with predictable overheads
(I don't know if these are true, just suggestions)
- GauntletWizard 3y agoI'd be most interested in a Zookeeper alternative that doesn't have massive bugs in leader election.
- abronan 3y agoSince the article states that they're using Raft and not ZAB for the consensus algorithm and leader election, it must be less prone to bugs when it comes to electing a leader. Since Raft is easier to reason about and the leader election process is more straightforward (Raft minimizes the chance that any two nodes will be candidates at the same time and thus avoids starting multiple concurrent elections).
- antonio2368 3y agoJust adding on to a nice response from abronan, our internal protocol also does some optimizations when it comes to leader election, e.g. Pre Vote protocol (https://github.com/eBay/NuRaft/blob/master/docs/prevote_protocol.md https://github.com/eBay/NuRaft/blob/master/docs/prevote_prot...) Also, we apply many different faults in our Jepsen tests which are run 3 times a day and we never had a problem with leader election. I know this doesn't confirm that there is no bug in it but it's pretty reassuring I would say.
- tbragin 3y agoDisclaimer: I worked on this blog with the team at ClickHouse. I like your suggestions! Some of the benefits we summarized in this summary page https://clickhouse.com/clickhouse/keeper https://clickhouse.com/clickhouse/keeper include ease of setup and operation, no overflow issues, better compression, faster recovery, (dramatically) less memory used, etc.. There was actually a reason why C++ was important for us at ClickHouse, and it's because C++ is our main code base and managing a Java project as part of it was not natural, but you right - for standalone use of this alternative, that doesn't matter.
- jsiepkes 3y agoHow much did memory safety factor in the decision? I mean only last week the IT world was yet again bitten by a major memory safety bug, in libwebp.
- nanolith 3y agoGiven that their entire code base is written in C++, and switching to a different language would be a significant retooling for the team, I think it's reasonable to assume that it did not. Language choice is rarely made on the grounds of specific features, and is often made on the grounds of ergonomics and team knowledge. A more revealing question is, "How are you dealing with memory safety in this implementation?" There are ways to improve memory safety in C++ through tooling and idiomatic style. Are these things being used?
- antonio2368 3y agoKeeper is tested in the same way as ClickHouse. There are Keeper only tests, but we run ClickHouse with Keeper for all of our server tests. For each test we try to use all useful tools for verifying safety and correctness like sanitizers. E.g. an interesting tool we introduced in our codebase for thread safety https://clang.llvm.org/docs/ThreadSafetyAnalysis.html# https://clang.llvm.org/docs/ThreadSafetyAnalysis.html# We found some issues using sanitizers in our codebase and NuRaft library itself which were instantly fixed. And let's not forget about Jepsen which showed some really tricky bugs but were more related to the correctness.
- nanolith 3y agoThanks. That is useful. I would suggest looking into CBMC and similar tools as well. Model checking is incredibly useful.
- antonio2368 3y agoSadly I never put enough effort into trying out such checks. but your excitement about them gives me motivation to properly try them out.
- betaby 3y agoJava vs C++ is very important implementation detail, especially for 'the benefits to the user'. Java is commercial platform which requires a fee to Oracle if used in enterprise, while C++ complied binary does not.
- pjmlp 3y agoThe usual FUD, no it doesn't require any fee, use OpenJDK. Several distributions to chose from.
- icedchai 3y agoOr you can just use OpenJDK, right?
- jsiepkes 3y agoThat's a _very_ incorrect statement. You can use any OpenJDK (which is GPLv2 with classpath exception) distribution you want to run Apache Zookeeper without having to have any agreement with Oracle or pay any fee. The Oracle JDK is just Oracle's commercial version of their OpenJDK distribution with Oracle support. You can use the OpenJDK distro shipped in your Linux distro (RedHat, Debian, etc.), you can use Microsoft's OpenJDK distro[1], you can use the Eclipse OpenJDK distro, you can use Amazon's OpenJDK distro [3] and there are a whole bunch more. [1] https://www.microsoft.com/openjdk https://www.microsoft.com/openjdk [2] https://adoptium.net/ https://adoptium.net/ [3] https://aws.amazon.com/corretto/ https://aws.amazon.com/corretto/
- wiseowise 3y ago
- deleted 3y ago[deleted]
- nickHN2023 3y agoDisclaimer: I working on this blog at ClickHouse with the team. We'll look in to them and adding to some of our social promotion over the coming weeks. Will try to find a way to give you credit.
- stonemetal12 3y agoIt is to let you know to not use it, since it is a pile of Memory related CVEs just waiting for the joy of discovery.
- zX41ZdbW 3y agoThat's true - C++ libraries are typically bug-ridden and require exhaustive efforts to clean up. But the latest bugs found by ClickHouse continuous integration system in the related library were fixed about a year ago: https://github.com/eBay/NuRaft/pull/373 https://github.com/eBay/NuRaft/pull/373 https://github.com/eBay/NuRaft/pull/392 https://github.com/eBay/NuRaft/pull/392
- jsiepkes 3y agoIf even Google, Mozilla, Apple and Microsoft can't get C++ right in their apps with all the fuzing tools they invented I don't have much faith other companies are going to fare better with memory safety and C++.
- zX41ZdbW 3y agoZooKeeper alternative with: 1. Snapshots and logs take much less amount of space on disk due to better compression. 2. No limit on the default packet and node data size (it is 1 MB in ZooKeeper) 3. No zxid overflow issue (it forces restart for every 2 bn transactions in ZooKeeper) 4. Faster recovery after network partitions due to the use of a different distributed consensus protocol. 5. It uses less amount of memory for the same volume of data. 6. It is easier to setup, as it does not require specifying the JVM heap size or a custom gc implementation. 7. A larger coverage by Jepsen tests. (This could be hard to believe, but true - ZooKeeper is tested by Jepsen, but Keeper takes the existing tests and adds more). 8. The possibility to store snapshots and previous logs on S3. C++ isn't a key detail, just a consequence of the fact that the main ClickHouse code base is written in C++. If you need a distributed consensus system but not necessarily compatible with ZooKeeper, there are plenty of options: Etcd, Consul, FoundationDB...
- adra 3y ago2. is configurable (with specific caveats) 3. I believe this was solved a very long time ago? Don't epochs rollover automatically now? 6. is this remotely relevant? You still want limits in cloud deploys so not sure how this is remotely a consideration given it takes 2 minutes when you first set it up to use best practices settings.
- klysm 3y agoRunning services on the JVM is terrible and requires much more resources than other platforms