9 ms·
Rust vs Zig Benchmarks
- EscapeFromNY 3y agoDidn't these benchmarks used to have box plots for every comparison? Or am I thinking of another site?
- cristoperb 3y agoYou're probably thinking of this one: https://benchmarksgame-team.pages.debian.net/benchmarksgame/box-plot-summary-charts.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
- silisili 3y agoHappy for this new one. The one you linked was/is maintained by someone with rather strong opinions that exclude things and may skew results.
- igouy 3y agoInnuendo.
- EscapeFromNY 3y agoThanks yeah that's the one. It's a completely different site, but the design looks similar to my eyes. That's probably why I got them confused
- igouy 3y ago"It's inspired by Benchmarks Game, some of the benchmark problems and implementation are borrowed from it." https://programming-language-benchmarks.vercel.app/ https://programming-language-benchmarks.vercel.app/
- josephg 3y agoTldr; they’re about even. Which is what I’d expect given they’re both performance oriented languages which compile via llvm. As I see it, performance wise Rust has one advantage and one disadvantage compared to zig. Rust’s advantage is that it can add the equivalent of C’s noalias all over the place because of the rules imposed by the borrow checker. This can help the optimizer. And the drawback of rust is that all array accesses are bounds checked. (Well, at least in safe rust). But thanks to prediction intrinsics, the slow down from this is much less than I always expect. Bounds checks do bloat the binary size though. So rust and zig trading blows benchmark to benchmark is about what I would expect to happen. And that’s exactly what I’m seeing here.
- drcode 3y agoI did view some of the source files though, and the rust ones look to be a lot longer :-)
- galangalalgol 3y agoNbody looked about the same, zig had really long lines for stuff that rustfmt split. But that aside, I agree the zif metaprogramming really shome here.
- dleslie 3y agoWhich is amazing, because Rust comes with more safety guarantees.
- bsder 3y agoWhy is this amazing? One would expect the language with more restrictions to compile to faster code, no?
- deleted 3y ago[deleted]
- simias 3y agoOnly if the restrictions are static and don't need to be checked dynamically, naturally. For dynamic arrays or dynamic indices it's not always possible to do that in Rust.
- Laremere 3y agoJust posting numbers without commentary doesn't seem that useful. Per the GitHub "Note that implementations might be using different optimizations, e.g. with or w/o multithreading, please do read the source code to check if it's a fair comparision or not." I think this shows that Zig isn't a slow language despite its relative youth, but it'd be much more useful if someone did the work to look through the code and provide commentary on comparable the entries are.
- galangalalgol 3y agoThe optimizations both looked correct. Both told the compiler to target broadwell. The fastest nbody was rust, but it was non-portably using x86 intrinsics. Zig has explicit simd vectors in the stdlib and so did better than the portable explicit simd of the third place rust entry. However, zig is using optimized float mode equivalent to gcc ffastmath so it is almost certainly getting the wrong answers simce it didn't use the iterative sqrt trick. https://github.com/hanabi1224/Programming-Language-Benchmarks/issues/246 https://github.com/hanabi1224/Programming-Language-Benchmark...
- ynik 3y agoIs zig's optimized float mode also extremely error-prone like gcc ffastmath? Reminder: With gcc/clang, -ffastmath makes it undefined behavior to run a calculation that results in an infinity or NaN. Due to the way UB works, the compiler can end up miscompiling not just the floating-point calculation, but also other code nearby (e.g. delete array bounds checks). This is why Rust does not have any fastmath-equivalent: it would allow violating memory safety in safe code.
- dataangel 3y agoSeems like there should just be a fastmath mode that doesn't consider it UB. IIUC most of the gains come from being able to assume addition, multiplication etc are associative and commutative.
- vanderZwan 3y ago
- devit 3y agoSeems kind of pointless, since the implementations for different languages are independently written and thus the results depend mostly on the skill of the programmers who wrote them (at least for languages that are efficient), and on the quality of the specific standard library features being used if any.
- gabeio 3y agoNot entirely pointless since this is open source and programmers of some caliber may be able to contribute better or more efficient code. All benchmarks should be taken with a huge grain of salt since this is rarely what these languages are used for by the majority. As well you shouldn’t be choosing a language solely based on speed and not on the individual application of the language. Memory usage though is something useful and impressive in these examples.
- nmz 3y agoHow do you know/measure what's efficient then?
- pverghese 3y agoKind of shows how many rewrites of the rust code are needed to equalize or surpass the zig implementations which are much simpler to write
- JZerf 3y agoIt's not a good idea to use this site (or the Computer Language Benchmarks Games https://benchmarksgame-team.pages.debian.net/benchmarksgame/ https://benchmarksgame-team.pages.debian.net/benchmarksgame/ that it is partially based off) as an indicator of how may rewrites are necessary in order to generate good programs. The skill levels of the contributors and the size/popularity of the various programming language communities can vary a lot. The benchmark rules have been changing over time and contributors have been figuring out better algorithms over time so these both result in the contributed programs getting updated over time as well. Older programming languages have been around longer than newer programming languages and will have had more contributed programs consequently. Some programming languages are under more active development so require more revamps of existing programs. Etc...
- sergiotapia 3y agoI think a more interesting comparison is Zig vs Nim. Where Nim beats or at the very least matches performance against Zig in multiple tests, while having an infinitely nicer, friendlier syntax. https://programming-language-benchmarks.vercel.app/zig-vs-nim https://programming-language-benchmarks.vercel.app/zig-vs-ni... I love Nim so much, I wish it became more popular somehow. Unfortunately it has stagnated in popularity.
- Buttons840 3y agoI think Nim looks good, but I'm more likely to learn Rust or Zig personally. I've tried to put my finger on why that is, and I think it's because Nim looks like another good language that is just good all around, it makes good trade-offs and finds a local optimum being good at everything. A lot of language have tried this though, I've seen it all before. Rust did something new, they accepted that "yeah, our language might be really painful to write your seat-of-the-pants business logic in" and they did the borrow checker stuff. Zig is similar, keeping many of the pains of C, but making some important improvements.
- hugs 3y agoI feel the same way. I wish Nim was more popular, but I'm not letting that stop me from using it. It's already extremely useful enough. I spent the last few weeks writing Nim code (specifically an OpenCV binding that wraps just the C++ libraries I need). Nim let's me write fast, compiled, "Pythonic" code. And the INim interactive shell let's me program "1 line at a time" the same way I would with the Python REPL or JavaScript console.
- deleted 3y ago[deleted]
- danielEM 3y agoThat pythonic syntax is something that prevents Nim to gain more popularity. I think it would be a great move from Nim side to introduce dual syntax support: python like and c/d/java/JavaScript/typescript/go/rust/zig ... like
- nbittich 3y ago[flagged]
- stephc_int13 3y agoI think that this kind of benchmarks should always be taken with a grain of salt, as anyone with some experience with benchmarks knows that there are many ways to distort the results. Having said that, performance is a key feature for both languages and from what I can see the methodology seems legit. I'd prefer to also have a C or C++ to use as a performance baseline and to get a clearer overall picture of where the current optimizers stands relative to each others. Also, competition is good, I believe that we'll all benefit from it in the end, even if the heated debates and quasi-religious stances can be annoying, in the long run this is mostly noise.
- skemper911 3y agoB->C->C++->D->Rust->Zig->ZZtop?
- Kamq 3y agoYou forgot BCPL
- gkfasdfasdf 3y agoI imagine leetcode has a lot of data on benchmarks of highly optimized solutions implemented in different languages. I wish they would do a blog post or something.
- intelVISA 3y agoHuge missed opportunity imo, can't imagine it would be too hard for them to glue together either.
- tharne 3y agoZig is cool and all, but someone should really rewrite it in Rust.
- 0xfedbee 3y ago[flagged]
- dang 3y agoCould you please stop posting unsubstantive comments and/or flamebait? You've unfortunately been doing it repeatedly. It's not what this site is for, and destroys what it is for. If you wouldn't mind reviewing https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html and taking the intended spirit of the site more to heart, we'd be grateful. (Rust probably does get too much coverage on HN - that's what happens with hype cycles - but posting dross comments about it only makes things worse.)
- soulbadguy 3y agoSlightly on a tangent here. But i think someone with more knowledge of either the zig implementation/language or programming language and runtime in general should take a second pass at Zig's wikipedia page. I see a lot of assertion which doesn't seem substantiated by the provided references, some of the reference almost look unrelated and some stuff also look wrong ( i am not a zig experts) > The goals of Zig are in contrast to the many similar languages introduced in the 2020s time-frame, like Go, Rust, Carbon, Nim and many others. Generally, these languages are more complex with additional features like operator overloading, functions that masquerade as values (properties), generic types and many other features intended to aid the construction of large programs. These sorts of features have more in common with C++'s approach, and these languages are more along the lines of that language. > reference (https://www.infoworld.com/article/3113083/new-challenger-joins-rust-to-upend-c-language.html https://www.infoworld.com/article/3113083/new-challenger-joi...) The provided reference doesn't really mention this statement, and C++ did not originate a lot of those features. On a more abstract level, no having abstraction features doesn't always make a language simpler. > A common solution to these problems is a garbage collector (GC), which examines the program for pointers to previously malloced memory, and removing any blocks that no longer have anything pointing to them. Although this greatly reduces, or even eliminates, memory errors, GC systems are relatively slow compared to manual memory management, and have unpredictable performance that makes them unsuited to systems programming. > reference (https://docs.elementscompiler.com/Concepts/ARCvsGC/ https://docs.elementscompiler.com/Concepts/ARCvsGC/) I am not sure how authoritative the given reference is, but even assuming it is, there is no mentions on manual vs GC speed, or anything about how GC systems are inadequate for system programing (what ever that means in this context). I know the the memory management styles trade-off are still hot debates, and a lot of progress have been made with regard to modern implementation. > Another solution is automatic reference counting (ARC), which implements the same basic concept of looking for pointers to removed memory, but does so at malloc time by recording the number of pointers to that block, meaning there does not need to perform an exhaustive search, but instead adds time to every malloc and release operation. This doesn't sound like the best way to describe ARC..., again the provided references doesn't substantiate this definition. > Zig aims to provide performance similar or better than C, so GC and ARC are not suitable solutions. Instead, it uses a modern, as of 2022, concept known as optional types, or smart pointers. I don't think smart pointer and optional types are the same. Quickly glancing at zig official website , this looks like a mistake > Instead of a pointer being allowed to point to nothing, or nil, a separate type is used to indicate data that is optionally empty. This is similar to using a structure with a pointer and a boolean that indicates whether the pointer is valid, but the state of the boolean is invisibly managed by the language and does not need to be explicitly managed by the programmer. So, for instance, when the pointer is declared it is set to "unallocated", and when that pointer receives a value from a malloc, it is set to "allocated" if the malloc succeeded. Mixing optional types and they way zig uses it to represent references.
- tw061023 3y agoThese are microbenchmarks, thoroughly optimized to the point some of them are written almost completely in SSE intrinsics. I would expect Rust to be 2x-3x slower in real production code, solely due to approach to memory allocation.
- aw1621107 3y ago> solely due to approach to memory allocation Would you mind elaborating on this? I don't think I've read anything (yet?) that touches on performance differences due to that particular aspect of the languages.
- kibwen 3y agoThe only thing I can think of is that Zig makes it easier to use different allocators on a per-collection basis, whereas for Rust the only way to alter the allocator used by the standard containers is to change the global allocator. But Rust programs aren't allocating that much in the first place, certainly not so much that swapping out an allocator on a per-collection basis is going to triple the performance of arbitrary programs; at best, fine-grained allocator control will give modest gains on certain extremely allocation-heavy workloads. And collections in Rust can certainly use custom allocators if they want to, e.g. https://crates.io/crates/bumpalo https://crates.io/crates/bumpalo , so that's not something inherent to the language.
- tw061023 3y agoI respectfully disagree. > to use different allocators on a per-collection basis It's far more than that. There are different memory management optimization patterns that are hard to implement in Rust, ie. splitting allocations to different arenas based on allocation lifetime. > But Rust programs aren't allocating that much in the first place Are Rust programs allocating mostly from the stack then? > is going to triple the performance of arbitrary programs The difference between malloc and any hand-rolled O(1) allocator is enormous, it's just people rarely benchmark this particular aspect. There is a reason why Rust used to ship jemalloc and why #[global_allocator] is a thing. I would argue there's even more reason to avoid touching heap in the first place. Edit: formatting.
- sedatk 3y agoWe should have developer benchmarks for programming languages such as "time to first successful compile", "time to first running copy", "time to find the root cause of a certain bug". Similar to that Sun vs NeXT competition back in the day: https://www.youtube.com/watch?v=UGhfB-NICzg https://www.youtube.com/watch?v=UGhfB-NICzg Maybe platforms like Leetcode and HackerRank can publish such statistics. I'd find such benchmarks way more interesting than comparisons of milliseconds for solutions to some arbitrary algorithms that we'll never use in real life. I'm not saying they aren't useful, but they aren't interesting.
- hurril 3y agoAnd also: retries needed until last bug is fixed.
- xigoi 3y agoHow would you objectively perform such benchmarks?
- sedatk 3y agoLive competition. Benchmarks would just be metrics recorded during the live broadcast, not goals. You can later aggregate those metrics to come up with accurate benchmarks about many scenarios about software development. Think of NBA but for programming.
- fulafel 3y agoThis compares Zig unsafe mode to Rust. Is there a comparison available for the "safe" mode? (I'm assuming the Rust code isn't using "unsafe", if it is ten the "safe" zig numbers are be uninteresting)