7 ms·
The LKM post mentions binary size improvements. One issues I have had with Rust applications is the huge binary size (yes, I know this has improved a bit latel
by kramerger 3y ago
The LKM post mentions binary size improvements.
One issues I have had with Rust applications is the huge binary size (yes, I know this has improved a bit lately). Is there a good comparison between kernel C and kernel Rust code in this regard?
- makapuf 3y agoFrom (1) I guess most of the binsize comes from stdlib that the kernel does not use, so I guess that's two different problems. (1) https://github.com/johnthagen/min-sized-rust https://github.com/johnthagen/min-sized-rust
- vlovich123 3y agoThat wouldn’t explain what the size improvements are from the upgrade as the kernel always built nostd.
- surajrmal 3y agoMost size issues come from not using release builds, using too many dependencies, or overuse of generics. The rust std lib being linked in statically also contributes.The kernel shouldn't suffer from any of these problems. Plenty of embedded use is able to use rust in highly constrained environments without size issues compared to C.
- zadokshi 3y agoSaying part of the problem is “using too many dependencies” is not an overly helpful thing if the ecosystem keeps on trying to download 3Gb of build dependencies because you tried to use some simple little library. The problem is obvious, it’s the solution that is much more difficult.
- Aurornis 3y ago> if the ecosystem keeps on trying to download 3Gb of build dependencies because you tried to use some simple little library. Downloading 3GB of dependencies is not a thing that happens in the Rust ecosystem. Reality is orders of magnitude smaller than that. Why are you exaggerating so much? Some people bristle at the thought of external dependencies, but if you want to do common tasks it makes sense to pull in common dependencies. That’s life.
- gregors 3y agoMaybe they meant node_modules as the joke goes.
- nindalf 3y agoI think it was a false equivalence between node_modules and Rust. Like any language where developers rely on a package manager to pull in libraries will necessarily be 3GB in size.
- Narishma 3y agoThe download size may not be a big issue, but all those dependencies take up a lot of storage space once they're compiled.
- bscphil 3y ago> Downloading 3GB of dependencies is not a thing that happens in the Rust ecosystem. Reality is orders of magnitude smaller than that. Assuming they're talking about the built size of dependencies that are left lying around after cargo builds a binary, they're really not exaggerating by much. I have no difficulty of believing that there are Rust projects that leave 3GB+ of dependency bloat on your file system after you build them. To take the last Rust project I built, magic-wormhole.rs [1], the source code I downloaded from Github was 1.6 MB. After running `cargo build --release`, the build directory is now 618 MB and there's another 179 MB in ~/.cargo, for a total of 800 MB used. All this to build a little command line program that sends and receives files over the network over a simple protocol (build size 14 MB). God forbid I build something actually complicated written in Rust, like a text editor. [1] https://github.com/magic-wormhole/magic-wormhole.rs https://github.com/magic-wormhole/magic-wormhole.rs
- pdimitar 3y agoI am not a fan of this as well but you have to consider that a good part of these are caches.
- pjmlp 3y agoThis is why XCode, Android Studio/NDK, VC++ and co have such huge sizes people complain about, compiled binaries for all major variations of compile flags are part of the download. Also why those GNU/Linux repos are actually multiple DVDs nowadays.
- surajrmal 3y agoIt's not a problem when you compare it to C. You have few available dependencies to choose from with C. If you are equally picky and constrain yourself to parts of the ecosystem which care about binary size, you still have more options and can avoid size issues. For things like a kernel, it is moot as most deps are simply not possible to use anyway. When you consider the full ecosystem, you need to really compare it to alternatives in largely managed languages like Java, go, node, etc. those binaries are far larger.
- bscphil 3y ago> If you are equally picky and constrain yourself to parts of the ecosystem which care about binary size, you still have more options and can avoid size issues. What's an example of this for, say, libcurl? On my system it has a tiny number of recursive dependencies, around a dozen. [0] Furthermore if I want to write a C program that uses libcurl I have to download zero bytes of data ... because it's a shared library that is already installed on my system, since so many programs already use it. I don't really know the appropriate comparison for Rust. reqwest seems roughly comparable, but it's an HTTP client library, and not a general purpose network client like curl. Obviously curl can do a lot more. Even the list of direct dependencies for reqwest is quite long [1], and it's built on top of another http library [2] that has its own long list of dependencies, a list that includes tokio, no small library itself. In terms of final binary size, the installed size of the curl package on my system, which includes both the command line tool and development dependencies for libcurl, is 1875.03 KiB. [0] I'm excluding the dependency on the ca-certificates package, since this only provides the certificate chain for TLS and lots of programs rely on it. [1] https://crates.io/crates/reqwest/0.11.24/dependencies https://crates.io/crates/reqwest/0.11.24/dependencies [2] https://crates.io/crates/hyper/0.14.28/dependencies https://crates.io/crates/hyper/0.14.28/dependencies
- heinrich5991 3y agoProbably ureq[1]. [1]: https://crates.io/crates/ureq/2.9.6/dependencies https://crates.io/crates/ureq/2.9.6/dependencies
- pdimitar 3y ago
- DinoCoder99 3y ago[dead]
- wongarsu 3y agoIf you do release builds, strip debug symbols and turn LTO from thin to full, do dependencies and the static stdlib still matter? You should be only paying for code that's called at that point. At that point I suspect the biggest culprits are overuse of monomorphisation, and often just more stuff happening compared to equivalent C++ code because the language makes larger code bases more maintainable. I'd also count some niceties in that category like better string formatting or panic handling, which is an insignificant cost in any larger software but appears big in tiny hello-world type programs.
- surajrmal 3y agoOveruse of monorphisation by the community is typically the right choice given the average use case for them is servers where this need not make a meaningful difference. As with any ecosystem, choices folks make may not be suitable for everyone. Those differing requirements require fracturing into smaller ecosystems which share common requirements. Ultimately, that's what's happening with rust and it's very healthy to see in my opinion. You can't force the overall ecosystem to optimize for a minority of users. This is also true for everything in general. Having the one best thing for foo isn't as helpful as an array of choices, each with different tradeoffs. You simply choose the one best for your needs. Whether it's cheese at the supermarket, an webserver framework, operating system intrinsics, or command line argument handling. Some things can be standardized and serve as a common base for everyone, but it's challenging to do that without at least one person's requirements. Standards also always feature creep until someone tries to reset it with a new standard which is less complex, but I guess that's a different topic.
- tialaramex 3y agoI believe the menu of options is undesirable until you actually know you have requirements you can evaluate them against. As much as possible even if I do have a choice, there should be a default and I needn't be asked. When I make a Rust project, cargo notices I have git and, since I didn't say otherwise, it mints a Git repo for the new project automatically. It doesn't insist on asking if I want one, and then asking if it should have the obvious name, and then asking if it should use my default local git settings, the defaults for all of these are IMNSHO obvious and my tacit approval via not having explicitly turned this off is enough. Do you want a Doodad, a Gooba or a Wumsy? No idea? Me either. So until I care, I'd rather not be asked to choose. But once I discover that I need something with at least 40% Flounce, I can see that Doodads and Goobas both are rated at 50% Flounce, whereas Wumsy has only 10% Flounce, now we're making an informed choice, it should be easy enough to insist on a Doodad to meet my requirement. If I measure that Monomorphization is out of hand in my codebase I can use dyn to get that back under control for a fair price, but I think the default here is sound.
- Aurornis 3y ago> One issues I have had with Rust applications is the huge binary size Turn off the standard library and your binaries can be incredibly small. This is how it’s used in microcontrollers and the Linux Kernel doesn’t use the full standard library either.
- nequo 3y agoDue to dead code elimination, the compiler already omits all of that part of stdlib that your code doesn’t use.
- pornel 3y agoNot quite. Every Rust program will have some code path that may panic, and the default panic handler uses debug formatting, which uses dynamic dispatch, which prevents elimination of the rest of the printing machinery. There’s panic_immediate_abort unstable setting that makes Rust panics crash as hard as a C segfault, and only then you can get rid of a good chunk of stdlib.
- nequo 3y agoDoes this mean that only the printing machinery is not eliminated or that other parts of stdlib are present in the binary too even though unused?
- fl0ki 3y agoThe printing machinery alone is quite large when you consider that it includes the code & raw data for Unicode, whether or not similar facilities were already available on the host libc. Though you're not likely to avoid that in any non-trivial Rust program anyway, as even a pretty barebones CLI will need Unicode-aware string processing. I generally find Rust binaries to be "a few" megabytes if they don't have an async runtime, and a few more if they do. It has never bothered me on an individual program basis, but I can imagine it adding up over an entire distribution with hundreds of individual binaries. I see the very real concern there, but personally I would still not risk ABI hazards just to save on space.
- mcqueenjordan 3y agoAnother thing just to mention here is `strip`, which IIRC `cargo build --release` doesn't do by default. I think `stripping` binaries can reduce binary size by up to 80-85% in some cases (but certainly not all; just tried it locally on a 1M rust binary and got 40% reduction). FWIW, you can configure this in Cargo.toml: [profile.release] strip = true
- habibur 3y agoYou can strip C compiled binaries too. And that halves the binary size. The point is for example a hello world Rust binaries is 300kb after striping while C compiled one is 15kb. A difference of 20 times.
- pornel 3y agoSuch comparison exaggerates the difference, because it’s a one-time constant overhead, not a multiplicative overhead. i.e. all programs are larger by 275KB, not larger by 20x. Rust doesn’t have the privilege of having a system-wide shared stdlib to make hello world executables equally small. The overhead comes from Rust having more complex type-safe printf, and error handling code for when the print fails. C doesn’t handle the print error, and C doesn’t print stack traces on error. Most of that 200KB Rust overhead is a parser for dwarf debug info to print the stack trace.
- Narishma 3y agoBut C doesn't statically link the standard library by default like Rust does.
- habibur 3y agoHello world deps for C : linux-vdso.so.1 (0x00007fff25cb8000) libc.so.6 => /lib64/libc.so.6 (0x00007fe5f08d9000) /lib64/ld-linux-x86-64.so.2 (0x00007fe5f0ae2000) And for Rust linux-vdso.so.1 (0x00007ffc109f9000) libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f8eda404000) libc.so.6 => /lib64/libc.so.6 (0x00007f8eda222000) /lib64/ld-linux-x86-64.so.2 (0x00007f8eda4a8000) Rather Rust has 1 more dynamically linked library than C.
- bluejekyll 3y agoThis is a good guide on building small Rust binaries: https://github.com/johnthagen/min-sized-rust https://github.com/johnthagen/min-sized-rust This talks about going to extreme lengths on making the smallest Rust binary possible, 400 bytes when it was written, https://darkcoding.net/software/a-very-small-rust-binary-indeed/ https://darkcoding.net/software/a-very-small-rust-binary-ind... The thing is, you lose a lot of nice features when you do this, like panic unwinding, debug symbols, stdlib… for kernel and some embedded development it’s definitely important, but for most use cases, does it matter?
- ericbarrett 3y ago> debug symbols In ye olden days it was common to distribute a binary without debug symbols, but to keep a copy of them for every released build¹. If an application crashed (panicked, signalled, etc.) you got a core dump that you could debug using the stripped binary together with the symbol file. This gave you both smaller binary sizes and full debugging capability at the cost of some extra administration. I'm not sure if this is possible with "stock" Rust, but if you need lean binaries but want to do forensic investigation it's something to look into. 1. https://sourceware.org/gdb/current/onlinedocs/gdb.html/Separate-Debug-Files.html https://sourceware.org/gdb/current/onlinedocs/gdb.html/Separ...
- wongarsu 3y agoIt's possible, but rust follows platform conventions in only doing this by default on Windows. However it is now easy to configure by setting split-debuginfo in your Cargo.toml [1] 1: https://doc.rust-lang.org/cargo/reference/profiles.html#split-debuginfo https://doc.rust-lang.org/cargo/reference/profiles.html#spli...
- carterschonwald 3y agoSo one issue I can imagine being the culprit with rust is the specializing / c++ style semantics of rust generics. C code generics tend to be void* flavored or point to a struct of function pointers. Which will generate less code. Not sure how this translates to the kernel setting thoughb
- pornel 3y agoThat is true. Rust makes it easy to overuse monomorphisation. There are tools like `cargo-bloat` that find these. However, most complaints are about size of “Hello World”, which in Rust is due to libstd always having debug info (to be fixed soon), and panic handling code that includes backtrace printing (because print to stdout can fail). Printing of backtrace is very bloaty, because it parses and decompresses debug info.
- loeg 3y agoAre you talking about https://github.com/rust-lang/compiler-team/issues/688 https://github.com/rust-lang/compiler-team/issues/688 ? I think that issue provides a lot of interesting context for this specific improvement.
- forrestthewoods 3y ago> is the huge binary size Can you quantify this? How big is too big? Ideally for a real program, and not an experiment to make the tiniest possible program. On my Windows machine ripgrep rg.exe is just 4.2mb. Making that smaller feels irrelevant. I’m not convinced that binary size is a real problem. But I’m open to evidence!
- cout 3y agoIt would be nice if it fit on a standard size 1.44MB floppy, but given that I haven't used a floppy drive in about a decade, yeah I guess it doesn't matter much.
- raoulkent 3y agoJust out of interest, what kind of systems do you work on if you've been using floppy discs in the last 25+ years?
- cout 3y agoI think my current motherboard does not have pins for a floppy drive, but every motherboard I've owned before that does. I just kept moving the floppy drive from chassis to chassis every time I upgraded just in case I needed it. IIRC the last time I used a floppy was either to archive old data to CD-ROM or boot the computer when I couldn't find a USB thumb drive. I do still own my first computer, an IBM PS/2 Model 50Z, which still has its original floppy drive. Other parts I upgraded -- the 286 was replaced with a 386 SX/Now!, the 30MB ESDI was upgraded to 100MB, and it now has a full 2MB of RAM. I keep the floppy drive because it reads disks that no other floppy drive has been able to read.