33 ms·

Cargo: predictable dependency management

211 points by aturon 10y ago

dikaiosune 10y ago
I like many things about Rust, but Cargo alone is what initially got me started with it. I was in the process of setting up a fresh C++ project with vendored dependencies, and it was just an absolute nightmare. In contrast, it took me about 5 minutes to get a Rust project set up with comparable dependency complexity.
- akhilcacharya 10y ago
  Speaking of which, is there any good package management system for modern C/C++?
  - dikaiosune 10y ago
    I've had biicode recommended to me (https://github.com/biicode/biicode https://github.com/biicode/biicode), but I haven't tried it yet. EDIT: Also worth noting that a common response I've heard to "I need a better manager for my dependencies" is "you have a system package manager for a reason."
    - McP 10y ago
      What do they say when you reply "Windows"?
      - dikaiosune 10y ago
        I think it goes without saying that those with that sort of response tend not to use Windows, and also often seem to assume that a perfectly in-order Linux/Unix install is the only way to build software.
        FreeFull 10y ago
        And also that you're fine with creating a package for any dependency that doesn't have one.
        dikaiosune 10y ago
        Creating, maintaining, shepherding through distro processes, etc. All so you can then eventually build your own code.
  - McP 10y ago
    Conan seems to have appeared recently but I haven't yet tried it myself. https://www.conan.io/ https://www.conan.io/
  - rsaarelm 10y ago
    The Meson build system http://mesonbuild.com/ http://mesonbuild.com/ seems to have a package manager like dependency management system.
    - imron 10y ago
      It's also very quick and easy to get projects up and running. For *nix based platforms Meson is now my go to build manager for c++ projects.
  - ingve 10y ago
    Conan is the most promising one, they just released a new version: http://blog.conan.io/2016/05/03/New-conan-release-0-9.html http://blog.conan.io/2016/05/03/New-conan-release-0-9.html Arne Mertz has a Vagrant box for people who want to experiment with a "4C" development environment (Clang, Cmake, Conan, Clion). https://github.com/arnemertz/Xubuntu1604_DevBox https://github.com/arnemertz/Xubuntu1604_DevBox
  - boris 10y ago
    Check out https://build2.org https://build2.org
  - fungos 10y ago
    Even if it doesn't has too much support, I highly recommend fips: https://github.com/floooh/fips https://github.com/floooh/fips I have been using it, adding some libs and even contributing some changes. You should really give it a try!
- deleted 10y ago
  [deleted]
- qznc 10y ago
  I understand that old languages like C++ or Java have problems providing a single package manager. It seems that newer languages have roughly equivalent services. Are there any big differences between Cargo and Python/Go/D/Ruby/Javascript/etc?
  - steveklabnik 10y ago
    It depends on how you define "big". Cargo is closest to Bundler + rubygems, but with some aspects of npm.
  - tatterdemalion 10y ago
    I find cargo easier to use than bundler or npm. Specifically I never deal with "environment hell" where some config hasn't been picked up and I'm pulling in the wrong version of a dependency or the language or whatever. This is partly because of the way cargo is designed, but its also helped a lot by the fact that Rust generates statically linked binaries in one build instead of being a continuous interpreter process.
  - haddr 10y ago
    I wouldn't say Java has this problem. Look at Maven for instance. Then look back at Crates, you will notice how many common solutions they share. On the other hand I see that Crates solves the problem of dependency when 2 different version of the same library are used across the project. Maven doesn't solve that and it's painful. However this seems a bigger problem that just a package manager I suppose.
  - mavelikara 10y ago
    Java has Maven Repository. This can be use from within Ant (Ivy), Maven, Gradle, sbt etc. Both Ruby and Python are almost as old, if not older than Java. JavaScript is only slightly younger.
  - riffraff 10y ago
    notice that some of the really old ones (perl, tex) have had package managers for a really, really long time. I think linux (*nix?) vs non-linux heritage might have something to do with it.
SimonSapin 10y ago
It was a happy day when Servo ditched a big hairy pile of makefiles + git submodules and switched to Cargo :)
- Gankro 10y ago
  Happier than Rust 1.0 and the end of The Rustup?
  - cm3 10y ago
    End of The Rustup? AFAICS, it's still downloading a very specific servo-only rustc.
    - kibwen 10y ago
      Servo's efforts to upgrade their version of rustc used to be legendarily traumatic, these days it's comparatively tame. :)
      - cm3 10y ago
        Right, and if the binary toolchains would be built with musl (not targeting musl) statically, we could have a rustc+cargo that works on both glibc and musl, avoiding two versions.
        dbaupp 10y ago
        Could you explain what you're referring to? It seems like a total non sequitur, since as far as I know, there has never been any pain (or even rumour of pain) in a Rust upgrade caused by what libc the compiler uses, but rather breakage in the unstable features that servo uses.
        cm3 10y ago
        If the official rust+cargo was built statically with musl, then a single download would work on glibc and musl distros. Right now, we don't have a download that works on musl, and one has to bootstrap on a glibc system, targeting musl, which makes it impossible to bootstrap rust+cargo on a musl distro without glibc because bootstrapping requires a snapshot.
gue5t 10y ago
Cargo assumes you want to cache downloaded/built dependencies at the granularity of a unix user account. It's very easy to break things if you try to force current cargo cache dependencies at the per-project level and there's been no interest in caching things at a per-machine or shared-between-machines level. If duplicating compilation work is desirable to ensure some amount of noninterference, it should be supported to have a package cache per project. If we can treat Cargo as a pure function from inputs to outputs, we should be sharing these build results across the Internet, at least within the set of machines used by one person. Cargo also refuses to follow the XDG directory specification for its per-unix-user configuration; it should put things in $XDG_CONFIG_HOME and $XDG_CACHE_HOME, but instead dumps both configuration and downloaded/compiled stuff in ~/.cargo.
- wyldfire 10y ago
  > it should put things in $XDG_CONFIG_HOME and $XDG_CACHE_HOME I was going to submit an issue but one already exists [1][2] [1] https://github.com/rust-lang/cargo/pull/148 https://github.com/rust-lang/cargo/pull/148 [2] https://github.com/rust-lang/cargo/issues/1734 https://github.com/rust-lang/cargo/issues/1734
- yberreby 10y ago
  > If we can treat Cargo as a pure function from inputs to outputs, we should be sharing these build results across the Internet, at least within the set of machines used by one person. AFAIK, all it takes is a single `build.rs` to make rustc, and therefore Cargo, an impure 'function'. I'm not sure about compiler plugins, but I expect them to behave the same way.
- dikaiosune 10y ago
  There's an ongoing discussion of using build caches with Cargo: https://internals.rust-lang.org/t/is-a-shared-build-cache-a-good-fit-for-cargo https://internals.rust-lang.org/t/is-a-shared-build-cache-a-... TLDR: There's interest, but some wrinkles need to be sorted out.
  - cm3 10y ago
dastbe 10y ago
How does Cargo handle indirect dependency visibility? This article talks about visibility in terms of "do I need to manually include indirect dependencies? No!" but not in terms of "can I accidentally write code against an indirect dependency?" which isn't sufficiently answered. If the answer is that indirect dependencies are still visible, I'd be interested in knowing if rust-lang/cargo plan to change that, similar to JDK9's module system for re-export.
- SimonSapin 10y ago
  Short answer: `extern crate foo;` doesn’t work if `foo` is an indirect dependency. So you cannot accidentally use something without declaring it in `Cargo.toml`. Longer answer: when running rustc, Cargo passes individual `--extern bar=/path/to/bar.rlib` options for each direct dependency, not just a path to a directory with everything. You can see the exact command with `cargo build --verbose` or `cargo build -v`.
  - deleted 10y ago
    [deleted]
- tatterdemalion 10y ago
  As the sibling answer says, they are not visible. However, crates can re-export symbols they have imported from their dependency, which couples their semver to that dependency's in a way that people have often not realized. This has been the source of some upgrade headaches so far, but its less of a problem than it has been with languages in my experience.
- keeperofdakeys 10y ago
  It's also relevant that rust can use multiple direct/indirect dependencies of different versions, due to the way that symbols are exported.
eridius 10y ago
Something that I'm surprised this page doesn't talk about, and which is very important considering the recent hubbub over left-pad, is that any dependency you get on Cargo can be relied upon to continue to exist forever (well, as long as the crates.io site still exists, but if that goes away so does the Cargo index). The reason is because you can't ever remove a published version of your crate from crates.io. You can yank a version, which tells Cargo not to allow any projects to form new dependencies on that version, but the version isn't actually deleted and any projects that have existing dependencies on that version will continue to be allowed to use it. This is documented at http://doc.crates.io/crates-io.html#cargo-yank http://doc.crates.io/crates-io.html#cargo-yank.
- waffle_ss 10y ago
  > You can yank a version, which tells Cargo not to allow any projects to form new dependencies on that version, but the version isn't actually deleted That's how RubyGems used to work, except you could contact the support team and ask them to permanently delete your gem version if something really sensitive and irrevocable was put into it. They had to change that due to their support log getting too big: http://blog.rubygems.org/2015/04/13/permadelete-on-yank.html http://blog.rubygems.org/2015/04/13/permadelete-on-yank.html
  - kibwen 10y ago
    And of course, a centralized site can still be compelled to remove a package via legal action if push comes to shove.
  - eridius 10y ago
    Cargo's policy is that if you upload secrets, you need to change the secrets because the code can't be deleted. From the aforelinked page: > A yank does not delete any code. This feature is not intended for deleting accidentally uploaded secrets, for example. If that happens, you must reset those secrets immediately.
    - waffle_ss 10y ago
Siecje 10y ago
How does Cargo manage the case when dependencies X and Y both require a different version of dependency Z?
- steveklabnik 10y ago
  It attempts to unify them if it can, but if they cannot be unified, it will include both versions.
  - cm3 10y ago
    Include both how? Make renamed copies to have two and modify the code to use the new names?
    - steveklabnik 10y ago
      It doesn't have to do any renaming or modifying of names. Rust has a module system, so each dependency will use the version they need. The only time it doesn't Just Work is if one of those crates re-exports a type from the sub-crate in a public manner, and then you try to call something from the other crate with a value of that type. You'll get a compile-time error about mismatched types.
      - cm3 10y ago
        But it sounds like that's what's going on. Using foo-1.1 in bar.rs and foo-2.2 in rabbit.rs.
        steveklabnik 10y ago
        Sure. That will Just Work in every case _except_ if bar and rabbit both re-export a type from foo, and you try to use them together. If they don't re-export anything, then it all works just fine. No renaming or transformations needed.
    - kibwen 10y ago
      Rust mangles symbols with version information, so it's no trouble to have two versions of the same library in the same binary if necessary.
cm3 10y ago
I see there is `cargo install` which will fetch and build an executable from crates.io, is there no `cargo update` for `install`?
- steveklabnik 10y ago
  https://github.com/rust-lang/cargo/pull/2405 https://github.com/rust-lang/cargo/pull/2405 landed, but https://github.com/rust-lang/cargo/issues/2082 https://github.com/rust-lang/cargo/issues/2082 is similar too and is still open.
  - cm3 10y ago
    Interesting, but I was thinking more of something like "update installed binaries with latest version from index". This doesn't look like it.
    - steveklabnik 10y ago
      Ah yeah, it's per-thing, not every thing you've installed, currently. cargo install --list does list all the things you've installed, so it wouldn't be too big of a deal to add; the pieces are all there.
xkarga00 10y ago
I have been working as a Go programmer for the past two years and I really love the language but dependency management has been one of the biggest pains for me. Godep really sucks (that's what we use in my current gig - we plan on switching to Glide, still haven't played around with it) and I am not sure I like the fact that I need to go out in the wild and choose between the rest of the existing solutions. I would prefer to have something that works ootb. The Rust team really got this right and Cargo is amazing. One of the main reason I have been dabbling with Rust lately.
- dalailambda 10y ago
  In my experience Glide is the closest to a modern package manager for Go.
cm3 10y ago
It's great that Raph's line-breaking code has been reused, but in contrast to Mozilla projects any contribution to it will now require assigning copyright to Google (like Ubuntu or Microsoft's CLA require as well). At least it's a tiny piece, so may not hurt too much in terms of prevented contributions.
- gpm 10y ago
  Or just forking it, it's released under a open source license. This wouldn't be the first time Servo has forked a dependency, for example they maintain a fork of glutin (library that handles opening windows) because they want certain features that upstream doesn't. https://github.com/servo/glutin https://github.com/servo/glutin
  - cm3 10y ago
    That can work. Curious what features servo/glutin has that are exclusive?
conradev 10y ago
I appreciate that predictability is a big priority with Cargo. I feel like Rust and Cargo are in a great position to deterministically (per compiler version, per set of build flags) build libraries. That would be an amazing step forward in security if it could be enforced. Does anyone know if this is planned? https://reproducible-builds.org https://reproducible-builds.org
- steveklabnik 10y ago
  In general, we are very interested in reproducible builds, and have fixed bugs where accidental non-determinism has crept in. However, it can be easy to reintroduce with build.rs, which allows for executing arbitrary Rust code before a build. Syntax extensions are another problem here. But the scope is much reduced, it's true.
  - conradev 10y ago
    That's awesome to hear. Would it be possible to make an attribute like #[deny(non_deterministic)] which would error if `build.rs` or similar is present? Also, for usage of things like the `file!` macro, which might mess up determinism based on the build directory. Theoretically, syntax extensions could be written to be deterministic as well (for a given version, of course). Is there a tracking issue on this currently?
    - steveklabnik 10y ago
      There isn't, it might be an interesting idea! Not allowing build.rs would eliminate a lot of crates, and not all build.rs' are nondeterministic... So that's tough.
      - conradev 10y ago
        Yeah. Perhaps developers could attribute functions in `build.rs` specifically with "deterministic", so you could see what in the dependency graph isn't deterministic. Unfortunately, unlike with borrowck, reproducibility is inherently hard to verify, and you'd have to do it manually with a VM or some different build environment. This is interesting: https://github.com/rust-lang/rust/pull/33296 https://github.com/rust-lang/rust/pull/33296
shadowmint 10y ago
Libraries can be compiled as static libraries or dynamic libraries. even static libraries might want to do some dynamic linking (for example, against the system version of openssl). haha... I like cargo a lot, but I feel this is glossing over the biggest issue building rust code has: Building and linking to native libraries. Similarly, applications are often built for different architectures, operating systems, or even operating system version... Compiling winapi v0.2.6 Compiling libc v0.2.10 Oh? I'd love to see that compiling on a different operating system. Linking against the system version of openssl? Are you sure it's the right version of openssl not to break your binding? The C ecosystem is fundamentally problematic to get repeatable cross platform builds with; and trying to get repeatable cross platform builds with rust when it touches any c / c++ library is quite a pain. Piston is a good example of how this is done... but people still struggle to compile it; the 'uses the compiler to explicitly build the dependency as a static library and links it' (eg. git binding afaik) is a much better solution, but it massively increases build times. It's a rough edge point; and I still feel like there's been no real progress towards solving the issues. Everyone just does something completely different in their native build (build.rs) step, and then your 'repeatable build' is largely, hit 'cargo build' and 'hope nothing goes wrong and all dependencies are installed'. There are other more important priorities for rust at the moment (like recovering from panics...), but this is certainly an area I'd love to see improvement.
- dbaupp 10y ago
  > Oh? I'd love to see that compiling on a different operating system. You'll note from the file paths in the blog post ("file:///Users/ykatz/...") that the builds are actually being run on a OSX machine: winapi is careful to not break builds when it is irrelevant. It is possible and even necessary to write code that depends on operating system details (e.g. to implement the high-level cross-platform code), and indeed that code may not compile on different platforms. The point is that rustc and cargo together provide the tools to make it possible and easy to write code that builds and runs everywhere everywhere, they don't/can't/shouldn't guarantee it.
- eyko 10y ago
  As far as I can tell, the article isn't claiming that cross compilation in rust is easy, and it's definitely not saying that you can easily dynamically link and cross compile simultaneously. I develop on OSX but deploy to Linux, and that has always been an issue for me in Rust, compared to Go for example. The easiest approach for me so far has been building on the target platform with vagrant/virtualbox and calling it a day.
dschiptsov 10y ago
So, dependencies in, say, cabal or go or CL are unpredictable? The trend to put meaningless but catchy adjectives, like "safe" and "predictable", as if everything else is opposite, is misleading. Designers of classic languages were not some arrogant punks (they were brilliant, like David Moon and other guys of his generation), and the most of findamental problems were well understood in times of Standard ML or Common Lisp. So, in comparison to C or C++ it might be "predictable" and even more "safe", but in comparison with well-researched classic languages, these mere redundant adjectives.
- dikaiosune 10y ago
  Rust isn't targeting all of the same use areas as the other languages you mentioned (and FWIW I've heard not great things about go dependency mgt). Many, if not most, of the rust materials in the wild are pitching it as an alternative to C and C++, because that's where the language has the clearest competitive advantage.
- dbaupp 10y ago
  By all reports ("cabal hell"), Cabal is unpredictable, and Go's dependency management problems prompted countless external tools until the recent vendoring work: with the default `go get`, there is no guarantee that a build will be the same from day to day, as the best one can do is point at a given branch of a repository (a moving target), not tags or commits. (I have no idea on CL's dependency management situation, maybe it is even better than Rust's, in which case it would be amazing.)
  - f2f 10y ago
    minor factual adjustment: "go get" doesn't update a package once it's been downloaded. you need to force it. "there is no guarantee that a build will be the same from day to day" is simply not true.
    - dbaupp 10y ago
      Oh, indeed. I must've got mixed up: it doesn't guarantee that builds will the same from machine to machine (unless you run go get at the exact same instant).
yazaddaruvala 10y ago
My Cargo wish list! 1. Like every enterprise build system I've ever gotten to use, I wish Cargo would manage the compiler version as well. 2. The compiler forces me to mark `unsafe` functions/blocks to use unsafe code. Why does Cargo not force me to mark `unsafe` dependencies to use unsafe code?[0] [0] Only dependencies that explicitly use `unsafe` should need to be marked in my Cargo.toml. eg. Iron doesn't use `unsafe`, but Mio does. Iron depends on Hyper depends on Mio. If I have a dependence on Iron, I really want to be able to mark `#[unsafe_deep_dep] mio`, or else my build fails.
- steveklabnik 10y ago
  At the fundamental level, the machine is unsafe. This means that if unsafe were transitive, all code would need to be marked unsafe. Use a String? Your crate needs to be marked unsafe.
  - yazaddaruvala 10y ago
    We agree. I think there was a bit of misunderstanding. I don't think transitive unsafe is appropriate. Only crates which use `unsafe` explicitly should need to be whitelisted. Fun Example: I need to depend on Iron. Iron boasts in its README that it doesn't use `unsafe`. Cool. However, Iron depends on Hyper. Hyper is complicated, it needs to use `unsafe`, I can understand that. Hyper is also used by a lot of people, I trust it, and whitelist Hyper. My build passes, I know only `std` and Hyper use `unsafe`. Why is this helpful? Hasn't happened yet but @reem, being human, adds an `unsafe`[0] and lets Iron's README go out of date. It happens. However, as a user, I don't like to trust documentation, and I don't like to trust other developers[1]. When I update to this version of Iron, I'd rather my builds now failed. Then I can decide if this escalation of responsibility is appropriate. This is especially true in an enterprise context where teams of new hires do whatever it takes to push features, and "accidentally" push some really unsafe code to a deep dependency[2]. [0] Lets pretend this `unsafe` was actually used in production code: https://github.com/iron/iron/commit/ba4d197030067d4347134c2e0897ef2fb4545353 https://github.com/iron/iron/commit/ba4d197030067d4347134c2e... [1] Hence my other preferences for type checking, core reviews, and tests. [2] Not everything is manually catchable in code reviews. Sometimes its 3am. Sometimes whoever it was, was really new and the CR is 14 pages long and needs to be deployed Thursday! Sometimes a reviewer is just having an off-day. It happens.
cetra3 10y ago
This article purports that Servo relies solely on Cargo for the build step. If you look at the build step from github it looks like it's relying on mach also: https://github.com/servo/servo#building https://github.com/servo/servo#building Shouldn't it instead be using cargo purely?
- steveklabnik 10y ago
  Servo relies on a specific version of Rust, and Mach helps with that. Cargo isn't a general build system, it builds Rust code. I use Make to drive Cargo for my OS project, because I use nasm to build the asm and Cargo to build the Rust, for example.
- SimonSapin 10y ago
  Mach is what you interact with when doing stuff in Servo, but all the build-* sub-commands end up calling Cargo, which is what does the heavy lifting. The typical `./mach build` command will download known-good versions of Rust Nightly and Cargo (unless that’s already done), call `cargo build` with some flags, then show a system notification when it’s done. That’s it.
indatawetrust 10y ago
the subject will be on the outside but what are examples of applications developed with rust?
- steveklabnik 10y ago
  There's a lot of OSS stuff as well, (well, "a lot" for how old Rust is) but we just recently added https://www.rust-lang.org/friends.html https://www.rust-lang.org/friends.html to the website to showcase production users.
Perseids 10y ago
Something which is missing entirely in this concept (or, hopefully, only in the blog entry), are security updates. For example, let's assume I use an HTTP library whose TLS dependency has an error where it doesn't properly match the domain name in the certificate to the hostname of the server I'm connecting to. My Cargo.lock file references a TLS library of version 1.7.1, but the security patch is applied on 1.7.6 which has some breaking changes. Thus, the security update cannot happen silently (which would violate the predictability property of cargo anyway). Instead, we need a command to selectively update only packages with security updates and respectively we need a way to mark version bumps as security critical.

211 points by aturon 10y ago

I like many things about Rust, but Cargo alone is what initially got me started with it. I was in the process of setting up a fresh C++ project with vendored dependencies, and it was just an absolute nightmare. In contrast, it took me about 5 minutes to get a Rust project set up with comparable dependency complexity.

akhilcacharya 10y ago
Speaking of which, is there any good package management system for modern C/C++?
- dikaiosune 10y ago
  I've had biicode recommended to me (https://github.com/biicode/biicode https://github.com/biicode/biicode), but I haven't tried it yet. EDIT: Also worth noting that a common response I've heard to "I need a better manager for my dependencies" is "you have a system package manager for a reason."
  - McP 10y ago
    What do they say when you reply "Windows"?
    - dikaiosune 10y ago
      I think it goes without saying that those with that sort of response tend not to use Windows, and also often seem to assume that a perfectly in-order Linux/Unix install is the only way to build software.
      - FreeFull 10y ago
        And also that you're fine with creating a package for any dependency that doesn't have one.
        dikaiosune 10y ago
        Creating, maintaining, shepherding through distro processes, etc. All so you can then eventually build your own code.
- McP 10y ago
  Conan seems to have appeared recently but I haven't yet tried it myself. https://www.conan.io/ https://www.conan.io/
- rsaarelm 10y ago
  The Meson build system http://mesonbuild.com/ http://mesonbuild.com/ seems to have a package manager like dependency management system.
  - imron 10y ago
    It's also very quick and easy to get projects up and running. For *nix based platforms Meson is now my go to build manager for c++ projects.
- ingve 10y ago
  Conan is the most promising one, they just released a new version: http://blog.conan.io/2016/05/03/New-conan-release-0-9.html http://blog.conan.io/2016/05/03/New-conan-release-0-9.html Arne Mertz has a Vagrant box for people who want to experiment with a "4C" development environment (Clang, Cmake, Conan, Clion). https://github.com/arnemertz/Xubuntu1604_DevBox https://github.com/arnemertz/Xubuntu1604_DevBox
- boris 10y ago
  Check out https://build2.org https://build2.org
- fungos 10y ago
  Even if it doesn't has too much support, I highly recommend fips: https://github.com/floooh/fips https://github.com/floooh/fips I have been using it, adding some libs and even contributing some changes. You should really give it a try!
deleted 10y ago
[deleted]
qznc 10y ago
I understand that old languages like C++ or Java have problems providing a single package manager. It seems that newer languages have roughly equivalent services. Are there any big differences between Cargo and Python/Go/D/Ruby/Javascript/etc?
- steveklabnik 10y ago
  It depends on how you define "big". Cargo is closest to Bundler + rubygems, but with some aspects of npm.
- tatterdemalion 10y ago
  I find cargo easier to use than bundler or npm. Specifically I never deal with "environment hell" where some config hasn't been picked up and I'm pulling in the wrong version of a dependency or the language or whatever. This is partly because of the way cargo is designed, but its also helped a lot by the fact that Rust generates statically linked binaries in one build instead of being a continuous interpreter process.
- haddr 10y ago
  I wouldn't say Java has this problem. Look at Maven for instance. Then look back at Crates, you will notice how many common solutions they share. On the other hand I see that Crates solves the problem of dependency when 2 different version of the same library are used across the project. Maven doesn't solve that and it's painful. However this seems a bigger problem that just a package manager I suppose.
- mavelikara 10y ago
  Java has Maven Repository. This can be use from within Ant (Ivy), Maven, Gradle, sbt etc. Both Ruby and Python are almost as old, if not older than Java. JavaScript is only slightly younger.
- riffraff 10y ago
  notice that some of the really old ones (perl, tex) have had package managers for a really, really long time. I think linux (*nix?) vs non-linux heritage might have something to do with it.

SimonSapin 10y ago

It was a happy day when Servo ditched a big hairy pile of makefiles + git submodules and switched to Cargo :)

Gankro 10y ago
Happier than Rust 1.0 and the end of The Rustup?
- cm3 10y ago
  End of The Rustup? AFAICS, it's still downloading a very specific servo-only rustc.
  - kibwen 10y ago
    Servo's efforts to upgrade their version of rustc used to be legendarily traumatic, these days it's comparatively tame. :)
    - cm3 10y ago
      Right, and if the binary toolchains would be built with musl (not targeting musl) statically, we could have a rustc+cargo that works on both glibc and musl, avoiding two versions.
      - dbaupp 10y ago
        Could you explain what you're referring to? It seems like a total non sequitur, since as far as I know, there has never been any pain (or even rumour of pain) in a Rust upgrade caused by what libc the compiler uses, but rather breakage in the unstable features that servo uses.
        cm3 10y ago
        If the official rust+cargo was built statically with musl, then a single download would work on glibc and musl distros. Right now, we don't have a download that works on musl, and one has to bootstrap on a glibc system, targeting musl, which makes it impossible to bootstrap rust+cargo on a musl distro without glibc because bootstrapping requires a snapshot.

gue5t 10y ago

Cargo assumes you want to cache downloaded/built dependencies at the granularity of a unix user account. It's very easy to break things if you try to force current cargo cache dependencies at the per-project level and there's been no interest in caching things at a per-machine or shared-between-machines level. If duplicating compilation work is desirable to ensure some amount of noninterference, it should be supported to have a package cache per project. If we can treat Cargo as a pure function from inputs to outputs, we should be sharing these build results across the Internet, at least within the set of machines used by one person. Cargo also refuses to follow the XDG directory specification for its per-unix-user configuration; it should put things in $XDG_CONFIG_HOME and $XDG_CACHE_HOME, but instead dumps both configuration and downloaded/compiled stuff in ~/.cargo.

wyldfire 10y ago
> it should put things in $XDG_CONFIG_HOME and $XDG_CACHE_HOME I was going to submit an issue but one already exists [1][2] [1] https://github.com/rust-lang/cargo/pull/148 https://github.com/rust-lang/cargo/pull/148 [2] https://github.com/rust-lang/cargo/issues/1734 https://github.com/rust-lang/cargo/issues/1734
yberreby 10y ago
> If we can treat Cargo as a pure function from inputs to outputs, we should be sharing these build results across the Internet, at least within the set of machines used by one person. AFAIK, all it takes is a single `build.rs` to make rustc, and therefore Cargo, an impure 'function'. I'm not sure about compiler plugins, but I expect them to behave the same way.
dikaiosune 10y ago
There's an ongoing discussion of using build caches with Cargo: https://internals.rust-lang.org/t/is-a-shared-build-cache-a-good-fit-for-cargo https://internals.rust-lang.org/t/is-a-shared-build-cache-a-... TLDR: There's interest, but some wrinkles need to be sorted out.
- cm3 10y ago

dastbe 10y ago

How does Cargo handle indirect dependency visibility? This article talks about visibility in terms of "do I need to manually include indirect dependencies? No!" but not in terms of "can I accidentally write code against an indirect dependency?" which isn't sufficiently answered. If the answer is that indirect dependencies are still visible, I'd be interested in knowing if rust-lang/cargo plan to change that, similar to JDK9's module system for re-export.

SimonSapin 10y ago
Short answer: `extern crate foo;` doesn’t work if `foo` is an indirect dependency. So you cannot accidentally use something without declaring it in `Cargo.toml`. Longer answer: when running rustc, Cargo passes individual `--extern bar=/path/to/bar.rlib` options for each direct dependency, not just a path to a directory with everything. You can see the exact command with `cargo build --verbose` or `cargo build -v`.
- deleted 10y ago
  [deleted]
tatterdemalion 10y ago
As the sibling answer says, they are not visible. However, crates can re-export symbols they have imported from their dependency, which couples their semver to that dependency's in a way that people have often not realized. This has been the source of some upgrade headaches so far, but its less of a problem than it has been with languages in my experience.
keeperofdakeys 10y ago
It's also relevant that rust can use multiple direct/indirect dependencies of different versions, due to the way that symbols are exported.

eridius 10y ago

Something that I'm surprised this page doesn't talk about, and which is very important considering the recent hubbub over left-pad, is that any dependency you get on Cargo can be relied upon to continue to exist forever (well, as long as the crates.io site still exists, but if that goes away so does the Cargo index). The reason is because you can't ever remove a published version of your crate from crates.io. You can yank a version, which tells Cargo not to allow any projects to form new dependencies on that version, but the version isn't actually deleted and any projects that have existing dependencies on that version will continue to be allowed to use it. This is documented at http://doc.crates.io/crates-io.html#cargo-yank http://doc.crates.io/crates-io.html#cargo-yank.

waffle_ss 10y ago
> You can yank a version, which tells Cargo not to allow any projects to form new dependencies on that version, but the version isn't actually deleted That's how RubyGems used to work, except you could contact the support team and ask them to permanently delete your gem version if something really sensitive and irrevocable was put into it. They had to change that due to their support log getting too big: http://blog.rubygems.org/2015/04/13/permadelete-on-yank.html http://blog.rubygems.org/2015/04/13/permadelete-on-yank.html
- kibwen 10y ago
  And of course, a centralized site can still be compelled to remove a package via legal action if push comes to shove.
- eridius 10y ago
  Cargo's policy is that if you upload secrets, you need to change the secrets because the code can't be deleted. From the aforelinked page: > A yank does not delete any code. This feature is not intended for deleting accidentally uploaded secrets, for example. If that happens, you must reset those secrets immediately.
  - waffle_ss 10y ago

Siecje 10y ago

How does Cargo manage the case when dependencies X and Y both require a different version of dependency Z?

steveklabnik 10y ago
It attempts to unify them if it can, but if they cannot be unified, it will include both versions.
- cm3 10y ago
  Include both how? Make renamed copies to have two and modify the code to use the new names?
  - steveklabnik 10y ago
    It doesn't have to do any renaming or modifying of names. Rust has a module system, so each dependency will use the version they need. The only time it doesn't Just Work is if one of those crates re-exports a type from the sub-crate in a public manner, and then you try to call something from the other crate with a value of that type. You'll get a compile-time error about mismatched types.
    - cm3 10y ago
      But it sounds like that's what's going on. Using foo-1.1 in bar.rs and foo-2.2 in rabbit.rs.
      - steveklabnik 10y ago
        Sure. That will Just Work in every case _except_ if bar and rabbit both re-export a type from foo, and you try to use them together. If they don't re-export anything, then it all works just fine. No renaming or transformations needed.
  - kibwen 10y ago
    Rust mangles symbols with version information, so it's no trouble to have two versions of the same library in the same binary if necessary.

cm3 10y ago

I see there is `cargo install` which will fetch and build an executable from crates.io, is there no `cargo update` for `install`?

steveklabnik 10y ago
https://github.com/rust-lang/cargo/pull/2405 https://github.com/rust-lang/cargo/pull/2405 landed, but https://github.com/rust-lang/cargo/issues/2082 https://github.com/rust-lang/cargo/issues/2082 is similar too and is still open.
- cm3 10y ago
  Interesting, but I was thinking more of something like "update installed binaries with latest version from index". This doesn't look like it.
  - steveklabnik 10y ago
    Ah yeah, it's per-thing, not every thing you've installed, currently. cargo install --list does list all the things you've installed, so it wouldn't be too big of a deal to add; the pieces are all there.

xkarga00 10y ago

I have been working as a Go programmer for the past two years and I really love the language but dependency management has been one of the biggest pains for me. Godep really sucks (that's what we use in my current gig - we plan on switching to Glide, still haven't played around with it) and I am not sure I like the fact that I need to go out in the wild and choose between the rest of the existing solutions. I would prefer to have something that works ootb. The Rust team really got this right and Cargo is amazing. One of the main reason I have been dabbling with Rust lately.

dalailambda 10y ago
In my experience Glide is the closest to a modern package manager for Go.

cm3 10y ago

It's great that Raph's line-breaking code has been reused, but in contrast to Mozilla projects any contribution to it will now require assigning copyright to Google (like Ubuntu or Microsoft's CLA require as well). At least it's a tiny piece, so may not hurt too much in terms of prevented contributions.

gpm 10y ago
Or just forking it, it's released under a open source license. This wouldn't be the first time Servo has forked a dependency, for example they maintain a fork of glutin (library that handles opening windows) because they want certain features that upstream doesn't. https://github.com/servo/glutin https://github.com/servo/glutin
- cm3 10y ago
  That can work. Curious what features servo/glutin has that are exclusive?

conradev 10y ago

I appreciate that predictability is a big priority with Cargo. I feel like Rust and Cargo are in a great position to deterministically (per compiler version, per set of build flags) build libraries. That would be an amazing step forward in security if it could be enforced. Does anyone know if this is planned? https://reproducible-builds.org https://reproducible-builds.org

steveklabnik 10y ago
In general, we are very interested in reproducible builds, and have fixed bugs where accidental non-determinism has crept in. However, it can be easy to reintroduce with build.rs, which allows for executing arbitrary Rust code before a build. Syntax extensions are another problem here. But the scope is much reduced, it's true.
- conradev 10y ago
  That's awesome to hear. Would it be possible to make an attribute like #[deny(non_deterministic)] which would error if `build.rs` or similar is present? Also, for usage of things like the `file!` macro, which might mess up determinism based on the build directory. Theoretically, syntax extensions could be written to be deterministic as well (for a given version, of course). Is there a tracking issue on this currently?
  - steveklabnik 10y ago
    There isn't, it might be an interesting idea! Not allowing build.rs would eliminate a lot of crates, and not all build.rs' are nondeterministic... So that's tough.
    - conradev 10y ago
      Yeah. Perhaps developers could attribute functions in `build.rs` specifically with "deterministic", so you could see what in the dependency graph isn't deterministic. Unfortunately, unlike with borrowck, reproducibility is inherently hard to verify, and you'd have to do it manually with a VM or some different build environment. This is interesting: https://github.com/rust-lang/rust/pull/33296 https://github.com/rust-lang/rust/pull/33296

shadowmint 10y ago

Libraries can be compiled as static libraries or dynamic libraries. even static libraries might want to do some dynamic linking (for example, against the system version of openssl). haha... I like cargo a lot, but I feel this is glossing over the biggest issue building rust code has: Building and linking to native libraries. Similarly, applications are often built for different architectures, operating systems, or even operating system version... Compiling winapi v0.2.6 Compiling libc v0.2.10 Oh? I'd love to see that compiling on a different operating system. Linking against the system version of openssl? Are you sure it's the right version of openssl not to break your binding? The C ecosystem is fundamentally problematic to get repeatable cross platform builds with; and trying to get repeatable cross platform builds with rust when it touches any c / c++ library is quite a pain. Piston is a good example of how this is done... but people still struggle to compile it; the 'uses the compiler to explicitly build the dependency as a static library and links it' (eg. git binding afaik) is a much better solution, but it massively increases build times. It's a rough edge point; and I still feel like there's been no real progress towards solving the issues. Everyone just does something completely different in their native build (build.rs) step, and then your 'repeatable build' is largely, hit 'cargo build' and 'hope nothing goes wrong and all dependencies are installed'. There are other more important priorities for rust at the moment (like recovering from panics...), but this is certainly an area I'd love to see improvement.

dbaupp 10y ago
> Oh? I'd love to see that compiling on a different operating system. You'll note from the file paths in the blog post ("file:///Users/ykatz/...") that the builds are actually being run on a OSX machine: winapi is careful to not break builds when it is irrelevant. It is possible and even necessary to write code that depends on operating system details (e.g. to implement the high-level cross-platform code), and indeed that code may not compile on different platforms. The point is that rustc and cargo together provide the tools to make it possible and easy to write code that builds and runs everywhere everywhere, they don't/can't/shouldn't guarantee it.
eyko 10y ago
As far as I can tell, the article isn't claiming that cross compilation in rust is easy, and it's definitely not saying that you can easily dynamically link and cross compile simultaneously. I develop on OSX but deploy to Linux, and that has always been an issue for me in Rust, compared to Go for example. The easiest approach for me so far has been building on the target platform with vagrant/virtualbox and calling it a day.

dschiptsov 10y ago

So, dependencies in, say, cabal or go or CL are unpredictable? The trend to put meaningless but catchy adjectives, like "safe" and "predictable", as if everything else is opposite, is misleading. Designers of classic languages were not some arrogant punks (they were brilliant, like David Moon and other guys of his generation), and the most of findamental problems were well understood in times of Standard ML or Common Lisp. So, in comparison to C or C++ it might be "predictable" and even more "safe", but in comparison with well-researched classic languages, these mere redundant adjectives.

dikaiosune 10y ago
Rust isn't targeting all of the same use areas as the other languages you mentioned (and FWIW I've heard not great things about go dependency mgt). Many, if not most, of the rust materials in the wild are pitching it as an alternative to C and C++, because that's where the language has the clearest competitive advantage.
dbaupp 10y ago
By all reports ("cabal hell"), Cabal is unpredictable, and Go's dependency management problems prompted countless external tools until the recent vendoring work: with the default `go get`, there is no guarantee that a build will be the same from day to day, as the best one can do is point at a given branch of a repository (a moving target), not tags or commits. (I have no idea on CL's dependency management situation, maybe it is even better than Rust's, in which case it would be amazing.)
- f2f 10y ago
  minor factual adjustment: "go get" doesn't update a package once it's been downloaded. you need to force it. "there is no guarantee that a build will be the same from day to day" is simply not true.
  - dbaupp 10y ago
    Oh, indeed. I must've got mixed up: it doesn't guarantee that builds will the same from machine to machine (unless you run go get at the exact same instant).

yazaddaruvala 10y ago

My Cargo wish list! 1. Like every enterprise build system I've ever gotten to use, I wish Cargo would manage the compiler version as well. 2. The compiler forces me to mark `unsafe` functions/blocks to use unsafe code. Why does Cargo not force me to mark `unsafe` dependencies to use unsafe code?[0] [0] Only dependencies that explicitly use `unsafe` should need to be marked in my Cargo.toml. eg. Iron doesn't use `unsafe`, but Mio does. Iron depends on Hyper depends on Mio. If I have a dependence on Iron, I really want to be able to mark `#[unsafe_deep_dep] mio`, or else my build fails.

steveklabnik 10y ago
At the fundamental level, the machine is unsafe. This means that if unsafe were transitive, all code would need to be marked unsafe. Use a String? Your crate needs to be marked unsafe.
- yazaddaruvala 10y ago
  We agree. I think there was a bit of misunderstanding. I don't think transitive unsafe is appropriate. Only crates which use `unsafe` explicitly should need to be whitelisted. Fun Example: I need to depend on Iron. Iron boasts in its README that it doesn't use `unsafe`. Cool. However, Iron depends on Hyper. Hyper is complicated, it needs to use `unsafe`, I can understand that. Hyper is also used by a lot of people, I trust it, and whitelist Hyper. My build passes, I know only `std` and Hyper use `unsafe`. Why is this helpful? Hasn't happened yet but @reem, being human, adds an `unsafe`[0] and lets Iron's README go out of date. It happens. However, as a user, I don't like to trust documentation, and I don't like to trust other developers[1]. When I update to this version of Iron, I'd rather my builds now failed. Then I can decide if this escalation of responsibility is appropriate. This is especially true in an enterprise context where teams of new hires do whatever it takes to push features, and "accidentally" push some really unsafe code to a deep dependency[2]. [0] Lets pretend this `unsafe` was actually used in production code: https://github.com/iron/iron/commit/ba4d197030067d4347134c2e0897ef2fb4545353 https://github.com/iron/iron/commit/ba4d197030067d4347134c2e... [1] Hence my other preferences for type checking, core reviews, and tests. [2] Not everything is manually catchable in code reviews. Sometimes its 3am. Sometimes whoever it was, was really new and the CR is 14 pages long and needs to be deployed Thursday! Sometimes a reviewer is just having an off-day. It happens.

cetra3 10y ago

This article purports that Servo relies solely on Cargo for the build step. If you look at the build step from github it looks like it's relying on mach also: https://github.com/servo/servo#building https://github.com/servo/servo#building Shouldn't it instead be using cargo purely?

steveklabnik 10y ago
Servo relies on a specific version of Rust, and Mach helps with that. Cargo isn't a general build system, it builds Rust code. I use Make to drive Cargo for my OS project, because I use nasm to build the asm and Cargo to build the Rust, for example.
SimonSapin 10y ago
Mach is what you interact with when doing stuff in Servo, but all the build-* sub-commands end up calling Cargo, which is what does the heavy lifting. The typical `./mach build` command will download known-good versions of Rust Nightly and Cargo (unless that’s already done), call `cargo build` with some flags, then show a system notification when it’s done. That’s it.

indatawetrust 10y ago

the subject will be on the outside but what are examples of applications developed with rust?

steveklabnik 10y ago
There's a lot of OSS stuff as well, (well, "a lot" for how old Rust is) but we just recently added https://www.rust-lang.org/friends.html https://www.rust-lang.org/friends.html to the website to showcase production users.

Perseids 10y ago

Something which is missing entirely in this concept (or, hopefully, only in the blog entry), are security updates. For example, let's assume I use an HTTP library whose TLS dependency has an error where it doesn't properly match the domain name in the certificate to the hostname of the server I'm connecting to. My Cargo.lock file references a TLS library of version 1.7.1, but the security patch is applied on 1.7.6 which has some breaking changes. Thus, the security update cannot happen silently (which would violate the predictability property of cargo anyway). Instead, we need a command to selectively update only packages with security updates and respectively we need a way to mark version bumps as security critical.