6 ms·

I often see initiatives and articles like this but no mention of Nix. Is it just not well known enough for comparison? Because to me that’s the standard.

esseph 1y ago
It's an article about Fedora, specifically.
- binarymax 1y ago
  Yes, I know that. But when talking about reproducible packages, we can and should learn from existing techniques. Is Fedoras system capable of this goal? Is it built the right way? Should it adopt an alternate package manager to achieve this with less headache?
  - dogleash 1y ago
    "What about Nix tho?" is the "Rewrite it in Rust" of reproducible builds.
  - deleted 1y ago
    [deleted]
  - johnny22 1y ago
    No other currently distro whether it be debian, fedora, or arch, or opensuse seems to want to switch up their design to match nix's approach. They have too many people familiar with the current approaches.
    - nonameiguess 1y ago
      That's not even really true. Hacker News readers seem to fixate heavily on the most popular desktop versions of Linux distros, but that is far from all they make. SUSE with SLE Micro and Edge Image Builder is a declarative build system for creating your own installer images. If you don't want SLE Micro, they even give you the Elemental Toolkit, which can be used to build your own custom distro in a declarative manner. I don't know much about Debian and Red Hat's approaches to this, but I know they have similar tooling. The "problem," if you want to call it that, is developers, if I may be uncharitable for a brief moment here, are amazingly solipsistic. The major Linux vendors are targeting enterprise edge computing use cases, whereas developers only ever think about the endpoint PCs they themselves use for development. Plenty of distro tooling will give you declaratively built, transactionally updated, immutable server images, but if they don't give you the same for your personal laptop, those efforts are invisible to the modal Hacker News commenter. For what it's worth, there's also Guix, which is literally a clone of Nix but part of the GNU project, so it only uses free software and opts for Guile instead of a custom DSL for configuration. It wasn't a pre-existing distro that changed, of course.
      - johnny22 1y ago
        Guix counts, but what suse is doing does not. It is still based on rpm packages underneath last i checked.
djha-skin 1y ago
It's very, very complicated. It's so far past the maximum effort line of most linux users as to be in its own class of tools. Reproducibility in the imperative package space is worth a lot. Lots of other tools are built on RPM/DEB packages that offer similar advantages of Nix -- Ansible, for one. This is more of a "rising tide raises all boats" situation.
steeleduncan 1y ago
I use Nix extensively, but the Nix daemon doesn't do much of use that can't be achieved by building your code from a fixed OCI container with internet turned off. The latter is certainly more standard across the industry, and sadly a lot easier too. Nix is not a revolutionary containerisation technology, nor honestly a very good one. The value in Nix comes from the package set, nixpkgs. What is revolutionary is how nixpgks builds a Linux distribution declaratively, and reproducibly, from source through purely functional expressions. However, nixpkgs is almost an entire universe unto itself, and it is generally incompatible with the way any other distribution would handle things, so it would be no use to Fedora, Debian, and others
- fpoling 1y ago
  At work we went back to a Docker build to make reproducible images. The primary reason is poor cross-compilation support in Nix on Arm when developers needed to compile for an amd64 service and derive image checksums that are put into tooling that are run locally for service version verification and reproducibility. With Docker it turned out relatively straightforward. With Nix even when it runs in Linux Arm VM we tried but just gave up.
  - binarymax 1y ago
    Funny, I had that experience with Docker - mostly due to c++ dependencies - that were fine in Nix
lima 1y ago
Contrary to popular opinion, Nix builds aren't reproducible: https://luj.fr/blog/is-nixos-truly-reproducible.html https://luj.fr/blog/is-nixos-truly-reproducible.html
- pxc 1y ago
  That's such a weird characterization of this article, which (in contrast to other writing on this subject) clearly concludes (a) that Nix achieves a very high degree of reproducibility and is continuously improving in this respect, and (b) Nix is moreover reproducible in a way that most other distros (even distros that do well in some measures of bitwise reproducibility) are not (namely, time traveling— being able to reproduce builds in different environments, even months or years later, because the build environment itself is more reproducible). The article you linked is very clear that both qualitatively and quantitatively, NixOS has made achieved high degrees of reproducibility, and even explicitly rejects the possibility of assessing absolute reproducibility. NixOS may not be the absolute leader here (that's probably stagex, or GuixSD if you limit yourself to more practical distros with large package collections), but it is indeed very good. Did you mean to link to a different article?
  - looofooo0 1y ago
    What does high degree mean? Either you achieve bit for bit reproducibility of your builds or not.
    - kfajdsl 1y ago
      If 99% of the packages in your repos are bit-for-bit reproducible, you can consider your distro to have a high degree of reproducibility.
      - deleted 1y ago
        [deleted]
      - pxc 1y ago
        And with even a little bit of imagination, it's easy to think of other possible measures of degrees of reproducibility, e.g.: • % of deployed systems which consist only of reproducibly built packages • % of commonly downloaded disk images (install media, live media, VM images, etc.) consist only of reproducibly built packages • total # of reproducibly built packages available • comparative measures of what NixOS is doing right like: of packages that are reproducibly built in some distros but not others, how many are built reproducibly in NixOS • binary bootstrap size (smaller is better, obviously) It's really not difficult to think of meaningful ways that reproducibility of different distros might be compared, even quantitatively.
12345hn6789 1y ago
Nix is to Linux users what Linux is to normies.
__MatrixMan__ 1y ago
In the near term it makes more sense to position nix as a common interface between app developers and distro maintainers and not as a direct-to-user way to cut their distro maintainers out of the loop entirely (although it is quite useful for that). Ideally, a distro maintainer would come across a project packaged with nix and think: > Oh good, the app dev has taken extra steps to make life easy for me. As-is, I don't think that's the case. You can add a flake output to your project which builds an .rpm or a .deb file, but it's not commonly done. I'm guessing that most of the time, distro maintainers would instead hook directly into a language specific build-tool like cmake or cargo and ignore the nix stuff. They benefit from nix only indirectly in cases where it has prevented the app dev from doing crazy things in their build (or at least has made that crazyness explicit, versus some kind of works-on-my-machine accident or some kind of nothing-to-see here skulduggery). If we want to nixify the world I think we should focus less on talking people out of using package managers which they like and more on making the underlying packages more uniform.
skrtskrt 1y ago
Because Nix is a huge pain ramp up on and to use for anyone who is not an enthusiast about the state of their computer. What will happen is concepts from Nix will slowly get absorbed into other, more user-friendly tooling while Nix circles the complexity drain
diffeomorphism 1y ago
Different notions of reproducible. This project cares specifically about bit-for-bit identical builds (e.g. no time stamps, parallel compile artifacts etc). Nix is more about being declarative and "repeatable" or whatever a good name for that would be. Both notions are useful for different purposes and nix is not particularly good at the first one. https://reproducible-builds.org/citests/ https://reproducible-builds.org/citests/
jzb 1y ago
Oh, I assure you, it's hard to escape knowing about Nix if you write about this sort of thing. Someone will be along almost immediately to inform you about it. Nix wasn't mentioned (I'm the author) because it really isn't relevant here -- the comparable distributions, when discussing what Fedora is doing, are Debian and other distributions that use similar packaging schemes and such.
- __MatrixMan__ 1y ago
  I agree that NixOS/nixpkgs would not be a good a basis for comparison. Do you have an opinion about the use of nix by app devs to specify their builds, i.e. as a make alternative, not as a Fedora alternative? Quoting the article: > Irreproducible bits in packages are quite often "caused by an error or sloppiness in the code". For example, dependence on hardware architecture in architecture-independent (noarch) packages is "almost always unwanted and/or a bug", and reproducibility tests can uncover those bugs. This is the sort of thing that nix is good at guarding against, and it's convenient that it doesn't require users to engage with the underlying toolchain if they're unfamiliar with it. For instance I can use the command below to build helix at a certain commit without even knowing that it's a rust package. Although it doesn't guarantee all aspects of repeatability, it will fail if the build depends on any bits for which a hash is not known ahead of time, which gets you half way there I think. nix build github:helix-editor/helix/340934db92aea902a61b9f79b9e6f4bd15111044 Used in this way, can nix help Fedora's reproducibility efforts? Or does it appear to Fedora as a superfluous layer to be stripped away so that they can plug into cargo more directly?
  - microtonal 1y ago
    Used in this way, can nix help Fedora's reproducibility efforts? Or does it appear to Fedora as a superfluous layer to be stripped away so that they can plug into cargo more directly? A lot of Nix-based package builds will burn Nix store paths directly into the binary. If you are lucky it's only the rpath and you can strip it, but in some cases other Nix store paths end up in the binary. Seems pretty useless to Fedora. Besides many of the difficult issues are not solved by Nix either. (E.g. build non-determinism by ordering differences due to the use of a hashmap somewhere in the build.)

esseph 1y ago

It's an article about Fedora, specifically.

binarymax 1y ago
Yes, I know that. But when talking about reproducible packages, we can and should learn from existing techniques. Is Fedoras system capable of this goal? Is it built the right way? Should it adopt an alternate package manager to achieve this with less headache?
- dogleash 1y ago
  "What about Nix tho?" is the "Rewrite it in Rust" of reproducible builds.
- deleted 1y ago
  [deleted]
- johnny22 1y ago
  No other currently distro whether it be debian, fedora, or arch, or opensuse seems to want to switch up their design to match nix's approach. They have too many people familiar with the current approaches.
  - nonameiguess 1y ago
    That's not even really true. Hacker News readers seem to fixate heavily on the most popular desktop versions of Linux distros, but that is far from all they make. SUSE with SLE Micro and Edge Image Builder is a declarative build system for creating your own installer images. If you don't want SLE Micro, they even give you the Elemental Toolkit, which can be used to build your own custom distro in a declarative manner. I don't know much about Debian and Red Hat's approaches to this, but I know they have similar tooling. The "problem," if you want to call it that, is developers, if I may be uncharitable for a brief moment here, are amazingly solipsistic. The major Linux vendors are targeting enterprise edge computing use cases, whereas developers only ever think about the endpoint PCs they themselves use for development. Plenty of distro tooling will give you declaratively built, transactionally updated, immutable server images, but if they don't give you the same for your personal laptop, those efforts are invisible to the modal Hacker News commenter. For what it's worth, there's also Guix, which is literally a clone of Nix but part of the GNU project, so it only uses free software and opts for Guile instead of a custom DSL for configuration. It wasn't a pre-existing distro that changed, of course.
    - johnny22 1y ago
      Guix counts, but what suse is doing does not. It is still based on rpm packages underneath last i checked.

djha-skin 1y ago

It's very, very complicated. It's so far past the maximum effort line of most linux users as to be in its own class of tools. Reproducibility in the imperative package space is worth a lot. Lots of other tools are built on RPM/DEB packages that offer similar advantages of Nix -- Ansible, for one. This is more of a "rising tide raises all boats" situation.

steeleduncan 1y ago

I use Nix extensively, but the Nix daemon doesn't do much of use that can't be achieved by building your code from a fixed OCI container with internet turned off. The latter is certainly more standard across the industry, and sadly a lot easier too. Nix is not a revolutionary containerisation technology, nor honestly a very good one. The value in Nix comes from the package set, nixpkgs. What is revolutionary is how nixpgks builds a Linux distribution declaratively, and reproducibly, from source through purely functional expressions. However, nixpkgs is almost an entire universe unto itself, and it is generally incompatible with the way any other distribution would handle things, so it would be no use to Fedora, Debian, and others

fpoling 1y ago
At work we went back to a Docker build to make reproducible images. The primary reason is poor cross-compilation support in Nix on Arm when developers needed to compile for an amd64 service and derive image checksums that are put into tooling that are run locally for service version verification and reproducibility. With Docker it turned out relatively straightforward. With Nix even when it runs in Linux Arm VM we tried but just gave up.
- binarymax 1y ago
  Funny, I had that experience with Docker - mostly due to c++ dependencies - that were fine in Nix

lima 1y ago

Contrary to popular opinion, Nix builds aren't reproducible: https://luj.fr/blog/is-nixos-truly-reproducible.html https://luj.fr/blog/is-nixos-truly-reproducible.html

pxc 1y ago
That's such a weird characterization of this article, which (in contrast to other writing on this subject) clearly concludes (a) that Nix achieves a very high degree of reproducibility and is continuously improving in this respect, and (b) Nix is moreover reproducible in a way that most other distros (even distros that do well in some measures of bitwise reproducibility) are not (namely, time traveling— being able to reproduce builds in different environments, even months or years later, because the build environment itself is more reproducible). The article you linked is very clear that both qualitatively and quantitatively, NixOS has made achieved high degrees of reproducibility, and even explicitly rejects the possibility of assessing absolute reproducibility. NixOS may not be the absolute leader here (that's probably stagex, or GuixSD if you limit yourself to more practical distros with large package collections), but it is indeed very good. Did you mean to link to a different article?
- looofooo0 1y ago
  What does high degree mean? Either you achieve bit for bit reproducibility of your builds or not.
  - kfajdsl 1y ago
    If 99% of the packages in your repos are bit-for-bit reproducible, you can consider your distro to have a high degree of reproducibility.
    - deleted 1y ago
      [deleted]
    - pxc 1y ago
      And with even a little bit of imagination, it's easy to think of other possible measures of degrees of reproducibility, e.g.: • % of deployed systems which consist only of reproducibly built packages • % of commonly downloaded disk images (install media, live media, VM images, etc.) consist only of reproducibly built packages • total # of reproducibly built packages available • comparative measures of what NixOS is doing right like: of packages that are reproducibly built in some distros but not others, how many are built reproducibly in NixOS • binary bootstrap size (smaller is better, obviously) It's really not difficult to think of meaningful ways that reproducibility of different distros might be compared, even quantitatively.

12345hn6789 1y ago

Nix is to Linux users what Linux is to normies.

__MatrixMan__ 1y ago

In the near term it makes more sense to position nix as a common interface between app developers and distro maintainers and not as a direct-to-user way to cut their distro maintainers out of the loop entirely (although it is quite useful for that). Ideally, a distro maintainer would come across a project packaged with nix and think: > Oh good, the app dev has taken extra steps to make life easy for me. As-is, I don't think that's the case. You can add a flake output to your project which builds an .rpm or a .deb file, but it's not commonly done. I'm guessing that most of the time, distro maintainers would instead hook directly into a language specific build-tool like cmake or cargo and ignore the nix stuff. They benefit from nix only indirectly in cases where it has prevented the app dev from doing crazy things in their build (or at least has made that crazyness explicit, versus some kind of works-on-my-machine accident or some kind of nothing-to-see here skulduggery). If we want to nixify the world I think we should focus less on talking people out of using package managers which they like and more on making the underlying packages more uniform.

skrtskrt 1y ago

Because Nix is a huge pain ramp up on and to use for anyone who is not an enthusiast about the state of their computer. What will happen is concepts from Nix will slowly get absorbed into other, more user-friendly tooling while Nix circles the complexity drain

diffeomorphism 1y ago

Different notions of reproducible. This project cares specifically about bit-for-bit identical builds (e.g. no time stamps, parallel compile artifacts etc). Nix is more about being declarative and "repeatable" or whatever a good name for that would be. Both notions are useful for different purposes and nix is not particularly good at the first one. https://reproducible-builds.org/citests/ https://reproducible-builds.org/citests/

jzb 1y ago

Oh, I assure you, it's hard to escape knowing about Nix if you write about this sort of thing. Someone will be along almost immediately to inform you about it. Nix wasn't mentioned (I'm the author) because it really isn't relevant here -- the comparable distributions, when discussing what Fedora is doing, are Debian and other distributions that use similar packaging schemes and such.

__MatrixMan__ 1y ago
I agree that NixOS/nixpkgs would not be a good a basis for comparison. Do you have an opinion about the use of nix by app devs to specify their builds, i.e. as a make alternative, not as a Fedora alternative? Quoting the article: > Irreproducible bits in packages are quite often "caused by an error or sloppiness in the code". For example, dependence on hardware architecture in architecture-independent (noarch) packages is "almost always unwanted and/or a bug", and reproducibility tests can uncover those bugs. This is the sort of thing that nix is good at guarding against, and it's convenient that it doesn't require users to engage with the underlying toolchain if they're unfamiliar with it. For instance I can use the command below to build helix at a certain commit without even knowing that it's a rust package. Although it doesn't guarantee all aspects of repeatability, it will fail if the build depends on any bits for which a hash is not known ahead of time, which gets you half way there I think. nix build github:helix-editor/helix/340934db92aea902a61b9f79b9e6f4bd15111044 Used in this way, can nix help Fedora's reproducibility efforts? Or does it appear to Fedora as a superfluous layer to be stripped away so that they can plug into cargo more directly?
- microtonal 1y ago
  Used in this way, can nix help Fedora's reproducibility efforts? Or does it appear to Fedora as a superfluous layer to be stripped away so that they can plug into cargo more directly? A lot of Nix-based package builds will burn Nix store paths directly into the binary. If you are lucky it's only the rpath and you can strip it, but in some cases other Nix store paths end up in the binary. Seems pretty useless to Fedora. Besides many of the difficult issues are not solved by Nix either. (E.g. build non-determinism by ordering differences due to the use of a hashmap somewhere in the build.)