7 ms·

Just a thought of mine: why don’t we switch fully to git? Commit signing, tag signing, Decentralize. Doesn’t that sound like a good alternativ?

by salzig 1y ago

zdragnar 1y ago
Someone has to run the git server. Then, someone has to find the git server to pull each gem from, since not every git server is likely to be up-to-date with the each gem, or the correct version. Since these are all decentralized, each individual owner of a git server has to independently scale as more people start using each one. The benefit to being centralized is... everything is in one place. Everything scales at once. Every update is available at the same time. We did this back in the day using artifactory and co. to proxy NPM and a few other package managers as well as docker containers and some other things. No third party service going down could keep us from deploying. Not everyone does it because as a solo developer or a small team, as it feels like pointless overhead.
- salzig 1y ago
  So GitHub would be one option. Developers already discover all kind of things there. And each gem can still be provided by its “main repository”, but I don’t mind on whatever domain that repository is located. Somewhat how container images are referenced/distributed already. I think go already does it like that too. having a decentralized, and maybe sometime unavailable, infrastructure would make more people think about the problem and maybe brings us more stable solutions than we have now.
pornel 1y ago
The git protocol is more complex and harder to scale. It's especially wasteful if people are going to redownload all packages every time their amnesiac CI runs. Single-file archives are much easier to distribute. Digests and signatures have standard algorithms, not unique to git. Key/identity management is the hard part, but git doesn't solve it for you (if you don't confuse git with GitHub).
- webstrand 1y ago
  git bundles exist to solve the single-file caching and distribution problems
- salzig 1y ago
  Going crazy: we cold also adopt the container registry api for distributing gems, similar to how helm charts are also distributed nower days.
phoronixrly 1y ago
You've seen golang's package... situation?... and you still think switching to Git is a good idea?
- lr0 1y ago
  What "situation"?
  - phoronixrly 1y ago
    Let's review for example Traefik's dependency list: https://github.com/traefik/traefik/blob/master/go.mod https://github.com/traefik/traefik/blob/master/go.mod 1. Heavy dependency on Github. AKA Microsoft owns much of the golang ecosystem. Not just the source... The package distribution as well! 2. Many packages are referencing a git (short!) commit hash instead of a version. It still boggles my mind that this is an acceptable practice. Not to mention that git tags can be deleted and recreated... A pinnacle of secure package distribution practices. 3. Stuff like ambiguous imports because apparently nothing enforces proper go.mod files? They are not packages to be compiled after all, they're just repos with some conventional structure (optional)... Mind you, this is popular production-grade software... I think this is much worse than even node packages, let alone bundler and rubygems...
    - SirSavary 1y ago
      1. GitHub dominance is a social phenomenon, not a technical requirement. The go.mod file you linked references ~14 different Git hosts other than GitHub. Go's design doesn't create this centralization; it merely reflects where developers choose to host code. 2. You complain about commit hashes while simultaneously noting that tags can be deleted and recreated. Hashes are precisely the solution to mutable tags. The "short hash" concern is a red herring; Git uses sufficient entropy that collisions are not a practical concern for dependency resolution. As for "secure package distribution," go.sum files verify files verify consistent downloads. What additional security do you believe centralized registries provide? 3. Can you provide a concrete example of an ambiguous import you've encountered? I'm not familiar enough with Go to understand this criticism.
lr0 1y ago
Exactly. Go already adopts that.
dismalaf 1y ago
Well, rubygems (the software) can pull from any git repository. So we kind of have it already anyway.

Just a thought of mine: why don’t we switch fully to git? Commit signing, tag signing, Decentralize. Doesn’t that sound like a good alternativ?

by salzig 1y ago

Just a thought of mine: why don’t we switch fully to git? Commit signing, tag signing, Decentralize. Doesn’t that sound like a good alternativ?

zdragnar 1y ago

Someone has to run the git server. Then, someone has to find the git server to pull each gem from, since not every git server is likely to be up-to-date with the each gem, or the correct version. Since these are all decentralized, each individual owner of a git server has to independently scale as more people start using each one. The benefit to being centralized is... everything is in one place. Everything scales at once. Every update is available at the same time. We did this back in the day using artifactory and co. to proxy NPM and a few other package managers as well as docker containers and some other things. No third party service going down could keep us from deploying. Not everyone does it because as a solo developer or a small team, as it feels like pointless overhead.

salzig 1y ago
So GitHub would be one option. Developers already discover all kind of things there. And each gem can still be provided by its “main repository”, but I don’t mind on whatever domain that repository is located. Somewhat how container images are referenced/distributed already. I think go already does it like that too. having a decentralized, and maybe sometime unavailable, infrastructure would make more people think about the problem and maybe brings us more stable solutions than we have now.

pornel 1y ago

The git protocol is more complex and harder to scale. It's especially wasteful if people are going to redownload all packages every time their amnesiac CI runs. Single-file archives are much easier to distribute. Digests and signatures have standard algorithms, not unique to git. Key/identity management is the hard part, but git doesn't solve it for you (if you don't confuse git with GitHub).

webstrand 1y ago
git bundles exist to solve the single-file caching and distribution problems
salzig 1y ago
Going crazy: we cold also adopt the container registry api for distributing gems, similar to how helm charts are also distributed nower days.

phoronixrly 1y ago

You've seen golang's package... situation?... and you still think switching to Git is a good idea?

lr0 1y ago
What "situation"?
- phoronixrly 1y ago
  Let's review for example Traefik's dependency list: https://github.com/traefik/traefik/blob/master/go.mod https://github.com/traefik/traefik/blob/master/go.mod 1. Heavy dependency on Github. AKA Microsoft owns much of the golang ecosystem. Not just the source... The package distribution as well! 2. Many packages are referencing a git (short!) commit hash instead of a version. It still boggles my mind that this is an acceptable practice. Not to mention that git tags can be deleted and recreated... A pinnacle of secure package distribution practices. 3. Stuff like ambiguous imports because apparently nothing enforces proper go.mod files? They are not packages to be compiled after all, they're just repos with some conventional structure (optional)... Mind you, this is popular production-grade software... I think this is much worse than even node packages, let alone bundler and rubygems...
  - SirSavary 1y ago
    1. GitHub dominance is a social phenomenon, not a technical requirement. The go.mod file you linked references ~14 different Git hosts other than GitHub. Go's design doesn't create this centralization; it merely reflects where developers choose to host code. 2. You complain about commit hashes while simultaneously noting that tags can be deleted and recreated. Hashes are precisely the solution to mutable tags. The "short hash" concern is a red herring; Git uses sufficient entropy that collisions are not a practical concern for dependency resolution. As for "secure package distribution," go.sum files verify files verify consistent downloads. What additional security do you believe centralized registries provide? 3. Can you provide a concrete example of an ambiguous import you've encountered? I'm not familiar enough with Go to understand this criticism.

lr0 1y ago

Exactly. Go already adopts that.

dismalaf 1y ago

Well, rubygems (the software) can pull from any git repository. So we kind of have it already anyway.