11 ms·

How Go mitigates supply chain attacks

311 points by spacey 4y ago

synergy20 4y ago
node, python, ruby...etc including go modules can lock their dependencies, Go really wins when it can be wrapped in a reasonably-sized one binary, nothing beats that in deployment, for all the others you have to pull in lots of packages into the target system
unixbane 4y ago
Dependencies being immutable and identified by hash was such an obvious thing 20 years ago. The problem was fitting that into the flow of these crappy UN*X based build systems where to do any mundane task you need to fiddle with files and encodings and semi documented folder heirarchies and use a CLI tool to change other stuff that is too cumbersome to encode directly into files / text. The most obvious concrete instance of this being that it's cumbersome to import a dependency by hash (in Java for example, you really want hashes everywhere instead of reverse TLDs) but obviously better tools (such as a structural code editor) solves this perfectly. It's also annoying every time a company takes a nano step asymptotically towards these proper solutions and spams their crap and everyone buys it, but that's to be expected when your field is broken industrially and academically and even in hobbiest communities.
- doliveira 4y ago
  I feel like much of the current software complexity is mostly caused by the pain that is compiling C dependencies and the outdated Unix build and configuration tools.
- armchairhacker 4y ago
  Ah, the old “we’ve already solved this 40 years ago”. Lambdas, immutable-by-default, async/await, etc.
  - unixbane 4y ago
    Having dependencies not change is literally trivial. Just link to them by hash. Async/await is a pragmatist garbage hack, and does not fit in the archetype you are trying to name.
Jon_Lowtek 4y ago
>> The only commands that will change the go.mod (and therefore the build) are go get and go mod tidy. These commands are not expected to be run automatically or in CI, so changes to dependency trees must be made deliberately and have the opportunity to go through code review. GO doesn't do jack shit to mitigate supply chain attacks. Version pinning with checksum and that is it. But what could go do? Solve supply chain attacks as a language feature? That doesn't even make sense. Application developers using Go must prevent supply chain attacks against their applications. So go get some SAST for your pipeline. Sure there is truth in saying: always verify your dependencies (and their dependencies) yourself with a code review on every update. But you should not do that alone, so let's talk about collaborative vulnerability management. (there is more to sast than vulnerability assessment, but we have to start somewhere) Let's say repositories that publishe go modules should also publish a curated list of known vulnerabilities (including known supply chain attacks) for the modules they publish. This curation is work: reports must be verified before being included in the list and they must be verified quickly. This work scales with the number of packages published. And worse, modules could be published in more than one repository, module publishing repositories can be different from the modules source code repository, and vulnerability lists can exist independent from these repositories - so reports should be synced between different list providers. Different implementations and lack of common standards make this a hard problem. And implicit trust for bulk imports could open the door for takedown attacks. There is an argument that vulnerability listing should be split from source and module publishing: each focusing on their core responsibility. For supply chain attacks this split in responsibilities also makes it harder for an attacker to both attack suppliers and suppress reports. But for all other issues it increase distance as reports must travel upstream. And it creates perverse incentives, like trying to keep reports exclusive to paying customers. To pile on the insanity: reports can be wrong. And there are unfixed CVEs that are many years old (well ok maybe not for go... yet). Downstream there are "mitigated" and "wont-fix" classifications for reports about dependencies and many SAST tooling can't parse that for transitive dependencies. Really, supply chain attacks are the easy case in vulnerability management, because they are so obviously a "must-fix" when detected. (and to please the never update crowd: for a downstream project "fix" can mean not updating a dependency into an attacked version. you are welcome) Long story short: go get some SAST in your pipelines to defend against supply chain attacks. Don't pretend pinning the version and half-assing a code review when you update them is actually solving supply chain attacks. Don't tell me everyone who uses go can find a sophisticated data bomb or intentional rce in some transitive dependency of some lib they update to a new feature release. And don't give me some "well if its transitive then the lib dev should have." Should have doesn't solve shit. Examples for Vulnerability Assessment in GO dependency graphs are GitLabs Gemnasium ( https://gitlab.com/gitlab-org/security-products/gemnasium-db/-/tree/master/go https://gitlab.com/gitlab-org/security-products/gemnasium-db... ) or GitHubs Dependabot ( https://github.com/advisories?query=type%3Areviewed+ecosystem%3Ago https://github.com/advisories?query=type%3Areviewed+ecosyste... ) among many, many others. Not recommendations, just examples! Tools like these help you sort out supply chain attacks that other people have already found, before you update into them and push them downstream. Collaboration is useful. Sure you are still left with reading the source changes of all dependency update, because who knows, you may be the first one to spot one, but hey, good for you.
Friday_ 4y ago
Keep it simple, just don't use dependencies.
- riking 4y ago
  This is the exact point of the final section of the article, yes.
- yjftsjthsd-h 4y ago
  I guess that's fine so long as you're covered by the standard library and/or are willing to reimplement a lot of stuff yourself, but that's a significant trade-off you're asking.
  - 0des 4y ago
    It sounds defensive but Go stdlib is all you need. I believe I'm qualified to say that as last year I challenged myself to only use stdlib, out of several languages I used over the course of the year on big projects, Go was painless and that was a unique experience. So far this year I haven't seen much need to add libraries to my work because everything is already within grasp.
- xxpor 4y ago
  Kind of ironic for a language that was the first that I know about where you can straight up import libraries with a URL directly in your source.
  - throwaway894345 4y ago
    It's not a URL, it's just an identifier that sort-of resembles a URL similar to Kubernetes annotation conventions.
    - benhoyt 4y ago
      It may not be quite a URL, but it contains the URL. It definitely more than just "sort-of resembles a URL". See how the module path to URL lookup is done here: https://go.dev/ref/mod#vcs-find https://go.dev/ref/mod#vcs-find
glenjamin 4y ago
Something this articles glosses over is that some of these approaches, especially the way 'All builds are “locked”' is achieved with minimum version selection, and “A little copying is better than a little dependency” are tradeoffs against an alternative security model, where transitive dependencies are automatically updated to pick up security fixes. Part of the churn and noise in the Node.js dependency ecosystem actually stems from security-related issues being noted in a low-level module, and the ripple effects caused by that when a bunch of maintainers have to go around bumping lockfiles and versions.
- bigdubs 4y ago
  There is a deeper strategy here with go vs. node; having a standard library maintained by professionals. I would rather build on a common set of libraries secured by people who are paid full-time to maintain them, and maybe have slightly worse ergonomics, than have a community of libraries that come and go and have inconsistent quality. This standard library approach yields fewer dependencies, fewer changes over time, and better consistency between projects.
  - dezren39 4y ago
    so, cathedral vs bazaar
  - hrns 4y ago
    Isn't Node API an equivalent of go standard library?
    - bigdubs 4y ago
      Yes Node.js ships with what is effectively a very thin standard library for some low level things like interacting with the file system, the process model, some security features like TLS.
    - jgod 4y ago
      Yes but it's super barebones. Its successor, Ryan Dahl's second attempt at JS runtime, Deno, has a much fuller standard library (inspired by Go).
37ef_ced3 4y ago
Go is a distillation of many decades of software engineering experience. The people behind Go (e.g., Russ Cox) have learned from history. The peanut gallery loves to complain about superficial aspects of Go. Typically these are people with little or no actual experience using the language and tools. They fixate on imagined problems that don't matter in practice. But anyone who has used Go full-time for a few years is likely to deeply respect and appreciate it.
- laerus 4y ago
  - icholy 4y ago
    let foo; try { foo = await is_this_better(); } catch (err) { console.log("if err != nil doesn't seem so bad all of a sudden"); }
    - ryeguy 4y ago
      This comparison misses the point. Go requires you to bubble up errors manually, that's the difference.
      - randomdata 4y ago
        At least one can respect Go for being consistent, even if requires a little extra manual labour. The languages that require you to manually bubble up some types, but not others, are just bizarre. Like, they're onto something neat, but why stop halfway? I don't want to put in the manual work for any type. If I've decided the manual effort is worthwhile, I don't want weird exceptions to worry about. Pick a lane, programming languages.
    - lostcolony 4y ago
      receive {lack_of_erlang_error_handling_is_a_missed_opportunity, true} -> ok after 0 -> throw let_it_crash end.
kubanczyk 4y ago
I don't like the authoritative tone of the article. Especially given the fact that author "conveniently forgets" about go mod edit and go work both of which are deliberately designed counter-mitigations, i.e. they exist to poke small holes in the pin-everything wall. I agree with the spirit of the message though, the surface is much smaller with Go and it shows much planning went into that.
munificent 4y ago
> There is no way for changes in the outside world—such as a new version of a dependency being published—to automatically affect a Go build. > Unlike most other package managers files, Go modules don’t have a separate list of constraints and a lock file pinning specific versions. The version of every dependency contributing to any Go build is fully determined by the go.mod file of the main module. I don't know if this was intentional on the author's part, but this reads to me like it's implying that with package managers that do use lockfiles a new version of a dependency can automatically affect a build. The purpose of a lockfile is to make that false. If you have a valid lockfile, then fetching dependencies is 100% deterministic across machines and the existence of new versions of a package will not affect the build. It is true that most package managers will automatically update a lockfile if it's incomplete instead of failing with an error. That's a different behavior from Go where it fails if the go.mod is incomplete. I suspect in practice this UX choice doesn't make much of a difference. If you're running CI with an incomplete lockfile, you've already gotten yourself into a weird state. It implies you have committed a dependency change without actually testing it, or that you tested it locally and then went out of your way to discard the lockfile changes. Either way, I don't see what this has to do with lockfiles as a concept. Unless I'm missing something, go.mod files are lockfiles.
- mirekrusin 4y ago
  Npm lockfile will refer to same package because you can't republish npm with the same tag (there is also content hash in lockfile). Referring to version tag in git from go's mod can't guarantee that because you can overwrite tag in git. Am I wrong?
  - coder543 4y ago
    Yes. The go.sum file that sits alongside go.mod keeps track of the hashes so that no modification like that can be made, and dependency fetches actually transparently go through a module proxy/mirror that keeps those same hashes as well, and it will prevent you from getting an altered version of a known module even if you’re starting a new project and don’t have a sum file yet. Versions can’t be republished.
Tainnor 4y ago
Most package managers have lockfiles. Yes, npm's decision to have both "npm install" and "npm ci", just so you can confuse and mislead developers, is a bit silly. But Ruby's Bundler, for example, has been refusing to run your code if your lockfile is inconsistent for as long as I remember. Locking dependencies is, generally, a solved problem across most ecosystems (despite node botching its UX). Go doesn't get to claim that it's superior here. But of course, supply chain attacks are still possible with lock files. Because somebody is going to update your dependencies at some point (often for security reasons). And at that point you might be pulling in a malicious dependency which you haven't carefully vetted (because nobody has time to vet all their dependencies thoroughly nowadays). That's still an unsolved problem, as far as I know. I don't think that Go has solved it.
- hobofan 4y ago
  > Most package managers have lockfiles > Locking dependencies is, generally, a solved problem across most ecosystems As someone who is building a package manager for work, and has looked at pretty much every package manager out there (and their ecosystem adoption), I can only say that those don't reflect the current reality of package management (no matter how much I wish it were true). Bundler was the first mainstream package manager to adopt a lockfile (AFAIK) a mere 12 years ago. Many many language ecosystems predate that and are still lacking lockfiles (or even widespread adoption of a single compatible package manager). NPM only got lockfiles 5 years ago (after being pressured by yarn). Gradle got them less than 3.5 years ago, and Maven still doesn't have them (though a niche plugin for it exists). The Python ecosystem is still a hot mess, with ~3 competing solutions (Poetry, Pipenv, Conda), of which Conda just got a 1.0 of their decent conda-lock incubation project a month ago, but due to how setuputils works, the cross-platform package management story is broken almost beyond recovery. In Conan lockfiles are still an experimental feature today. I could go on and on, but I hope that I could paint a picture that while one could argue that with the advent of lockfiles, locking dependencies has become a solved problem _conceptually_, the current status of implementation across ecosystems is still horrible. I'm also constantly amazed about how little love is put into package managers in most language communities, even though they are so crucial for their respective ecosystems. As far as I can tell nowadays Go does have one of the better package managers, which given their horrible starting point is quite the feat. As a nice side-effect of experiments in the Go package ecosystem, one of the people working on go dep also created one of the best resources around package managers: https://medium.com/@sdboyer/so-you-want-to-write-a-package-manager-4ae9c17d9527 https://medium.com/@sdboyer/so-you-want-to-write-a-package-m...
codeflo 4y ago
The article, and the comments praising this approach, don’t do a great job of explaining how any of this is substantively different from running the likes of yarn install --frozen-lockfile, or cargo build --frozen. Here’s the thing: You can argue about being secure by default and encouraging better CI practices. I’d fully agree it isn’t great that one has to know a somewhat obscure flag to get a secure CI build in those environments. But claiming in what I perceive to be in parts a somewhat grandiose tone to have reinvented the wheel, when you’re just describing a standard approach, can make you sound uninformed.
- skybrian 4y ago
  I think at most there's pride in their own solution, which is not something anyone should object to - it's pretty good. It's better than some other systems, but no point in being specific. Not doing specific comparisons is likely a deliberate strategy, since it means the blog post is less likely to go out of date, and it avoids controversy if they get something wrong. Comparisons will need to be written by people familiar with both systems, and they're likely to go out of date quickly.
- arccy 4y ago
  the difference is in the part where you don't have a lock file yet (new dependency, upgrades), and need to choose versions for those.
verdverm 4y ago
There are many gems in its design and implementation in addition to those mentioned in OP. https://verdverm.com/go-mods/ https://verdverm.com/go-mods/
- sigzero 4y ago
  There is a spelling error: but also for security resaons. << reasons not resaons
nu11ptr 4y ago
> Unlike most other package managers files, Go modules don’t have a separate list of constraints and a lock file pinning specific versions. The weird thing about the Go devs is there is always that little bit of elitism under the surface that I detect in their writing (whether it be colors in the playground, the GC, etc). I spent years writing Go and have now moved to Rust. What I find odd is the Rust team has done (IMO) one of the greater achievements in PL history and yet they seem to not have this elitism thing going on (or maybe I just haven't noticed). Go on the other hand, IMO, made some "interesting" language choices (like keeping null) and they seem to want to be celebrated for it and claim their achievements as new and novel. EDIT: To clarify, I'm talking about the core Go devs - those that work on stdlib and the compiler
- vlunkr 4y ago
  I don't see that elitism in this article. Supply chain attacks are a hot topic right now, so it makes sense for them to make a statement about where the language stands with them. They make compelling points, and they're not calling out specific language or package manager as a comparison.
- jchw 4y ago
  This is a pretty genuinely confounding response, and I mean that with absolutely no offense intended. There is a tremendous amount of fighting between devs who prefer Go and Rust, and a tremendous amount of elitism as well, truly from both perspectives. Rust gained a reputation for elitism long before Go did; “Rust Evangelism Strike Force” was never meant to be pejorative, and “Rewrite it in Rust” was never meant to be a joke, but it became one anyways. It’s not hard to see why; Rust is genuinely novel in a way that few other programming languages are. It feels the most like the “future.” But I still like Go a lot. I like Go because of how easy and simple it feels. There is definitely elitism over simplicity, but the elitism I’ve seen and even received from Rust and C++ programmers (…despite that I have been coding C++ forever and do have a few Rust projects as well…) has been pretty much the opposite: Go is too stupid and simple; real programmers need absurdly complex metaprogramming to make basic CLI tools or what have you. Now for what it’s worth, that has cooled down in many regards, and also, Rust is amazing and there’s nothing wrong with advanced metaprogramming. (It’s just another set of tradeoffs, after all. Unquestionably has its benefits.) However, whereas people who have hated on Rust have often come to see it for what it is (an immensely cool, novel programming language,) Go has received the opposite treatment. People soured on it. Now everyone seems sure the GC latency (which of course is just about state of the art) is simply too much for most use cases. It’s seen adoption in all sorts of places and even been competitive with Rust software in performance, but it is commonly discussed as if Go is inherently obsolete because Rust is a better option in every way that matters. Bringing up Go in certain places often subjects you to ridicule, and I’m not joking. The memory ballast is a favorite among detractors to prove that the language is stupid and bad for production environments. So when people do try to tout the benefits of Go, it’s routinely discredited and downplayed for some reason. It’s a nice language to use with a stellar standard library, nice tooling, and pretty good runtime performance. This article doesn’t mention Rust (that I noticed) and Go is still being measured up to Rust in the comments. They both trade blows in different categories, but I truly believe that the fact that Go lacks the novelty of Rust with its borrow checker and language design has caused a lot of people to view it very negatively, and I think that is sad. People loved C for a lot of what it didn’t have. Go is a lot different than C, but for me, the sentiment is very much the same. I think people see what they want to see. I like Go and Rust, but I find myself going back to Go for various reasons and it feels like every year it leads more and more people to ask for justification that they wouldn’t for other languages. It’s a little tiring.
infogulch 4y ago
I'm annoyed by the false dichotomy that colors most discussions around package management that there are only two solutions to publishing software packages: 1. a carefully curated professionally maintained standard library, 2. the complete wild west where anything goes. It's not really "false" because this is the reality of how package managers are designed today, but it's false in the sense that it doesn't have to be this way. You can see this tension in virtually every discussion, users resisting using packages that aren't published in the standard library for fear of attacks and poor quality, and maintainers that resist publishing in the standard library for fear of changing requirements and the appearance of better designs. Sure there are admissible entitlement / responsibility arguments against these respective positions, but that's mostly a distraction because both have a valid point. The problem is that there's no space for intermediate solutions. We need packaging tools to aggregate and publish groups of packages that relate to a particular domain, and organizational tools to ensure quality and continuity of these package groups over time. This mitigates users' fears and reduces their cognitive load by curating the solution space, and it mitigates maintainers fears of ossification and backcompat hell by enabling them to create new package groups. I'm saying there's an entire dimension of valid tradeoffs in this space, but the current design trend of package managers force us into one extreme or the other.
- lawl 4y ago
  > We need packaging tools to aggregate and publish groups of packages that relate to a particular domain, and organizational tools to ensure quality and continuity of these package groups over time Whats stopping you? golang.org/x is kind of like that. Theres nobody stopping you from aggregating packages under foo.bar domain and build a reputation for high quality.
  - infogulch 4y ago
    Good question! The thing that the standard library has that I don't is version aggregation. The problem is not publishing under a single domain, the problem is publishing under a single version. Publishing a bunch of packages that I claim are high quality doesn't help users decide which versions to use, they still have to make this decision on a package-by-package basis. Note that I may be in the middle of a big redesign and some individually-complete packages use the new design but others don't, so you can't just say use the latest of each because they don't all work together yet. In this case I still want to publish each package individually for people who do want fine-grained control, but I wouldn't publish a new version of my "package group" with these versions because they're not integrated yet. My point is that aggregations of verified-interoperable packages is a separate problem domain and existing tools don't suffice to solve it.
alasdair_ 4y ago
Off topic but: Since I assumed this was about physical supply chain attacks (where someone nefarious will either intercept your package to install custom firmware etc. or even change the physical device in some way) - does anyone know where I could find a good guide on mitigating such attacks?
vlunkr 4y ago
The elephant in the room here is NPM, and I think the obvious problem there is the culture. I have a tiny app I've been playing with using create-react-app. There are over 800 directories in node_modules. That absolutely dwarfs the number of any other language I've used. Even in a medium sized rails app, you likely have some awareness of what every dependency is. It's just impossible with npm. One thought I've had to "reboot" the npm culture is to somehow curate packages that are proven to have minimal and safe dependencies, and projects can shift to using those. I imagine there has to be some sort of manual review to make that happen.
- dtgriscom 4y ago
  I think the NPM organization is completely aware just how dangerous this all is, and is eager to hide it. For example, if you look up an NPM package, it'll list its direct dependencies. But, there's no acknowledgement whatsoever of all the stuff that comes along for the ride. I'd love to have a well-supported ranking of NPM packages in terms of their dependencies (and their dependencies' dependencies, etc). Knowing the breadth of immediate dependencies, PLUS the depth of the total dependency tree, would give you some inkling of just how much you're taking for granted when you start using a package.
  - tln 4y ago
    I agree that it would be nice for NPM to show the total footprint of a module, especially if that provides some social incentive to reduce the dependency count.
- woojoo666 4y ago
  The creator of NodeJS talks about how one of the things he regrets is hard-coupling Node to the NPM registry[1]. I imagine this makes it hard to have curated or trusted third-party registries (although note that it is possible to configure private or third-party registries in Node). This is also one of the problems the creator tries to solve in his new runtime, Deno. [1]: https://www.youtube.com/watch?v=M3BM9TB-8yA https://www.youtube.com/watch?v=M3BM9TB-8yA
- coopierez 4y ago
staticassertion 4y ago
It doesn't. Some other tools are worse. Great, saved you a read.
tester756 4y ago
How about getting safe compiler?
tgsovlerkhgsel 4y ago
One downside of such pinning of versions is that if one of your transitive dependencies has a security vulnerability affecting your package, your package will remain affected until you update. As I understand it dependencies-of-dependencies are fetched from the config provided by the library that includes the dependency, in the case of a transitive dependency, everyone between you and the vulnerable package needs to update, and they need to do so in order (i.e. if you depend on A, A depends on B, B depends on C, and C fixes a vulnerability, then first B, then A have to update in order for you to pull in the change).
- hu3 4y ago
  You can force newer transient dependencies: https://stackoverflow.com/questions/70100325/force-a-transitive-dependency-version-in-golang https://stackoverflow.com/questions/70100325/force-a-transit...
strken 4y ago
Worth noting is that, while go build doesn't run arbitrary code, go generate does: package evil import "fmt" // the echo is intentional, in case someone actually tries this for some reason //go:generate echo rm -rf / func PretendGood() { fmt.Println("I am good") } When they say fetching and building code doesn't execute it, that's specific to go get and go build. There's no guarantee that every go subcommand is safe. This is pretty obvious if you know how go generate works and it isn't a flaw of the language, but if I were new to go, this is the kind of article I'd read but still not understand exactly what was safe and what wasn't.
morelisp 4y ago
I feel like everyone is beating around the bush here: NPM is a garbage fire, from the interface to the tooling implementation to the theory to the governance. We can talk at each other back and forth about theoretical benefits, and "friction" vs. "usability" or whatever, but NPM has been and continues to be an unmitigated security disaster. The module proxy could have a package takeover a month for the next three years and still not even come close to the ridiculous shit that has happened on NPM.

311 points by spacey 4y ago

node, python, ruby...etc including go modules can lock their dependencies, Go really wins when it can be wrapped in a reasonably-sized one binary, nothing beats that in deployment, for all the others you have to pull in lots of packages into the target system

unixbane 4y ago

Dependencies being immutable and identified by hash was such an obvious thing 20 years ago. The problem was fitting that into the flow of these crappy UN*X based build systems where to do any mundane task you need to fiddle with files and encodings and semi documented folder heirarchies and use a CLI tool to change other stuff that is too cumbersome to encode directly into files / text. The most obvious concrete instance of this being that it's cumbersome to import a dependency by hash (in Java for example, you really want hashes everywhere instead of reverse TLDs) but obviously better tools (such as a structural code editor) solves this perfectly. It's also annoying every time a company takes a nano step asymptotically towards these proper solutions and spams their crap and everyone buys it, but that's to be expected when your field is broken industrially and academically and even in hobbiest communities.

doliveira 4y ago
I feel like much of the current software complexity is mostly caused by the pain that is compiling C dependencies and the outdated Unix build and configuration tools.
armchairhacker 4y ago
Ah, the old “we’ve already solved this 40 years ago”. Lambdas, immutable-by-default, async/await, etc.
- unixbane 4y ago
  Having dependencies not change is literally trivial. Just link to them by hash. Async/await is a pragmatist garbage hack, and does not fit in the archetype you are trying to name.

Jon_Lowtek 4y ago

>> The only commands that will change the go.mod (and therefore the build) are go get and go mod tidy. These commands are not expected to be run automatically or in CI, so changes to dependency trees must be made deliberately and have the opportunity to go through code review. GO doesn't do jack shit to mitigate supply chain attacks. Version pinning with checksum and that is it. But what could go do? Solve supply chain attacks as a language feature? That doesn't even make sense. Application developers using Go must prevent supply chain attacks against their applications. So go get some SAST for your pipeline. Sure there is truth in saying: always verify your dependencies (and their dependencies) yourself with a code review on every update. But you should not do that alone, so let's talk about collaborative vulnerability management. (there is more to sast than vulnerability assessment, but we have to start somewhere) Let's say repositories that publishe go modules should also publish a curated list of known vulnerabilities (including known supply chain attacks) for the modules they publish. This curation is work: reports must be verified before being included in the list and they must be verified quickly. This work scales with the number of packages published. And worse, modules could be published in more than one repository, module publishing repositories can be different from the modules source code repository, and vulnerability lists can exist independent from these repositories - so reports should be synced between different list providers. Different implementations and lack of common standards make this a hard problem. And implicit trust for bulk imports could open the door for takedown attacks. There is an argument that vulnerability listing should be split from source and module publishing: each focusing on their core responsibility. For supply chain attacks this split in responsibilities also makes it harder for an attacker to both attack suppliers and suppress reports. But for all other issues it increase distance as reports must travel upstream. And it creates perverse incentives, like trying to keep reports exclusive to paying customers. To pile on the insanity: reports can be wrong. And there are unfixed CVEs that are many years old (well ok maybe not for go... yet). Downstream there are "mitigated" and "wont-fix" classifications for reports about dependencies and many SAST tooling can't parse that for transitive dependencies. Really, supply chain attacks are the easy case in vulnerability management, because they are so obviously a "must-fix" when detected. (and to please the never update crowd: for a downstream project "fix" can mean not updating a dependency into an attacked version. you are welcome) Long story short: go get some SAST in your pipelines to defend against supply chain attacks. Don't pretend pinning the version and half-assing a code review when you update them is actually solving supply chain attacks. Don't tell me everyone who uses go can find a sophisticated data bomb or intentional rce in some transitive dependency of some lib they update to a new feature release. And don't give me some "well if its transitive then the lib dev should have." Should have doesn't solve shit. Examples for Vulnerability Assessment in GO dependency graphs are GitLabs Gemnasium ( https://gitlab.com/gitlab-org/security-products/gemnasium-db/-/tree/master/go https://gitlab.com/gitlab-org/security-products/gemnasium-db... ) or GitHubs Dependabot ( https://github.com/advisories?query=type%3Areviewed+ecosystem%3Ago https://github.com/advisories?query=type%3Areviewed+ecosyste... ) among many, many others. Not recommendations, just examples! Tools like these help you sort out supply chain attacks that other people have already found, before you update into them and push them downstream. Collaboration is useful. Sure you are still left with reading the source changes of all dependency update, because who knows, you may be the first one to spot one, but hey, good for you.

Friday_ 4y ago

Keep it simple, just don't use dependencies.

riking 4y ago
This is the exact point of the final section of the article, yes.
yjftsjthsd-h 4y ago
I guess that's fine so long as you're covered by the standard library and/or are willing to reimplement a lot of stuff yourself, but that's a significant trade-off you're asking.
- 0des 4y ago
  It sounds defensive but Go stdlib is all you need. I believe I'm qualified to say that as last year I challenged myself to only use stdlib, out of several languages I used over the course of the year on big projects, Go was painless and that was a unique experience. So far this year I haven't seen much need to add libraries to my work because everything is already within grasp.
xxpor 4y ago
Kind of ironic for a language that was the first that I know about where you can straight up import libraries with a URL directly in your source.
- throwaway894345 4y ago
  It's not a URL, it's just an identifier that sort-of resembles a URL similar to Kubernetes annotation conventions.
  - benhoyt 4y ago
    It may not be quite a URL, but it contains the URL. It definitely more than just "sort-of resembles a URL". See how the module path to URL lookup is done here: https://go.dev/ref/mod#vcs-find https://go.dev/ref/mod#vcs-find

glenjamin 4y ago

Something this articles glosses over is that some of these approaches, especially the way 'All builds are “locked”' is achieved with minimum version selection, and “A little copying is better than a little dependency” are tradeoffs against an alternative security model, where transitive dependencies are automatically updated to pick up security fixes. Part of the churn and noise in the Node.js dependency ecosystem actually stems from security-related issues being noted in a low-level module, and the ripple effects caused by that when a bunch of maintainers have to go around bumping lockfiles and versions.

bigdubs 4y ago
There is a deeper strategy here with go vs. node; having a standard library maintained by professionals. I would rather build on a common set of libraries secured by people who are paid full-time to maintain them, and maybe have slightly worse ergonomics, than have a community of libraries that come and go and have inconsistent quality. This standard library approach yields fewer dependencies, fewer changes over time, and better consistency between projects.
- dezren39 4y ago
  so, cathedral vs bazaar
- hrns 4y ago
  Isn't Node API an equivalent of go standard library?
  - bigdubs 4y ago
    Yes Node.js ships with what is effectively a very thin standard library for some low level things like interacting with the file system, the process model, some security features like TLS.
  - jgod 4y ago
    Yes but it's super barebones. Its successor, Ryan Dahl's second attempt at JS runtime, Deno, has a much fuller standard library (inspired by Go).

37ef_ced3 4y ago

Go is a distillation of many decades of software engineering experience. The people behind Go (e.g., Russ Cox) have learned from history. The peanut gallery loves to complain about superficial aspects of Go. Typically these are people with little or no actual experience using the language and tools. They fixate on imagined problems that don't matter in practice. But anyone who has used Go full-time for a few years is likely to deeply respect and appreciate it.

laerus 4y ago
- icholy 4y ago
  let foo; try { foo = await is_this_better(); } catch (err) { console.log("if err != nil doesn't seem so bad all of a sudden"); }
  - ryeguy 4y ago
    This comparison misses the point. Go requires you to bubble up errors manually, that's the difference.
    - randomdata 4y ago
      At least one can respect Go for being consistent, even if requires a little extra manual labour. The languages that require you to manually bubble up some types, but not others, are just bizarre. Like, they're onto something neat, but why stop halfway? I don't want to put in the manual work for any type. If I've decided the manual effort is worthwhile, I don't want weird exceptions to worry about. Pick a lane, programming languages.
  - lostcolony 4y ago
    receive {lack_of_erlang_error_handling_is_a_missed_opportunity, true} -> ok after 0 -> throw let_it_crash end.

kubanczyk 4y ago

I don't like the authoritative tone of the article. Especially given the fact that author "conveniently forgets" about go mod edit and go work both of which are deliberately designed counter-mitigations, i.e. they exist to poke small holes in the pin-everything wall. I agree with the spirit of the message though, the surface is much smaller with Go and it shows much planning went into that.

munificent 4y ago

> There is no way for changes in the outside world—such as a new version of a dependency being published—to automatically affect a Go build. > Unlike most other package managers files, Go modules don’t have a separate list of constraints and a lock file pinning specific versions. The version of every dependency contributing to any Go build is fully determined by the go.mod file of the main module. I don't know if this was intentional on the author's part, but this reads to me like it's implying that with package managers that do use lockfiles a new version of a dependency can automatically affect a build. The purpose of a lockfile is to make that false. If you have a valid lockfile, then fetching dependencies is 100% deterministic across machines and the existence of new versions of a package will not affect the build. It is true that most package managers will automatically update a lockfile if it's incomplete instead of failing with an error. That's a different behavior from Go where it fails if the go.mod is incomplete. I suspect in practice this UX choice doesn't make much of a difference. If you're running CI with an incomplete lockfile, you've already gotten yourself into a weird state. It implies you have committed a dependency change without actually testing it, or that you tested it locally and then went out of your way to discard the lockfile changes. Either way, I don't see what this has to do with lockfiles as a concept. Unless I'm missing something, go.mod files are lockfiles.

mirekrusin 4y ago
Npm lockfile will refer to same package because you can't republish npm with the same tag (there is also content hash in lockfile). Referring to version tag in git from go's mod can't guarantee that because you can overwrite tag in git. Am I wrong?
- coder543 4y ago
  Yes. The go.sum file that sits alongside go.mod keeps track of the hashes so that no modification like that can be made, and dependency fetches actually transparently go through a module proxy/mirror that keeps those same hashes as well, and it will prevent you from getting an altered version of a known module even if you’re starting a new project and don’t have a sum file yet. Versions can’t be republished.

Tainnor 4y ago

Most package managers have lockfiles. Yes, npm's decision to have both "npm install" and "npm ci", just so you can confuse and mislead developers, is a bit silly. But Ruby's Bundler, for example, has been refusing to run your code if your lockfile is inconsistent for as long as I remember. Locking dependencies is, generally, a solved problem across most ecosystems (despite node botching its UX). Go doesn't get to claim that it's superior here. But of course, supply chain attacks are still possible with lock files. Because somebody is going to update your dependencies at some point (often for security reasons). And at that point you might be pulling in a malicious dependency which you haven't carefully vetted (because nobody has time to vet all their dependencies thoroughly nowadays). That's still an unsolved problem, as far as I know. I don't think that Go has solved it.

hobofan 4y ago
> Most package managers have lockfiles > Locking dependencies is, generally, a solved problem across most ecosystems As someone who is building a package manager for work, and has looked at pretty much every package manager out there (and their ecosystem adoption), I can only say that those don't reflect the current reality of package management (no matter how much I wish it were true). Bundler was the first mainstream package manager to adopt a lockfile (AFAIK) a mere 12 years ago. Many many language ecosystems predate that and are still lacking lockfiles (or even widespread adoption of a single compatible package manager). NPM only got lockfiles 5 years ago (after being pressured by yarn). Gradle got them less than 3.5 years ago, and Maven still doesn't have them (though a niche plugin for it exists). The Python ecosystem is still a hot mess, with ~3 competing solutions (Poetry, Pipenv, Conda), of which Conda just got a 1.0 of their decent conda-lock incubation project a month ago, but due to how setuputils works, the cross-platform package management story is broken almost beyond recovery. In Conan lockfiles are still an experimental feature today. I could go on and on, but I hope that I could paint a picture that while one could argue that with the advent of lockfiles, locking dependencies has become a solved problem _conceptually_, the current status of implementation across ecosystems is still horrible. I'm also constantly amazed about how little love is put into package managers in most language communities, even though they are so crucial for their respective ecosystems. As far as I can tell nowadays Go does have one of the better package managers, which given their horrible starting point is quite the feat. As a nice side-effect of experiments in the Go package ecosystem, one of the people working on go dep also created one of the best resources around package managers: https://medium.com/@sdboyer/so-you-want-to-write-a-package-manager-4ae9c17d9527 https://medium.com/@sdboyer/so-you-want-to-write-a-package-m...

codeflo 4y ago

The article, and the comments praising this approach, don’t do a great job of explaining how any of this is substantively different from running the likes of yarn install --frozen-lockfile, or cargo build --frozen. Here’s the thing: You can argue about being secure by default and encouraging better CI practices. I’d fully agree it isn’t great that one has to know a somewhat obscure flag to get a secure CI build in those environments. But claiming in what I perceive to be in parts a somewhat grandiose tone to have reinvented the wheel, when you’re just describing a standard approach, can make you sound uninformed.

skybrian 4y ago
I think at most there's pride in their own solution, which is not something anyone should object to - it's pretty good. It's better than some other systems, but no point in being specific. Not doing specific comparisons is likely a deliberate strategy, since it means the blog post is less likely to go out of date, and it avoids controversy if they get something wrong. Comparisons will need to be written by people familiar with both systems, and they're likely to go out of date quickly.
arccy 4y ago
the difference is in the part where you don't have a lock file yet (new dependency, upgrades), and need to choose versions for those.

verdverm 4y ago

There are many gems in its design and implementation in addition to those mentioned in OP. https://verdverm.com/go-mods/ https://verdverm.com/go-mods/

sigzero 4y ago
There is a spelling error: but also for security resaons. << reasons not resaons

nu11ptr 4y ago

> Unlike most other package managers files, Go modules don’t have a separate list of constraints and a lock file pinning specific versions. The weird thing about the Go devs is there is always that little bit of elitism under the surface that I detect in their writing (whether it be colors in the playground, the GC, etc). I spent years writing Go and have now moved to Rust. What I find odd is the Rust team has done (IMO) one of the greater achievements in PL history and yet they seem to not have this elitism thing going on (or maybe I just haven't noticed). Go on the other hand, IMO, made some "interesting" language choices (like keeping null) and they seem to want to be celebrated for it and claim their achievements as new and novel. EDIT: To clarify, I'm talking about the core Go devs - those that work on stdlib and the compiler

vlunkr 4y ago
I don't see that elitism in this article. Supply chain attacks are a hot topic right now, so it makes sense for them to make a statement about where the language stands with them. They make compelling points, and they're not calling out specific language or package manager as a comparison.
jchw 4y ago
This is a pretty genuinely confounding response, and I mean that with absolutely no offense intended. There is a tremendous amount of fighting between devs who prefer Go and Rust, and a tremendous amount of elitism as well, truly from both perspectives. Rust gained a reputation for elitism long before Go did; “Rust Evangelism Strike Force” was never meant to be pejorative, and “Rewrite it in Rust” was never meant to be a joke, but it became one anyways. It’s not hard to see why; Rust is genuinely novel in a way that few other programming languages are. It feels the most like the “future.” But I still like Go a lot. I like Go because of how easy and simple it feels. There is definitely elitism over simplicity, but the elitism I’ve seen and even received from Rust and C++ programmers (…despite that I have been coding C++ forever and do have a few Rust projects as well…) has been pretty much the opposite: Go is too stupid and simple; real programmers need absurdly complex metaprogramming to make basic CLI tools or what have you. Now for what it’s worth, that has cooled down in many regards, and also, Rust is amazing and there’s nothing wrong with advanced metaprogramming. (It’s just another set of tradeoffs, after all. Unquestionably has its benefits.) However, whereas people who have hated on Rust have often come to see it for what it is (an immensely cool, novel programming language,) Go has received the opposite treatment. People soured on it. Now everyone seems sure the GC latency (which of course is just about state of the art) is simply too much for most use cases. It’s seen adoption in all sorts of places and even been competitive with Rust software in performance, but it is commonly discussed as if Go is inherently obsolete because Rust is a better option in every way that matters. Bringing up Go in certain places often subjects you to ridicule, and I’m not joking. The memory ballast is a favorite among detractors to prove that the language is stupid and bad for production environments. So when people do try to tout the benefits of Go, it’s routinely discredited and downplayed for some reason. It’s a nice language to use with a stellar standard library, nice tooling, and pretty good runtime performance. This article doesn’t mention Rust (that I noticed) and Go is still being measured up to Rust in the comments. They both trade blows in different categories, but I truly believe that the fact that Go lacks the novelty of Rust with its borrow checker and language design has caused a lot of people to view it very negatively, and I think that is sad. People loved C for a lot of what it didn’t have. Go is a lot different than C, but for me, the sentiment is very much the same. I think people see what they want to see. I like Go and Rust, but I find myself going back to Go for various reasons and it feels like every year it leads more and more people to ask for justification that they wouldn’t for other languages. It’s a little tiring.

infogulch 4y ago

I'm annoyed by the false dichotomy that colors most discussions around package management that there are only two solutions to publishing software packages: 1. a carefully curated professionally maintained standard library, 2. the complete wild west where anything goes. It's not really "false" because this is the reality of how package managers are designed today, but it's false in the sense that it doesn't have to be this way. You can see this tension in virtually every discussion, users resisting using packages that aren't published in the standard library for fear of attacks and poor quality, and maintainers that resist publishing in the standard library for fear of changing requirements and the appearance of better designs. Sure there are admissible entitlement / responsibility arguments against these respective positions, but that's mostly a distraction because both have a valid point. The problem is that there's no space for intermediate solutions. We need packaging tools to aggregate and publish groups of packages that relate to a particular domain, and organizational tools to ensure quality and continuity of these package groups over time. This mitigates users' fears and reduces their cognitive load by curating the solution space, and it mitigates maintainers fears of ossification and backcompat hell by enabling them to create new package groups. I'm saying there's an entire dimension of valid tradeoffs in this space, but the current design trend of package managers force us into one extreme or the other.

lawl 4y ago
> We need packaging tools to aggregate and publish groups of packages that relate to a particular domain, and organizational tools to ensure quality and continuity of these package groups over time Whats stopping you? golang.org/x is kind of like that. Theres nobody stopping you from aggregating packages under foo.bar domain and build a reputation for high quality.
- infogulch 4y ago
  Good question! The thing that the standard library has that I don't is version aggregation. The problem is not publishing under a single domain, the problem is publishing under a single version. Publishing a bunch of packages that I claim are high quality doesn't help users decide which versions to use, they still have to make this decision on a package-by-package basis. Note that I may be in the middle of a big redesign and some individually-complete packages use the new design but others don't, so you can't just say use the latest of each because they don't all work together yet. In this case I still want to publish each package individually for people who do want fine-grained control, but I wouldn't publish a new version of my "package group" with these versions because they're not integrated yet. My point is that aggregations of verified-interoperable packages is a separate problem domain and existing tools don't suffice to solve it.

alasdair_ 4y ago

Off topic but: Since I assumed this was about physical supply chain attacks (where someone nefarious will either intercept your package to install custom firmware etc. or even change the physical device in some way) - does anyone know where I could find a good guide on mitigating such attacks?

vlunkr 4y ago

The elephant in the room here is NPM, and I think the obvious problem there is the culture. I have a tiny app I've been playing with using create-react-app. There are over 800 directories in node_modules. That absolutely dwarfs the number of any other language I've used. Even in a medium sized rails app, you likely have some awareness of what every dependency is. It's just impossible with npm. One thought I've had to "reboot" the npm culture is to somehow curate packages that are proven to have minimal and safe dependencies, and projects can shift to using those. I imagine there has to be some sort of manual review to make that happen.

dtgriscom 4y ago
I think the NPM organization is completely aware just how dangerous this all is, and is eager to hide it. For example, if you look up an NPM package, it'll list its direct dependencies. But, there's no acknowledgement whatsoever of all the stuff that comes along for the ride. I'd love to have a well-supported ranking of NPM packages in terms of their dependencies (and their dependencies' dependencies, etc). Knowing the breadth of immediate dependencies, PLUS the depth of the total dependency tree, would give you some inkling of just how much you're taking for granted when you start using a package.
- tln 4y ago
  I agree that it would be nice for NPM to show the total footprint of a module, especially if that provides some social incentive to reduce the dependency count.
woojoo666 4y ago
The creator of NodeJS talks about how one of the things he regrets is hard-coupling Node to the NPM registry[1]. I imagine this makes it hard to have curated or trusted third-party registries (although note that it is possible to configure private or third-party registries in Node). This is also one of the problems the creator tries to solve in his new runtime, Deno. [1]: https://www.youtube.com/watch?v=M3BM9TB-8yA https://www.youtube.com/watch?v=M3BM9TB-8yA
coopierez 4y ago

staticassertion 4y ago

It doesn't. Some other tools are worse. Great, saved you a read.

tester756 4y ago

How about getting safe compiler?

tgsovlerkhgsel 4y ago

One downside of such pinning of versions is that if one of your transitive dependencies has a security vulnerability affecting your package, your package will remain affected until you update. As I understand it dependencies-of-dependencies are fetched from the config provided by the library that includes the dependency, in the case of a transitive dependency, everyone between you and the vulnerable package needs to update, and they need to do so in order (i.e. if you depend on A, A depends on B, B depends on C, and C fixes a vulnerability, then first B, then A have to update in order for you to pull in the change).

hu3 4y ago
You can force newer transient dependencies: https://stackoverflow.com/questions/70100325/force-a-transitive-dependency-version-in-golang https://stackoverflow.com/questions/70100325/force-a-transit...

strken 4y ago

Worth noting is that, while go build doesn't run arbitrary code, go generate does: package evil import "fmt" // the echo is intentional, in case someone actually tries this for some reason //go:generate echo rm -rf / func PretendGood() { fmt.Println("I am good") } When they say fetching and building code doesn't execute it, that's specific to go get and go build. There's no guarantee that every go subcommand is safe. This is pretty obvious if you know how go generate works and it isn't a flaw of the language, but if I were new to go, this is the kind of article I'd read but still not understand exactly what was safe and what wasn't.

morelisp 4y ago

I feel like everyone is beating around the bush here: NPM is a garbage fire, from the interface to the tooling implementation to the theory to the governance. We can talk at each other back and forth about theoretical benefits, and "friction" vs. "usability" or whatever, but NPM has been and continues to be an unmitigated security disaster. The module proxy could have a package takeover a month for the next three years and still not even come close to the ridiculous shit that has happened on NPM.