6 ms·

Tinycolor supply chain attack post-mortem

168 points by STRiDEX 1y ago

drdrey 1y ago
> A while ago, I collaborated on angulartics2, a shared repository where multiple people still had admin rights. That repo still contained a GitHub Actions secret — a npm token with broad publish rights. This collaborator had access to projects with other people which I believe explains some of the other 40 initial packages that were affected. > A new Shai-Hulud branch was force pushed to angulartics2 with a malicious github action workflow by a collaborator. The workflow ran immediately on push (did not need review since the collaborator is an admin) and stole the npm token. With the stolen token, the attacker published malicious versions of 20 packages. Many of which are not widely used, however the @ctrl/tinycolor package is downloaded about 2 million times a week. I still don't get it. An admin on angulartics2 gets hacked, his Github access is used to push a malicious workflow that extracts an npm token. But why would an npm token in angulartics2 have publication rights to tinycolor?
- STRiDEX 1y ago
  Sorry if that wasn't clear. This was a token with global publish rights to my npm packages.
  - Scaevolus 1y ago
    I was confused too. Was it your npm token stored in angulartics2 as a Github Actions secret, so it could publish new angulartics2 versions?
    - STRiDEX 1y ago
      Yes, exactly.
- deleted 1y ago
  [deleted]
- tetha 1y ago
  > But why would an npm token in angulartics2 have publication rights to tinycolor? Imo, this is one of the most classical ways organizations get pwned: That one sin from your youth years ago comes to bite you in the butt. We also had one of these years ago. It wasn't the modern stack everyone was working to scan and optimize and keep us secure that allowed someone to upload stuff to our servers. It was the editor that had been replaced years and years ago, and it's replacement had also been replaced, the way it was packaged wasn't seen by the build-time security scans, but eventually someone found it with a URL scan. Whoopsie.
  - Terr_ 1y ago
    Thinking of biology, the reason often given for the disappearance of "unused" genes/base-pairs is that there's a metabolic cost to keeping them around and copying them on every cell division, so they vanish from a form of passive attrition. I wonder if someday we'll find there's also a more active process, which resembles "remove old shit because it may contain security vulnerabilities."
- hinkley 1y ago
  I have admin rights on someone else’s npm repo and I’ve done most of the recent releases. Becoming admin lit a fire under me to fix all of the annoying things and shitty design decisions that have been stuck in the backlog for years so most of the commits are also mine. I don’t want my name on broken code that “works”. I had just about convinced myself that we should be using a GitHub action to publish packages because there was always the possibility that publishing directly via 2FA, that one (or specifically I) could fuck up and publish something that wasn’t a snapshot of trunk. But I worried about stuff like this and procrastinated on forcing the issue with the other admins. And it looks like the universe has again rewarded my procrastination. I don’t know what the answer is but giving your credentials to a third party clearly isn’t it.
  - baobun 1y ago
    npm has had support for package-scoped publish tokens (with optional 2FA enforcement) for a few years by now. So in case of compromise, the blast radius would be a single package. The OP gave the GH repo too broad permissions. There is no good reason for the repo CI workflow to have full access to everything under their account.
    - hinkley 1y ago
      I’m using the 2FA tokens, my complaint is that “npm publish” from my own machine has no guarantees of being an exact snapshot of trunk. And a worm could inject code into my system to do the Kernighan exploit. I think over the last few weeks I have at least talked myself into going back to maintaining multiple user accounts on my laptop to separate personal, open source, and entertainment into separate accounts to reduce the last radius, but the fact is sometimes I like to do two things at once and that will be a pain.
      - baobun 1y ago
        You can get pretty close to guarantees. Fresh git checkout on prod publish. Run all npm/node commands in ephemeral rootless containers. Only have publish token exposed and injected when you are actually publishing (not on install/build just prior). Separating users like you mentioned doesn't hurt bt doesn't sound like your lower-hanging fruit nor something that would likely save your bacon on its own without other workflow adjustments. None of this is relatively difficult per se, just a bit extra friction which should be worth it to avoid these kinds of events.
rectang 1y ago
Two-factor auth for publishing is helpful, but requiring cryptographically signed approval by multiple authors would be more helpful. Then compromising a single author wouldn't be enough.
- tcoff91 1y ago
  Many packages have only 1 author.
  - chrisweekly 1y ago
    and (as in this case), that 1 author may use a single token to authz publishing many packages
  - deleted 1y ago
    [deleted]
  - rectang 1y ago
    The conclusion I'm coming to is that depending on packages which only have a single author is problematic. There are too many ways that packages published by one person can be compromised. Packages which don't have approval and review by a reliable third party shouldn't be visible by default in a package manager.
    - Hackbraten 1y ago
      How are you supposed to gain collaborators for a project that no one can possibly find?
      - rectang 1y ago
        There are ways, but at a high level, I don't care. I hate how modern package managers have come to value author convenience over downstream user security.
        Hackbraten 1y ago
bikeshaving 1y ago
> Local 2FA based publishing isn’t sustainable... Why is local 2FA unsustainable?! The real problem here is automated publishing workflows. The overwhelming majority of NPM packages do not publish often enough or have complicated enough release steps to justify tokens with the power to publish without human intervention. What is so fucking difficult about running `npm publish` manually with 2FA? If maintainers are unwilling to do this for their packages, they should reconsider the number of packages they maintain.
- STRiDEX 1y ago
  That's fair, I'm referring to the number of mistakes that happen with local publishing. Publishing the wrong branch, not building from latest etc
  - skydhash 1y ago
    So add a wrapper for that, a quick script that checks which branch and revision you are publishing from. The issue here is publishing from a CI you do not control that well and with automated events.
  - paxys 1y ago
    You can run the exact same script locally as you do in CI, with the only difference being the addition of a 2FA prompt.
    - STRiDEX 1y ago
      That's a good point, I would lose package provenance that way. I guess that is fine since it didn't prevent anything here. I can look into that.
indigodaddy 1y ago
Anyone know of a published tool/script to check for the existence of any of the vulnerable npm packages? I don't see anything like that in the stepsecurity page.
- retlehs 1y ago
  This won’t protect against everything, but it still seems like a good idea to implement: https://github.com/danielroe/provenance-action https://github.com/danielroe/provenance-action
  - indigodaddy 1y ago
    Yep I did see that, but I'm not planning on pushing anything, just want a tool to scan for any of the offending packages. Could make my own but feel like somebody must have already made something (and probably better than I can)
    - dflock 1y ago
      - [supply-chain-security · GitHub Topics · GitHub](https://github.com/topics/supply-chain-security https://github.com/topics/supply-chain-security) - [GitHub - safedep/vet: Protect against malicious open source packages](https://github.com/safedep/vet https://github.com/safedep/vet) - [GitHub - AikidoSec/safe-chain](https://github.com/AikidoSec/safe-chain https://github.com/AikidoSec/safe-chain) - npm audit
      - indigodaddy 1y ago
        vet and safe-chain look good thanks! I'm just dabbling with Node only (no experience really), so haven't used npm audit but will see how that works too. Appreciate the links.
- sibeliuss 1y ago
  `npm audit` for known issues
cyberax 1y ago
> exfiltrated a npm token with broad publish rights I freaking HATE tokens. I hate them. There should be a better way to do authentication than a glorified static password. An example of how to do it correctly: Github as a token provider for AWS: https://aws.amazon.com/blogs/security/use-iam-roles-to-connect-github-actions-to-actions-in-aws/ https://aws.amazon.com/blogs/security/use-iam-roles-to-conne... But this is an exception, rather than a rule.
- er4hn 1y ago
  Well the idea behind tokens is that they should be time and authZ limited. In most cases they are not so they degrade to a glorified static password. Solutions like generating them live with a short lifetime, using solutions like oauth w/ proper scopes, biscuits that limit what they can do in detail, etc, all exist and are rarely used.
- chatmasta 1y ago
  These machine-to-machine OIDC flows seem secure, and maybe they are when they’re implemented properly, but they’re really difficult to configure. And I can’t shake the feeling that they’re basically just “tokens with more moving parts,” at least for a big chunk of exploitation paths. Without a human in the loop, there’s still some “thing” that gets compromised, whether it’s a token or something that generates time-limited tokens. In the case of this worm, the OIDC flow wouldn’t even help. The GitHub workflow was compromised. If the workflow was using an OIDC credential like this to publish to npm, the only difference would be the npm publish command wouldn’t use any credential because the GitHub workflow would inject some temporary identity into the environment. But the root problem would remain: an untrusted user shouldn’t be able to execute a workflow with secret parameters. Maybe OIDC would limit the impact to be more fine-grained, but so would changing the token permissions.
  - tetha 1y ago
    Hence you need to start thinking about threat models and levels of compromise, even in your build system. If I control the issuing and governance of these short-lived secrets, they very much help against many attacks. Go ahead and extract an upload token for one project which lives for 60 seconds, be my guest. Once I lose control how these tokens are created, most of these advantages go away - you can just create a token every minute, for any project this infrastructure might be responsible for. If I maintain control about my pipeline definition, I can again do a lot of work to limit damage. For example, if I am in control, I can make sure the stages running untrusted codes have as little access to secrets as possible, and possibly isolate them in bubblewrap, VMs, ..., minimize the code with access to publishing rights. Once I lose control about the pipeline structure, all that goes away. Just add a build step to push all information and secrets to mastodon in individual toots, yey. To me, this has very much raised questions about keeping pipeline definitions and code in one repository. Or at least, to keep a publishing/release process in there. I don't have a simple solution there, especially for OSS software with little infrastructure - it's not an easy topic. But with these supply chain attacks coming hot and fast every 2 weeks, it's something to think about.
darkamaul 1y ago
> That repo still contained a GitHub Actions secret — a npm token with broad publish rights. One of the advantages of Trusted Publishing [0] is that we no longer need long-lived tokens with publish rights. Instead, tokens are generated on the CI VM and are valid for only 15 minutes. This has already been implemented in several ecosystems (PyPI, npm, Cargo, Homebrew), and I encourage everyone to use it, it actually makes publishing a bit _easier_. More importantly, if the documentation around this still feels unclear, don’t hesitate to ask for help. Ecosystem maintainers are usually eager to see wider adoption of this feature. [0] https://docs.pypi.org/trusted-publishers/ https://docs.pypi.org/trusted-publishers/
- procaryote 1y ago
  What we need now is a flag in repos for projects that use this kind of thing, so you can easily prohibit dependencies that don't
- realityking 1y ago
  I had missed the announcement that this is now available for npm: https://github.blog/changelog/2025-07-31-npm-trusted-publishing-with-oidc-is-generally-available/ https://github.blog/changelog/2025-07-31-npm-trusted-publish... Guess I know what I’ll be doing this weekend.
mnahkies 1y ago
I think the point around incorporating MFA into the automated publishing flow isn't getting enough attention. I've got no problem with doing an MFA prompt to confirm publish by a CI workflow - but last I looked this was a convoluted process of opening a https tunnel out (using a third party solution) such that you could provide the code. I'd love to see either npm or GitHub provide an easy, out the box way, for me to provide/confirm a code during CI.
- tcoff91 1y ago
  Publishing a package involves 2 phases: uploading the package to npmjs, and making it availble to users. Right now these 2 phases are bundled together into 1 operation. I think the right way to approach this is to unbundle uploading the packages & publishing packages so that they're available to end-users. CI systems should be able to build & upload packages in a fully automated manner. Publishing the uploaded packages should require a human to log into npmjs's website & manually publish the package and go through MFA.
  - mnahkies 1y ago
    Completely agree tbh, and that would be one of my preferred approaches should npm be the actor to implement a solution. I also think it makes sense for GitHub to implement the ability to mark a workflow as sensitive and requiring "sudo mode" (MFA prompt) to run. It's not miles away from what they already do around requiring maintainer approval to run workflows on PRs. Ideally both of these would exist, as not every npm package is published via GitHub actions (or any CI system), and not every GitHub workflow taking a sensitive action is publishing an npm package.
  - deleted 1y ago
    [deleted]
  - klysm 1y ago
    npm should require this with packages that have a large enough blast radius
    - 1y ago
1oooqooq 1y ago
if the ci job force pushed something deep in the git history tree?
waterTanuki 1y ago
Something somewhere needs to change because the status quo just isn't working. Yes, we can cheer on the benefit of OIDC tokens and zero-trust solutions in CI pipelines on HN all we want, but the fact is there's a significant number of library developers out there with millions of package downloads per week that will refuse to do anything about security until they're compromised or npm blocks them from publishing until they do. And then there's other non-sensical proposals like spelunking deep into projects some which could be over a decade old and just rip out all the dependencies until there's nothing but a standard library is left. Look, I'm all for a better std lib, I think reducing the number of dependencies we have is good. But just saying "you should reduce dependencies" will do nothing concrete to fix the problem which already exists, because it's much easier said than done. So either tens of thousands or hundreds of thousands of developers stop using npm, and everyone refactors their projects to add more code and strip dependencies, or npm starts enforcing things like 2FA and OIDC for package developers with over X number of weekly downloads, and blocks publishing for those that don't follow the new security rules. I think it's clear which solution is more practical to implement. The only other option is for npm to completely lose its reputation and then we wind up with XKCD 927 again.
zeroq 1y ago
For the last 10 years I've been advocating for manual releases. I've encountered a lot of backlash, but is it really that alien concept these days? CI/CD is cool, but between this and recent CF drama it seems we have a pretty solid evidence it can lead to a serious problems. I worked at a BigBank once where deployments to production required at least five people present at a time and a lot of theatrics, but at least we knew what we were deploying.
- cyphar 1y ago
  I completely agree. You are infinitely more likely to get implicated in some widespread attack due to bugs in GitHub Actions or your automated release scripts than have your local machine's local build and signing infrastructure attacked. I have yet to see any evidence that fancy CI/CD systems are better than good old fashioned tarballs and detached signatures. Bonus points for distribution packaging systems where they add an additional layer of review and separate validation of releases. People seem to gloss over that fact the "stodgy old-fashioned" rigamarole of Debian is part of the reason why the entire internet didn't pwned by the xz attack. At the very least you should require a human to sign the blobs before the release is actually published. (This isn't always enough if the attacker can add themselves to the maintainers list and sign with their own key, which is why the distribution packaging systems where they maintain their own trusted copy of upstream keyrings is far more preferable.) If your threat model boils down to "if my GitHub account gets attacked or even a single API key is leaked all of my users are fucked" then you really need to take a long look at a mirror and ask yourself if that is reasonable.

168 points by STRiDEX 1y ago

> A while ago, I collaborated on angulartics2, a shared repository where multiple people still had admin rights. That repo still contained a GitHub Actions secret — a npm token with broad publish rights. This collaborator had access to projects with other people which I believe explains some of the other 40 initial packages that were affected. > A new Shai-Hulud branch was force pushed to angulartics2 with a malicious github action workflow by a collaborator. The workflow ran immediately on push (did not need review since the collaborator is an admin) and stole the npm token. With the stolen token, the attacker published malicious versions of 20 packages. Many of which are not widely used, however the @ctrl/tinycolor package is downloaded about 2 million times a week. I still don't get it. An admin on angulartics2 gets hacked, his Github access is used to push a malicious workflow that extracts an npm token. But why would an npm token in angulartics2 have publication rights to tinycolor?

STRiDEX 1y ago
Sorry if that wasn't clear. This was a token with global publish rights to my npm packages.
- Scaevolus 1y ago
  I was confused too. Was it your npm token stored in angulartics2 as a Github Actions secret, so it could publish new angulartics2 versions?
  - STRiDEX 1y ago
    Yes, exactly.
deleted 1y ago
[deleted]
tetha 1y ago
> But why would an npm token in angulartics2 have publication rights to tinycolor? Imo, this is one of the most classical ways organizations get pwned: That one sin from your youth years ago comes to bite you in the butt. We also had one of these years ago. It wasn't the modern stack everyone was working to scan and optimize and keep us secure that allowed someone to upload stuff to our servers. It was the editor that had been replaced years and years ago, and it's replacement had also been replaced, the way it was packaged wasn't seen by the build-time security scans, but eventually someone found it with a URL scan. Whoopsie.
- Terr_ 1y ago
  Thinking of biology, the reason often given for the disappearance of "unused" genes/base-pairs is that there's a metabolic cost to keeping them around and copying them on every cell division, so they vanish from a form of passive attrition. I wonder if someday we'll find there's also a more active process, which resembles "remove old shit because it may contain security vulnerabilities."
hinkley 1y ago
I have admin rights on someone else’s npm repo and I’ve done most of the recent releases. Becoming admin lit a fire under me to fix all of the annoying things and shitty design decisions that have been stuck in the backlog for years so most of the commits are also mine. I don’t want my name on broken code that “works”. I had just about convinced myself that we should be using a GitHub action to publish packages because there was always the possibility that publishing directly via 2FA, that one (or specifically I) could fuck up and publish something that wasn’t a snapshot of trunk. But I worried about stuff like this and procrastinated on forcing the issue with the other admins. And it looks like the universe has again rewarded my procrastination. I don’t know what the answer is but giving your credentials to a third party clearly isn’t it.
- baobun 1y ago
  npm has had support for package-scoped publish tokens (with optional 2FA enforcement) for a few years by now. So in case of compromise, the blast radius would be a single package. The OP gave the GH repo too broad permissions. There is no good reason for the repo CI workflow to have full access to everything under their account.
  - hinkley 1y ago
    I’m using the 2FA tokens, my complaint is that “npm publish” from my own machine has no guarantees of being an exact snapshot of trunk. And a worm could inject code into my system to do the Kernighan exploit. I think over the last few weeks I have at least talked myself into going back to maintaining multiple user accounts on my laptop to separate personal, open source, and entertainment into separate accounts to reduce the last radius, but the fact is sometimes I like to do two things at once and that will be a pain.
    - baobun 1y ago
      You can get pretty close to guarantees. Fresh git checkout on prod publish. Run all npm/node commands in ephemeral rootless containers. Only have publish token exposed and injected when you are actually publishing (not on install/build just prior). Separating users like you mentioned doesn't hurt bt doesn't sound like your lower-hanging fruit nor something that would likely save your bacon on its own without other workflow adjustments. None of this is relatively difficult per se, just a bit extra friction which should be worth it to avoid these kinds of events.

rectang 1y ago

Two-factor auth for publishing is helpful, but requiring cryptographically signed approval by multiple authors would be more helpful. Then compromising a single author wouldn't be enough.

tcoff91 1y ago
Many packages have only 1 author.
- chrisweekly 1y ago
  and (as in this case), that 1 author may use a single token to authz publishing many packages
- deleted 1y ago
  [deleted]
- rectang 1y ago
  The conclusion I'm coming to is that depending on packages which only have a single author is problematic. There are too many ways that packages published by one person can be compromised. Packages which don't have approval and review by a reliable third party shouldn't be visible by default in a package manager.
  - Hackbraten 1y ago
    How are you supposed to gain collaborators for a project that no one can possibly find?
    - rectang 1y ago
      There are ways, but at a high level, I don't care. I hate how modern package managers have come to value author convenience over downstream user security.
      - Hackbraten 1y ago

bikeshaving 1y ago

> Local 2FA based publishing isn’t sustainable... Why is local 2FA unsustainable?! The real problem here is automated publishing workflows. The overwhelming majority of NPM packages do not publish often enough or have complicated enough release steps to justify tokens with the power to publish without human intervention. What is so fucking difficult about running `npm publish` manually with 2FA? If maintainers are unwilling to do this for their packages, they should reconsider the number of packages they maintain.

STRiDEX 1y ago
That's fair, I'm referring to the number of mistakes that happen with local publishing. Publishing the wrong branch, not building from latest etc
- skydhash 1y ago
  So add a wrapper for that, a quick script that checks which branch and revision you are publishing from. The issue here is publishing from a CI you do not control that well and with automated events.
- paxys 1y ago
  You can run the exact same script locally as you do in CI, with the only difference being the addition of a 2FA prompt.
  - STRiDEX 1y ago
    That's a good point, I would lose package provenance that way. I guess that is fine since it didn't prevent anything here. I can look into that.

indigodaddy 1y ago

Anyone know of a published tool/script to check for the existence of any of the vulnerable npm packages? I don't see anything like that in the stepsecurity page.

retlehs 1y ago
This won’t protect against everything, but it still seems like a good idea to implement: https://github.com/danielroe/provenance-action https://github.com/danielroe/provenance-action
- indigodaddy 1y ago
  Yep I did see that, but I'm not planning on pushing anything, just want a tool to scan for any of the offending packages. Could make my own but feel like somebody must have already made something (and probably better than I can)
  - dflock 1y ago
    - [supply-chain-security · GitHub Topics · GitHub](https://github.com/topics/supply-chain-security https://github.com/topics/supply-chain-security) - [GitHub - safedep/vet: Protect against malicious open source packages](https://github.com/safedep/vet https://github.com/safedep/vet) - [GitHub - AikidoSec/safe-chain](https://github.com/AikidoSec/safe-chain https://github.com/AikidoSec/safe-chain) - npm audit
    - indigodaddy 1y ago
      vet and safe-chain look good thanks! I'm just dabbling with Node only (no experience really), so haven't used npm audit but will see how that works too. Appreciate the links.
sibeliuss 1y ago
`npm audit` for known issues

cyberax 1y ago

> exfiltrated a npm token with broad publish rights I freaking HATE tokens. I hate them. There should be a better way to do authentication than a glorified static password. An example of how to do it correctly: Github as a token provider for AWS: https://aws.amazon.com/blogs/security/use-iam-roles-to-connect-github-actions-to-actions-in-aws/ https://aws.amazon.com/blogs/security/use-iam-roles-to-conne... But this is an exception, rather than a rule.

er4hn 1y ago
Well the idea behind tokens is that they should be time and authZ limited. In most cases they are not so they degrade to a glorified static password. Solutions like generating them live with a short lifetime, using solutions like oauth w/ proper scopes, biscuits that limit what they can do in detail, etc, all exist and are rarely used.
chatmasta 1y ago
These machine-to-machine OIDC flows seem secure, and maybe they are when they’re implemented properly, but they’re really difficult to configure. And I can’t shake the feeling that they’re basically just “tokens with more moving parts,” at least for a big chunk of exploitation paths. Without a human in the loop, there’s still some “thing” that gets compromised, whether it’s a token or something that generates time-limited tokens. In the case of this worm, the OIDC flow wouldn’t even help. The GitHub workflow was compromised. If the workflow was using an OIDC credential like this to publish to npm, the only difference would be the npm publish command wouldn’t use any credential because the GitHub workflow would inject some temporary identity into the environment. But the root problem would remain: an untrusted user shouldn’t be able to execute a workflow with secret parameters. Maybe OIDC would limit the impact to be more fine-grained, but so would changing the token permissions.
- tetha 1y ago
  Hence you need to start thinking about threat models and levels of compromise, even in your build system. If I control the issuing and governance of these short-lived secrets, they very much help against many attacks. Go ahead and extract an upload token for one project which lives for 60 seconds, be my guest. Once I lose control how these tokens are created, most of these advantages go away - you can just create a token every minute, for any project this infrastructure might be responsible for. If I maintain control about my pipeline definition, I can again do a lot of work to limit damage. For example, if I am in control, I can make sure the stages running untrusted codes have as little access to secrets as possible, and possibly isolate them in bubblewrap, VMs, ..., minimize the code with access to publishing rights. Once I lose control about the pipeline structure, all that goes away. Just add a build step to push all information and secrets to mastodon in individual toots, yey. To me, this has very much raised questions about keeping pipeline definitions and code in one repository. Or at least, to keep a publishing/release process in there. I don't have a simple solution there, especially for OSS software with little infrastructure - it's not an easy topic. But with these supply chain attacks coming hot and fast every 2 weeks, it's something to think about.

darkamaul 1y ago

> That repo still contained a GitHub Actions secret — a npm token with broad publish rights. One of the advantages of Trusted Publishing [0] is that we no longer need long-lived tokens with publish rights. Instead, tokens are generated on the CI VM and are valid for only 15 minutes. This has already been implemented in several ecosystems (PyPI, npm, Cargo, Homebrew), and I encourage everyone to use it, it actually makes publishing a bit _easier_. More importantly, if the documentation around this still feels unclear, don’t hesitate to ask for help. Ecosystem maintainers are usually eager to see wider adoption of this feature. [0] https://docs.pypi.org/trusted-publishers/ https://docs.pypi.org/trusted-publishers/

procaryote 1y ago
What we need now is a flag in repos for projects that use this kind of thing, so you can easily prohibit dependencies that don't
realityking 1y ago
I had missed the announcement that this is now available for npm: https://github.blog/changelog/2025-07-31-npm-trusted-publishing-with-oidc-is-generally-available/ https://github.blog/changelog/2025-07-31-npm-trusted-publish... Guess I know what I’ll be doing this weekend.

mnahkies 1y ago

I think the point around incorporating MFA into the automated publishing flow isn't getting enough attention. I've got no problem with doing an MFA prompt to confirm publish by a CI workflow - but last I looked this was a convoluted process of opening a https tunnel out (using a third party solution) such that you could provide the code. I'd love to see either npm or GitHub provide an easy, out the box way, for me to provide/confirm a code during CI.

tcoff91 1y ago
Publishing a package involves 2 phases: uploading the package to npmjs, and making it availble to users. Right now these 2 phases are bundled together into 1 operation. I think the right way to approach this is to unbundle uploading the packages & publishing packages so that they're available to end-users. CI systems should be able to build & upload packages in a fully automated manner. Publishing the uploaded packages should require a human to log into npmjs's website & manually publish the package and go through MFA.
- mnahkies 1y ago
  Completely agree tbh, and that would be one of my preferred approaches should npm be the actor to implement a solution. I also think it makes sense for GitHub to implement the ability to mark a workflow as sensitive and requiring "sudo mode" (MFA prompt) to run. It's not miles away from what they already do around requiring maintainer approval to run workflows on PRs. Ideally both of these would exist, as not every npm package is published via GitHub actions (or any CI system), and not every GitHub workflow taking a sensitive action is publishing an npm package.
- deleted 1y ago
  [deleted]
- klysm 1y ago
  npm should require this with packages that have a large enough blast radius
  - 1y ago

1oooqooq 1y ago

if the ci job force pushed something deep in the git history tree?

waterTanuki 1y ago

Something somewhere needs to change because the status quo just isn't working. Yes, we can cheer on the benefit of OIDC tokens and zero-trust solutions in CI pipelines on HN all we want, but the fact is there's a significant number of library developers out there with millions of package downloads per week that will refuse to do anything about security until they're compromised or npm blocks them from publishing until they do. And then there's other non-sensical proposals like spelunking deep into projects some which could be over a decade old and just rip out all the dependencies until there's nothing but a standard library is left. Look, I'm all for a better std lib, I think reducing the number of dependencies we have is good. But just saying "you should reduce dependencies" will do nothing concrete to fix the problem which already exists, because it's much easier said than done. So either tens of thousands or hundreds of thousands of developers stop using npm, and everyone refactors their projects to add more code and strip dependencies, or npm starts enforcing things like 2FA and OIDC for package developers with over X number of weekly downloads, and blocks publishing for those that don't follow the new security rules. I think it's clear which solution is more practical to implement. The only other option is for npm to completely lose its reputation and then we wind up with XKCD 927 again.

zeroq 1y ago

For the last 10 years I've been advocating for manual releases. I've encountered a lot of backlash, but is it really that alien concept these days? CI/CD is cool, but between this and recent CF drama it seems we have a pretty solid evidence it can lead to a serious problems. I worked at a BigBank once where deployments to production required at least five people present at a time and a lot of theatrics, but at least we knew what we were deploying.

cyphar 1y ago
I completely agree. You are infinitely more likely to get implicated in some widespread attack due to bugs in GitHub Actions or your automated release scripts than have your local machine's local build and signing infrastructure attacked. I have yet to see any evidence that fancy CI/CD systems are better than good old fashioned tarballs and detached signatures. Bonus points for distribution packaging systems where they add an additional layer of review and separate validation of releases. People seem to gloss over that fact the "stodgy old-fashioned" rigamarole of Debian is part of the reason why the entire internet didn't pwned by the xz attack. At the very least you should require a human to sign the blobs before the release is actually published. (This isn't always enough if the attacker can add themselves to the maintainers list and sign with their own key, which is why the distribution packaging systems where they maintain their own trusted copy of upstream keyrings is far more preferable.) If your threat model boils down to "if my GitHub account gets attacked or even a single API key is leaked all of my users are fucked" then you really need to take a long look at a mirror and ask yourself if that is reasonable.