21 ms·

Npm operational incident, 6 Jan 2018

323 points by bradleyboy 9y ago

simooooo 9y ago
They need to pull their socks up and start signing these packages
- busterarm 9y ago
  Been discussed. Repeatedly. For years. Can't understand why it hasn't happened still.
  - mdekkers 9y ago
    Can't understand why it hasn't happened still. https://github.com/npm/npm/pull/4016 https://github.com/npm/npm/pull/4016 - I read this as "it´s hard, and we don´t care." took them forever to look at and reject a patch that would have gotten the ball rolling
- regecks 9y ago
  What would signed packages have achieved in this incident? Who needs to sign the packages? The developers of the packages, right? Otherwise developer credential compromise subverts the entire point. Which keys are authorized to sign for which packages? How to prevent credential compromise from affecting those authorizations? What's the difference between a signed legitimate package and a signed malicious package? If they introduced package signing, would packages would adopt it fast enough for users of packages to only use signed packages? What happens when signing keys are lost or compromised? Do we need to use countersignatures from timestamping services as with other forms of code signing, so that CI systems do not break if a key is pulled? I think this is very much not straightforward. npm+pgp may be well intentioned but seems grossly inadequate.
  - 0x0 9y ago
    Android and the Google Play store works pretty well. The systems "locks on to" a public key signature so further updates for a given app must be signed with the same key. That would fix the problem with reproducible builds here. Only adding new top level packages to a project for the first time is at risk then.
  - revelation 9y ago
    No one is asking for perfect, just an ssh style warning of "the key of the system you are connecting to changed from last time" would be a good start. Implementing that is trivial, straightforward, requires zero setup and would have easily prevented the copycats here.
FunnyLookinHat 9y ago
"We don’t discuss all of our security processes and technologies in specific detail for what should be obvious reasons..." - Security by obfuscation? That's not security. Your protocols and processes should stand up even if made public.
- platz 9y ago
  What is your home address and phone number?
  - deleted 9y ago
    [deleted]
  - samastur 9y ago
    445 12th Street, SW Washington, DC 20554 1-202-418-1440
    - iaw 9y ago
      That's probably an illegal sublet fyi.
      - greenleafjacob 9y ago
        It’s the headquarters of the FCC.
    - dang 9y ago
      Please don't do this here.
  - akerro 9y ago
    The correct question should be: "what firewall and IDS do you use?"
  - reificator 9y ago
    > What is your home address and phone number? Neither of those are security systems. That's like asking for their password and claiming you've proven security through obscurity. The idea behind dismissing obscurity is that if everything but private tokens were exposed, your system should still remain secure.
smt88 9y ago
As others have said countless times on the original thread: new packages from different users should never have been allowed to replace the missing packages. Even if a package is removed as malware, a user should never unexpectedly download the work of a totally new author that they haven't vetted. Some of the protections should be done on the user side, but that's no reason for NPM to have dangerous policies.
- allover 9y ago
  > but that's no reason for NPM to have dangerous policies. Oversight != 'dangerous policies'. Seems like they thought they had this fixed.
Shank 9y ago
> no malicious actors were involved in yesterday’s incident, and the security of npm users’ accounts and the integrity of these 106 packages were never jeopardized Maybe not in the incident itself, but the sheer fact that many of the packages were replaced by other people constitutes a jeopardization of applications that depend on NPM. The only reason why some big package didn't get replaced with code that exfiltrated data from production or subtly backdoored it is sheer luck.
- no29 9y ago
  it's beyond disingenuous and flat out dishonest for npm to say the integrity of the packages was never jeopardized. within minutes there were reports that packages had been replaced with questionable and malicious content. https://news.ycombinator.com/item?id=16087079 https://news.ycombinator.com/item?id=16087079 this is probably a good indication of how a much more serious security event would be treated by the organization.. just sayin
  - mannykannot 9y ago
    At the very least, Npm should explain why we can be sure the packages now offered are exactly the same as they were before the incident, if, in fact, it is in a position to make that claim.
  - strictnein 9y ago
    Yeah, I have no idea how they can even pretend to claim that.
kevinmannix 9y ago
I'm curious of the manual review process. Is this synchronous, i.e. immediately when publishing a package? Or is it after the fact, where suspicious code code have already been distributed? There are plenty UX trade-offs in either direction, of course.
0x0 9y ago
How can they claim no malicious actors were involved when packages such as duplexer3 were apparently replaced with undesirable code as reported in https://news.ycombinator.com/item?id=16087126 https://news.ycombinator.com/item?id=16087126 ?
- akerro 9y ago
  This is not a claim or a fact, that's probably an uneducated guess, npm is broken on so many levels they can't say anything for sure. [1] npm is full of fundamental bugs in their software, workflows, architecture, and community. We learnt this in March 2016 when one developers got a copyright claim and had to remove their package. [2] Why is it even possible to remove a package? Why is it possible to create new package with the same name? Almost no software published there have unit tests, asking developers how they tested it in a GitHub issue leaves the issue open with no response. Everything is centralized on GitHub, which was proven to fail many times when GitHub was down in a region or globally due to a DDoS. Why is Node allowed to exist? We're not in 1998 anymore. [1] https://news.ycombinator.com/item?id=16092584 https://news.ycombinator.com/item?id=16092584 (comment above) [2] https://www.theregister.co.uk/2016/03/23/npm_left_pad_chaos/ https://www.theregister.co.uk/2016/03/23/npm_left_pad_chaos/
  - detaro 9y ago
    At least get the company name right if you have to make guesses about what they know internally and what they don't. (lol, and edited it silently so my comment looks dumb now)
    - akerro 9y ago
      >We don’t discuss all of our security processes and technologies in specific detail for what should be obvious reasons, but here is a high-level overview > they know internally and what they don't. They know internally that they do a lot of things badly and this will result in more disasters for npm and JS communities, that's for sure. Check out how GitLab fixed their backup failure. They made new software for testing backups, they live-streamed their work on youtube, they made at least 2 write ups on this case. They were totally transparent about their mistakes and fixes, everyone applauded them for this. What did we get from npm?
tedivm 9y ago
> Unfortunately, the process was complicated by well-meaning members of the npm community who believed that a malicious actor or security breach was to blame and independently attempted to publish their own replacements for these packages. Ensuring the integrity of the affected packages required additional steps and time. That is such a bad response to this. The problem isn't that "well-meaning members of the community" decided to upload packages. The problem is that when their system decides that a package shouldn't be up it completely removes the package, as if it never existed, and allows the namespace to be reused immediately. Those "well-meaning members" should not even be able to hijack packages this way, as it means the people who aren't "well-meaning" can also do it. What should happen is that they block downloads of the package while they investigate. That way people who attempt to download the packages get a meaningful error and people are unable to hijack the package name.
- ivraatiems 9y ago
  > The problem isn't that "well-meaning members of the community" decided to upload packages. The problem is that when their system decides that a package shouldn't be up it completely removes the package, as if it never existed, and allows the namespace to be reused immediately. Those "well-meaning members" should not even be able to hijack packages this way, as it means the people who aren't "well-meaning" can also do it. Hasn't this been an ongoing issue with npm since pretty much its inception? I remember reading articles about this vulnerability and the hijacking of packages that were taken down temporarily years ago. How has this not been dealt with systematically yet?
  - tedivm 9y ago
    Yeah, about two years ago NPM stole a package name from an existing user and gave it to a company to use. The user then said that if he can't trust NPM to actually treat package naming fairly then he was just going to delete all of his packages[1]. This broke a ton of packages on people (in part due to "left-pad" disappearing), so the community went ahead and registered/uploaded all of the packages to NPM again. Afterwords NPM came out with a blog post[2] where they went out of their way to take as little blame as possible and basically said it was the developers fault. They said they "stand by [their] package name dispute resolution policy, and the decision to which it led"- basically ensuring that no developer should ever trust their repository in the long term, as they'll happily hand over any package name to a corporate entity if that entity asks for it. The weird thing is that they claimed to make it impossible to "unpublish" packages, so that developers could no longer rage quit their site, but apparently they didn't extend that new requirement to their own "security" systems. [1] http://azer.bike/journal/i-ve-just-liberated-my-modules/ http://azer.bike/journal/i-ve-just-liberated-my-modules/ [2] http://blog.npmjs.org/post/141577284765/kik-left-pad-and-npm http://blog.npmjs.org/post/141577284765/kik-left-pad-and-npm
Jare 9y ago
If I was managing npm-using systems, this press release would not put my fears to rest. They need to publish a full review of the contents of packages replacements that were uploaded (even if just for 5 minutes), and publish those replacements in a safe form so they can be reviewed personally by any concerned npm user (any user - not just those who downloaded the replacements through npm itself).
incadenza 9y ago
Am I missing something in thinking that a reasonable solution is to temporarily block downloads of potentially malicious packages? Why remove them? EDIT: or better yet, don’t allow people to download the update, just keep users on the old, ostensibly safe version until everything gets sorted?
wakeywakeywakey 9y ago
The only thing missing from this post is 'we take security very seriously'
Lazare 9y ago
Very displeased about this response. > In yesterday’s case, we got it wrong, which prevented a publisher’s legitimate code from being distributed to developers whose projects depend on it. We identified the error within five minutes and followed defined processes to reverse this block. Unfortunately, the process was complicated by well-meaning members of the npm community who believed that a malicious actor or security breach was to blame and independently attempted to publish their own replacements for these packages. No. Assuming everything in that excerpt is true (and I happen to know it's not, but that's not even relevant here), that wasn't the problem. The problem is that NPM allowed packages to be re-uploaded by new authors after the initial versions had been spam filtered. Especially since allowing packages to be re-uploaded by new authors was the core issue of the left-pad debacle, and the one thing NPM said they'd fixed in response. Let's summarise here: 1. NPM has a big issue 2. They claimed they had fixed it 3. They had not 4. In their post mortem they're pretending the issue doesn't exist This guts any remaining trust I had in npm. Even if I wanted to trust them, they're not even admitting the problem exists; how am I meant to believe they're finally going to fix it? They've stopped even promising to fix this, and moved on to lies and denial. Unacceptable. Literally. This is pushing me away from the node ecosystem because I am not prepared to accept this sort of weaponized incompetence from the primary package repo for node.
- pwaai 9y ago
  With all that money they've raised, it's puzzling why they don't take PR seriously. I guess they have a mentality of "where the fuck else are you gonna go, the node community is tightly coupled to our service"
  - auggierose 9y ago
    They are Javascript programmers ...
    - atmartins 9y ago
      Not sure how that's relevant.
  - 9y ago
lend000 9y ago
I think a major security improvement for npm and other package managers using almost exclusively open-source/ non-compiled code is to require the source code be linked from a popular open-source platform like Github and then to take the package directly from there, to ensure the code can be audited and nothing else can be snuck in.
a2tech 9y ago
Npm isn't ready for production use. If you're going to use it you either need to ship the modules you need or run it against a private repo of modules.
- N_A_T_E 9y ago
  I agree in principal about not relying on npm during deployment. For a novice, what is the best way to ship modules?
  - md224 9y ago
    My first thought was “What about including node_modules in your version control?” Downside is that the repo would be much larger, but if it’s going to expand to that size on the client after running “npm install”, maybe that’s not a big deal? But it feels kinda wrong to do that...
    - acemarke 9y ago
      Committing your `node_modules` is a bad idea. The best answer is to either use an offline cache of package tarballs that have been committed to your repo, or an NPM caching proxy. Yarn has an "offline mirror" feature built in, and Shrinkpack is a tool that can create an offline mirror based on an npm-shrinkwrap.json file. I wrote about using Yarn's offline mirror here: http://blog.isquaredsoftware.com/2017/07/practical-redux-part-9-managing-dependencies/#managing-dependency-packages-for-offline-installation http://blog.isquaredsoftware.com/2017/07/practical-redux-par...
      - djsumdog 9y ago
        I know you can use JFrog/Artifactory to cache pulls of Jar files from maven repos, as well as Docker images. Does JFrog support node?
        acemarke 9y ago
        I haven't actually used an NPM caching proxy myself. It looks like Artifactory does support acting as an NPM proxy. The other tools I've heard of are Sinopia, Verdaccio, and local-npm. Addy Osmani has a related article at https://addyosmani.com/blog/using-npm-offline/ https://addyosmani.com/blog/using-npm-offline/ .
deleted 9y ago
[deleted]
cjbprime 9y ago
The last paragraph is good! The other paragraphs are bad. The two main ways they are bad are: 1) A system that detects "spam" and then allows for the complete removal of packages as if they never existed, allowing anyone to replace them, should never be described in the neutral terms used in this post. This system appears to be an existential threat to the company and project, and one of its largest mistakes. It won't take a long investigation to figure that out; it should be obvious today. 2) The claim that the security and integrity of these hijacked npm package names was not jeopardized appears to be 100% pants-on-fire false. If it is not false, I think npm's users are owed an explanation today of why it's false, rather than a bare assertion. I know it's hard to be in the hot seat. No animosity to any of the humans involved.
alexkavon 9y ago
I think it's time to replace npm in default Node installs. They've shown a history of negligence for package handling and procedure. Why should Node continue putting the trust of such a substantial part of not only their ecosystem but the JS ecosystem as well into such unreliable hands?
- sirius87 9y ago
  the official node.js Docker image comes with yarn as well as npm. but this is an issue with the npm registry, not the package manager shipped with node. both yarn and npm use the npm registry. EDIT: it may be worth debating why npm (a binary produced by npm, Inc) is shipped with node.js at all, but that's upto the node foundation
  - tatersolid 9y ago
    I’m pretty sure npm is shipped as Javascript source by node.js
    - sirius87 9y ago
      ah yes, you are correct. it isn't a binary
cobbzilla 9y ago
How difficult is it to run your own private npm repo? Looks like that's what security-conscious folks should be doing, given this response. Any pointers/gotchas/battle stories much appreciated.
- gedy 9y ago
  It's doable, and one very good option is Sonatype Nexus https://www.sonatype.com/download-oss-sonatype https://www.sonatype.com/download-oss-sonatype The one issue I've seen over the years is it seems npm Inc does not officially sanction on site npm repos (presumably to help monetize their private hosted repos), and finding info on how to set up a private repo is lacking.
gpsx 9y ago
I've used NPM only a little but this scares me. What are NPM alternatives?
- nabeards 9y ago
  This question is important: what are the alternatives to npm? Surely someone has something out there with code signing and single-use package names for node/JS?
- Fr4nc3sc0NL 9y ago
  There aren't any. Yarn is a better package manager than the npm client, but the problem is in the npm registry, and that's the same for both yarn and npm..
- allover 9y ago
  NPM shouldn't scare you. Simple fact is you should not be relying on ANY package registry at the time of deployment.
  - reificator 9y ago
    > NPM shouldn't scare you. It absolutely should, just like any dependency on any other third party code or servers. Especially when they regularly have incidents like this. > Simple fact is you should not be relying on ANY package registry at the time of deployment. This is true, but doesn't prove the previous claim.
    - allover 9y ago
      > NPM shouldn't scare you. >> It absolutely should, just like any dependency on any other third party code or servers There's no need to be scared of any of those things if you understand the trade-offs and risks. >> Especially when they regularly have incidents like this. They don't. >> Simple fact is you should not be relying on ANY package registry at the time of deployment. > This is true, but doesn't prove the previous claim. The 'previous claim' follows from it. If you don't treat a package registry like an essential part of your own infrastructure, there's no need to be scared of it.
fencepost 9y ago
Independent of npm's issues now and in the past, how vulnerable are other package managers to similar problems? There were typosquatting issues with pip, replacement issues with Rubygems, probably others that I don't recall. What's the current state of things for the more commonly used languages?
- _pmf_ 9y ago
  Maven/Gradle don't allow un-namespaced packages; nobody knows for sure how deep up their arse npm's designers have had their ass when they made non-namespaced packages possible (even the only way in the beginning; I don't know if it's still the default). Additionally, there's the problem of npm's fine-grained dependencies (that have become a laughing stock in itself), in my Java large projects, while I don't know every single dependency, I know all the "vendors", i.e. trusted upstream namespaces I use, and their transitive dependencies. There's the occasional non-Eclipse, non-Google, non-Apache dependency (say, XStream), but I am aware of it and fix it to a specific version. npm with its thousands of dependencies by hundreds of pseudonymous developers for simple projects is an abomination.
  - vbrandl 9y ago
    `Crates.io` (the Rust package ecosystem) doesn't have namespaces and operates after the first come first serve principle. They won't resolve namesquatting issues so something like the left-pad debacle won't happen. I think their policy[0] is quite sane. At least I'm not aware of any issues of that kind regarding crates.io. For typosquatting: I don't think that problem can be easily solved by package registries. They could warn you if there is a package with a similar name (and more votes, downloads, what ever) but aside from that, the developer should double check the dependency names and that's it. [0]: https://crates.io/policies https://crates.io/policies
dboreham 9y ago
Lesson : in a name space, never delete records. Mark them as unused, invalid, inactive, "deleted" but never actually delete.
knodi 9y ago
The pleasures of working with Node.js
glenjamin 9y ago
Lots of people are piling onto npm here. This reads to me like a fairly simple unintended consequence of what seemed like a good approach. Automated spam filters help to avoid dodgy packages. Spam filtering operates on heuristics so it’s sensible to not publicise how they work. The automated spam filter kills dodgy uploads, as these mostly happen on previously unused names a decision is made to not have the spam packages’ name remain taken. Among other things this stops the spammers from leaving all the good names blocked. The spam code gets a false positive and the above logic kicks in, leaving previously used names now available. This is noticed and corrected within a few hours. The npm team will likely improve their spam filtering heuristics and also ensure that formerly good packages that get spam flagged do not release their names - they have indicated roughly this on Twitter. This only just happened, and it’s the weekend, so I’d expect a full write up will be released during the week when they’ve had time to do a post-mortem and work through the salient points.
- Lazare 9y ago
  > This reads to me like a fairly simple unintended consequence of what seemed like a good approach. Yes, but you need to keep in mind: 1. This stuff is hideously important because getting it wrong is a incredibly serious security hole. Letting people publish a package with a name that other people are already requiring is a defcon 1 emergency. 2. They already had a major issue with this and said they had fixed it. You get, at most, one time when you can say "whups, we didn't realise how big an issue this was!". 3. It's their job to get it right. This is, literally, npm inc. It's not some hobbyists, or a service being provided by people in their spare time. So when you say: > The automated spam filter kills dodgy uploads, as these mostly happen on previously unused names a decision is made to not have the spam packages’ name remain taken. I have to disagree. They needed to check how many times the flagged package had been downloaded, and after the left-pad debacle, they knew (or should have known) that. They're not taking this seriously.
- deleted 9y ago
  [deleted]
Sephr 9y ago
> who believed that a malicious actor or security breach was to blame This was a security breach. Their anti-spam system should block repos by freezing the module name and returning blank files, not by deleting the entire module and subsequently allowing anyone to upload new modules. This is leftpad all over again. Update: Woah, so I was checking out my old NPM namespaces and apparently someone took control over https://www.npmjs.com/package/filesaver.js https://www.npmjs.com/package/filesaver.js
- djsumdog 9y ago
  It's not a "breach" technically. No malicious person caused the packages to be removed by circumventing authentication/authorization systems. It is a security issue though. A bug and serious security issue, yes, but not technically a breach.
  - jeremyjh 9y ago
    A breach didn't cause the issue, but there absolutely was a breach in response to NPM's incompetence. https://news.ycombinator.com/item?id=16087126 https://news.ycombinator.com/item?id=16087126
perlgeek 9y ago
A manual review misjudged 106 packages at the same time? That seems... dubious. Unless they have a bulk removal based on meta data, instead of careful analysis. Which would also be pretty dubious.
jchw 9y ago
Very disheartening to see that NPM has not grown from the kik or left-pad incidents. Users should not be able to republish on top of old package names without some kind of intervention. Of course, NPM's response to the kik/left-pad problem was also pretty awful. Make it so users can't delete packages. Cool. For those of you using NPM's private offerings, this also applies to you, so hope you don't care about cleaning up your private NPM registry content. Getting pretty tired of this. Their open source operation seems to suffer from poor handling of community and technical issues, at least from a high level. Their private registry operation is very lean on features, and also suffers from very confusing limitations. I'm surprised at how long it took there to be read-only API keys. Until last year, you literally had to give your CI keys that could publish to your organization if you wanted access to private packages. And you had to pay for an extra user for the privilege of doing it wrong. It's fixed now but it still blows my mind that it took so long. Aside from faster installation, it's actually a lot better to just use private Git URLs instead of NPM's private offerings. NPM will never properly take the fall. They will take just enough blame to seem responsible and then shift the majority of the poor decision making onto other actors in the community in their explanations. Third time in recent memory.
emerged 9y ago
I left an Ethereum related project precisely because of the house of cards feel of it being built off JS and npm modules. They're a great example of how a project can be exploited by a malicious module which could proceed to extract all the tokens. If my hat is black, I'm writing a daemon which monitors relevant npm modules and uploads subtlely modified versions if and when the possibility to do so occurs again. Particularly, but not only, targeting crypto currencies.
- strictnein 9y ago
  > If my hat is black, I'm writing a daemon which monitors relevant npm modules and uploads subtlely modified versions ... I started sketching out one yesterday, and then thought "What am I going to do with this? Probably just get in trouble" so I stopped.
bennofs 9y ago
Why does a spam system delete packages that have already been in use for quite some time? I could understand if it blocked some newly updated ones, but it seems like it has deleted already existing packages that also were used by other packages?
flavio81 9y ago
It is really another embarrasment for NPM, considering that this is not the first time something similar happens (see: left-pad issue, 23 march 2016) So they learnt zero from it. Then, WHEN ARE THEY GOING TO SIGN THOSE PACKAGES? Zero, nothing. They didn't learn from mistakes, they don't listen. NPM is still open to all sorts of malicious use. Besides NPM problems, yesterday many packages won't work because package "pinkie-promise" wasn't available. This is the full effective source code of pinkie-promise: module.exports = typeof Promise === 'function' ? Promise : require('pinkie'); Not just a NPM fault, but a fault of the js community as a whole for accepting systems made from hundreds of one-line packages, a sort of spaghetti code for the modern era.
- netsharc 9y ago
  So the kiddies haven't learned, huh? The whole left-pad debacle was because of a package that has 1 single function: it pads your string to a certain length by prefixing characters. (Writing this I wonder who made right-pad). And it broke almost everything.
- yoodenvranx 9y ago
  Someone should grab the 100 most used packages which contain less than 30 lines of code, merge it into one package and call it 'npm standard library'.
  - krapp 9y ago
    Guess I'll be that guy: https://xkcd.com/927/ https://xkcd.com/927/
    - i_am_nomad 9y ago
      I’ve gotten to the point where I recognize certain XKCD urls and no longer have to click on them to know which one it is. The ol’ 927, I see.
  - t3rmi 9y ago
    The problem would be maintaining and versioning this standard library.
sumitgt 9y ago
> We don’t discuss all of our security processes and technologies in specific detail for what should be obvious reasons, but here is a high-level overview. Isn't that the opposite of good practice? You shouldn't rely on obscurity. It's better to have the security processes out in the open so that it can be audited and flaws pointed out.
- siler 9y ago
  Yeah, I found this an important aspect of the article. I don't know anything about the culture at NPM, but this feels related to the mismanagement of communication with consumers/the public about the incident itself. i.e., being secretive for no good reason.
- xkcd-sucks 9y ago
  I think it's > We don’t discuss all of our security processes and technologies [with each other internally, so dumb stuff keeps happening]
- detaro 9y ago
  Discussing anti-spam/abuse systems in specific detail would be fairly unusual, can you name any larger operation that does? There's a lot of questions open that are non-specific to that though, so I'd agree that they could be sharing more.
vl-y 9y ago
Such explanation looks misleading to me. Even having the module detected as potentially malicious, why was it put into a state that such module doesn't exist, letting others publish package with the same name. For me it looks like there is something wrong at the npm side internally with the processes.
julienfr112 9y ago
Is it related to a package that write logs in colors and send visa card numbers to hackers ?
- slig 9y ago
  No, that article was a warning that a package like that could exist in the wild and nobody would notice. It doesn't actually exists, as the author states in the blog post.
andersriutta 9y ago
I'm trying to figure out whether node2nix [0] could help avoid package name/version hijacking vulnerabilities. Node2nix can convert a package-lock.json file to something called a Nix expression [1], like this [2], where each dependency has a checksum. The purpose is to make the Nix expression deterministic so that each Nix package is reproducible. [0] https://github.com/svanderburg/node2nix https://github.com/svanderburg/node2nix [1] https://nixos.org/nix/about.html https://nixos.org/nix/about.html [2] https://github.com/svanderburg/node2nix/blob/22d0c19575ecd19208cbdfbb54e8c2228c553bf7/node-packages.nix#L13 https://github.com/svanderburg/node2nix/blob/22d0c19575ecd19... Edit: fix link formatting
indigodaddy 9y ago
I've been meaning to set up a blog with hexojs for a few weeks now, and was seriously all set to install node and npm (don't do too much with node...), and start tinkering yesterday, and then I checked hacker news, and was like 'well sheeyit'...
- indigodaddy 9y ago
  I guess this would be safe enough probably: https://github.com/billryan/docker-hexo https://github.com/billryan/docker-hexo
indigodaddy 9y ago
Looks like they don't allow comments on their blog posts??... I guess with good reason.
- pvdebbe 9y ago
  They use Tumblr as their blog platform, and Tumblr AFAIK doesn't use comments.
  - dmitrygr 9y ago
    > They use Tumblr as their blog platform Wow! professionalism++
tzs 9y ago
From what I've read (I've not yet worked on anything that used npm), it appears that npm uses a single shared flat namespace for all packages. If I publish a package named "order_pizza" nobody else can name a package "order_pizza", unless mine gets deleted first, and then they can...and anyone who was using my "order_pizza" will now silently get the other developer's "order_pizza" the next time they build. Anyone know why they did it that way instead of giving each developer a subspace of the namespace that only that developer can publish under? I'm thinking of the way it works on Java, where my package would be net.tzs.order_pizza, or GitHub (and numerous others) where it would be tzs/order_pizza.
- always_good 9y ago
  I think it's annoying but also toxic to an ecosystem. You have to name your packages the same way you look for a domain name, people can squat on names, and it causes drama like how the 'kik' package was stolen from a developer and given to a company who never used it. Nobody would've cared it the package was "tzs/kik". For example, the Elm package system does this right. Another example of this failure is the Reddit subreddit system where a subreddit like /r/bitcoin makes it seem like the authority when it's a carefully moderated and biased subreddit. And nobody can really compete with it without getting a worse subreddit name like truebitcoin and bitcoin2 or something. Would be much more clear if it was /r/@theymos/bitcoin and /r/@spez/politics.
  - MikkoFinell 9y ago
    The failure is yours alone if you actually consider Reddit an authority on anything.
    - MikkoFinell 9y ago
      Downvoters: I of course meant an authority on anything ELSE than memes and karmawhoring. Sorry for not being clear about that.
      - always_good 9y ago
noobermin 9y ago
With this and leftpad etc., I think it's time for the tech community to remember Java and Oracle, Sourceforge, etc., or look it up if you are too young to remember. It is beneficial to learn from history instead of making the same mistakes all over again. Centralized trusted authorities cannot be trusted if they are private companies who don't answer to you. If you aren't paying them and you don't have a management position in said company, things like this will happen.
swang 9y ago
maybe everyone should wait for the full report before jumping to conclusions? from what it appears, the reason someone else was allowed to register the same package name was because their database didn't pull in the entries for that package. an error? yes. an issue? yes. but it is not a repeat of the same incidence of left-pad. i'm not going to pass judgement until i read the full report. i think at the very least, whatever mechanism they were using to copy/backup the database seems to have an issue with deleting entries and needs to be addressed
dpwm 9y ago
Others may be interested to note this blog post from about a week ago by the CTO relating to npm's outage handling [0]. I'm not sure if I would classify this as an outage, but this seems to be the official narrative [1]. Given that status page was updated relatively quickly, I'm guessing that at least something like this procedure kicked in. I am a little shocked at the brevity of the procedure. I am guessing by the blog post pointing out the brevity that I am meant to be shocked, but if I turned out that 310 word procedure for downtime I'd be pretty confident it was not good enough. I'm not confident that I would be able to read that document and understand what to do if I was the one on pager duty. Maybe it would be different if I worked there and perhaps the procedures are more granular and each of the stages well explained in other procedures. Maybe. I'm finding it hard to understand what npm does that is so different to other languages' package managers that it can have so many problems and such bad public relations. Linux distros have repositories that handle packages with far greater sizes and quite a few are purely voluntary efforts. They also manage to include package signing as well. I've known of a few hiccups, but none that been handled quite so bad as to completely undermine my trust in the tool altogether. Is there any compelling technical reason why we couldn't have a tool that works more like go get for node? For those that haven't used it, you just point it at a git repository. [0] http://ceejbot.tumblr.com/post/169045932039/how-to-handle-an-outage http://ceejbot.tumblr.com/post/169045932039/how-to-handle-an... [1] https://status.npmjs.org/incidents/41zfb8qpvrdj https://status.npmjs.org/incidents/41zfb8qpvrdj
kaikai 9y ago
A bunch of folks here are complaining that the explanation isn't good enough, doesn't go into enough detail, etc. Literally the second sentence is: > Early this coming week, we will share a full analysis and technical explanation of the incident. I'm looking forward to reading the final post-mortem, but in the mean time I really appreciate their update.
dzonga 9y ago
noob question : would using yarn have mitigated any of the risks of missing/broken packages ?
- therealasdf 9y ago
  No, both Yarn and npm use the npm registry.
swsieber 9y ago
Everybody is mad/disappointed at/by the NPM for their handling... Does this have any real impact on the NPM package adoption though? There is no alternative node module package directory, is there? It seems like any action they could have taken would have had exactly the same effect (well, maybe some prospective products don't adopt JS backends, but you have all webpack projects locked in pretty tight, right?)
- Lazare 9y ago
  I've been using gitpkg (https://github.com/ramasilveyra/gitpkg https://github.com/ramasilveyra/gitpkg) lately; it works pretty well. Basically lets you use a git repo as a package repo, instead of npm. Very little about how I use npm requires it to be centralized; I don't ever search npm looking for a package to install; I find a package I want and then look to install it.
  - dbbk 9y ago
    Couldn't you just use github IDs in your package.json directly? npm and yarn both have support for it.
    - Lazare 9y ago
      Short answer: Sure, but it works much better with gitpkg. Long answer: Sometimes you can do that, yes. But you'll run into issues with "modern" frontned toolchains and complicated packages where you want to write ES2017 code, transpile it with babel, bundle it with webpack, make a universal lib, etc., and push the result to your package repo, but not commit it to your git repo. Gitpkg handles all the wierd corner cases, and lets you keep build artifacts out of your git repo, and raw source out of your package repos. And if you use github you can even use the github IDs in your package.json, but the result is going to be a lot better. If you're just writing a plain one file package for use in node that doesn't need any processing, eh, no need for gitpkg.
whatever_dude 9y ago
Just when I started thinking NPM had turned the ship around after the previous incidents. Given the amount of people who depend on their service, this is extremely amateur.
throwaway66666 9y ago
Pay attention to the tags they themselves added to their report. Namely, "#SPAM IS WHY WE CAN'T HAVE NICE THINGS". Oh npm...
yunong 9y ago
Has the community discussed adding cryptographic signatures to npm packages? It's generally a good idea in any case, but this outage further highlights the security risks of a public registry where packages are not signed.
INTPenis 9y ago
Since everything uses HTTP these days I think they could use one of the 20x codes to signal when a package has been affected by spam management or suspension. That way the official clients would support it but you could easily ignore it if you wanted to.
z3t4 9y ago
idea: Peer reviewing packages with the ability to suggest a better name.

323 points by bradleyboy 9y ago

simooooo 9y ago

They need to pull their socks up and start signing these packages

busterarm 9y ago
Been discussed. Repeatedly. For years. Can't understand why it hasn't happened still.
- mdekkers 9y ago
  Can't understand why it hasn't happened still. https://github.com/npm/npm/pull/4016 https://github.com/npm/npm/pull/4016 - I read this as "it´s hard, and we don´t care." took them forever to look at and reject a patch that would have gotten the ball rolling
regecks 9y ago
What would signed packages have achieved in this incident? Who needs to sign the packages? The developers of the packages, right? Otherwise developer credential compromise subverts the entire point. Which keys are authorized to sign for which packages? How to prevent credential compromise from affecting those authorizations? What's the difference between a signed legitimate package and a signed malicious package? If they introduced package signing, would packages would adopt it fast enough for users of packages to only use signed packages? What happens when signing keys are lost or compromised? Do we need to use countersignatures from timestamping services as with other forms of code signing, so that CI systems do not break if a key is pulled? I think this is very much not straightforward. npm+pgp may be well intentioned but seems grossly inadequate.
- 0x0 9y ago
  Android and the Google Play store works pretty well. The systems "locks on to" a public key signature so further updates for a given app must be signed with the same key. That would fix the problem with reproducible builds here. Only adding new top level packages to a project for the first time is at risk then.
- revelation 9y ago
  No one is asking for perfect, just an ssh style warning of "the key of the system you are connecting to changed from last time" would be a good start. Implementing that is trivial, straightforward, requires zero setup and would have easily prevented the copycats here.

FunnyLookinHat 9y ago

"We don’t discuss all of our security processes and technologies in specific detail for what should be obvious reasons..." - Security by obfuscation? That's not security. Your protocols and processes should stand up even if made public.

platz 9y ago
What is your home address and phone number?
- deleted 9y ago
  [deleted]
- samastur 9y ago
  445 12th Street, SW Washington, DC 20554 1-202-418-1440
  - iaw 9y ago
    That's probably an illegal sublet fyi.
    - greenleafjacob 9y ago
      It’s the headquarters of the FCC.
  - dang 9y ago
    Please don't do this here.
- akerro 9y ago
  The correct question should be: "what firewall and IDS do you use?"
- reificator 9y ago
  > What is your home address and phone number? Neither of those are security systems. That's like asking for their password and claiming you've proven security through obscurity. The idea behind dismissing obscurity is that if everything but private tokens were exposed, your system should still remain secure.

smt88 9y ago

As others have said countless times on the original thread: new packages from different users should never have been allowed to replace the missing packages. Even if a package is removed as malware, a user should never unexpectedly download the work of a totally new author that they haven't vetted. Some of the protections should be done on the user side, but that's no reason for NPM to have dangerous policies.

allover 9y ago
> but that's no reason for NPM to have dangerous policies. Oversight != 'dangerous policies'. Seems like they thought they had this fixed.

Shank 9y ago

> no malicious actors were involved in yesterday’s incident, and the security of npm users’ accounts and the integrity of these 106 packages were never jeopardized Maybe not in the incident itself, but the sheer fact that many of the packages were replaced by other people constitutes a jeopardization of applications that depend on NPM. The only reason why some big package didn't get replaced with code that exfiltrated data from production or subtly backdoored it is sheer luck.

no29 9y ago
it's beyond disingenuous and flat out dishonest for npm to say the integrity of the packages was never jeopardized. within minutes there were reports that packages had been replaced with questionable and malicious content. https://news.ycombinator.com/item?id=16087079 https://news.ycombinator.com/item?id=16087079 this is probably a good indication of how a much more serious security event would be treated by the organization.. just sayin
- mannykannot 9y ago
  At the very least, Npm should explain why we can be sure the packages now offered are exactly the same as they were before the incident, if, in fact, it is in a position to make that claim.
- strictnein 9y ago
  Yeah, I have no idea how they can even pretend to claim that.

kevinmannix 9y ago

I'm curious of the manual review process. Is this synchronous, i.e. immediately when publishing a package? Or is it after the fact, where suspicious code code have already been distributed? There are plenty UX trade-offs in either direction, of course.

0x0 9y ago

How can they claim no malicious actors were involved when packages such as duplexer3 were apparently replaced with undesirable code as reported in https://news.ycombinator.com/item?id=16087126 https://news.ycombinator.com/item?id=16087126 ?

akerro 9y ago
This is not a claim or a fact, that's probably an uneducated guess, npm is broken on so many levels they can't say anything for sure. [1] npm is full of fundamental bugs in their software, workflows, architecture, and community. We learnt this in March 2016 when one developers got a copyright claim and had to remove their package. [2] Why is it even possible to remove a package? Why is it possible to create new package with the same name? Almost no software published there have unit tests, asking developers how they tested it in a GitHub issue leaves the issue open with no response. Everything is centralized on GitHub, which was proven to fail many times when GitHub was down in a region or globally due to a DDoS. Why is Node allowed to exist? We're not in 1998 anymore. [1] https://news.ycombinator.com/item?id=16092584 https://news.ycombinator.com/item?id=16092584 (comment above) [2] https://www.theregister.co.uk/2016/03/23/npm_left_pad_chaos/ https://www.theregister.co.uk/2016/03/23/npm_left_pad_chaos/
- detaro 9y ago
  At least get the company name right if you have to make guesses about what they know internally and what they don't. (lol, and edited it silently so my comment looks dumb now)
  - akerro 9y ago
    >We don’t discuss all of our security processes and technologies in specific detail for what should be obvious reasons, but here is a high-level overview > they know internally and what they don't. They know internally that they do a lot of things badly and this will result in more disasters for npm and JS communities, that's for sure. Check out how GitLab fixed their backup failure. They made new software for testing backups, they live-streamed their work on youtube, they made at least 2 write ups on this case. They were totally transparent about their mistakes and fixes, everyone applauded them for this. What did we get from npm?

tedivm 9y ago

> Unfortunately, the process was complicated by well-meaning members of the npm community who believed that a malicious actor or security breach was to blame and independently attempted to publish their own replacements for these packages. Ensuring the integrity of the affected packages required additional steps and time. That is such a bad response to this. The problem isn't that "well-meaning members of the community" decided to upload packages. The problem is that when their system decides that a package shouldn't be up it completely removes the package, as if it never existed, and allows the namespace to be reused immediately. Those "well-meaning members" should not even be able to hijack packages this way, as it means the people who aren't "well-meaning" can also do it. What should happen is that they block downloads of the package while they investigate. That way people who attempt to download the packages get a meaningful error and people are unable to hijack the package name.

ivraatiems 9y ago
> The problem isn't that "well-meaning members of the community" decided to upload packages. The problem is that when their system decides that a package shouldn't be up it completely removes the package, as if it never existed, and allows the namespace to be reused immediately. Those "well-meaning members" should not even be able to hijack packages this way, as it means the people who aren't "well-meaning" can also do it. Hasn't this been an ongoing issue with npm since pretty much its inception? I remember reading articles about this vulnerability and the hijacking of packages that were taken down temporarily years ago. How has this not been dealt with systematically yet?
- tedivm 9y ago
  Yeah, about two years ago NPM stole a package name from an existing user and gave it to a company to use. The user then said that if he can't trust NPM to actually treat package naming fairly then he was just going to delete all of his packages[1]. This broke a ton of packages on people (in part due to "left-pad" disappearing), so the community went ahead and registered/uploaded all of the packages to NPM again. Afterwords NPM came out with a blog post[2] where they went out of their way to take as little blame as possible and basically said it was the developers fault. They said they "stand by [their] package name dispute resolution policy, and the decision to which it led"- basically ensuring that no developer should ever trust their repository in the long term, as they'll happily hand over any package name to a corporate entity if that entity asks for it. The weird thing is that they claimed to make it impossible to "unpublish" packages, so that developers could no longer rage quit their site, but apparently they didn't extend that new requirement to their own "security" systems. [1] http://azer.bike/journal/i-ve-just-liberated-my-modules/ http://azer.bike/journal/i-ve-just-liberated-my-modules/ [2] http://blog.npmjs.org/post/141577284765/kik-left-pad-and-npm http://blog.npmjs.org/post/141577284765/kik-left-pad-and-npm

Jare 9y ago

If I was managing npm-using systems, this press release would not put my fears to rest. They need to publish a full review of the contents of packages replacements that were uploaded (even if just for 5 minutes), and publish those replacements in a safe form so they can be reviewed personally by any concerned npm user (any user - not just those who downloaded the replacements through npm itself).

incadenza 9y ago

Am I missing something in thinking that a reasonable solution is to temporarily block downloads of potentially malicious packages? Why remove them? EDIT: or better yet, don’t allow people to download the update, just keep users on the old, ostensibly safe version until everything gets sorted?

wakeywakeywakey 9y ago

The only thing missing from this post is 'we take security very seriously'

Lazare 9y ago

Very displeased about this response. > In yesterday’s case, we got it wrong, which prevented a publisher’s legitimate code from being distributed to developers whose projects depend on it. We identified the error within five minutes and followed defined processes to reverse this block. Unfortunately, the process was complicated by well-meaning members of the npm community who believed that a malicious actor or security breach was to blame and independently attempted to publish their own replacements for these packages. No. Assuming everything in that excerpt is true (and I happen to know it's not, but that's not even relevant here), that wasn't the problem. The problem is that NPM allowed packages to be re-uploaded by new authors after the initial versions had been spam filtered. Especially since allowing packages to be re-uploaded by new authors was the core issue of the left-pad debacle, and the one thing NPM said they'd fixed in response. Let's summarise here: 1. NPM has a big issue 2. They claimed they had fixed it 3. They had not 4. In their post mortem they're pretending the issue doesn't exist This guts any remaining trust I had in npm. Even if I wanted to trust them, they're not even admitting the problem exists; how am I meant to believe they're finally going to fix it? They've stopped even promising to fix this, and moved on to lies and denial. Unacceptable. Literally. This is pushing me away from the node ecosystem because I am not prepared to accept this sort of weaponized incompetence from the primary package repo for node.

pwaai 9y ago
With all that money they've raised, it's puzzling why they don't take PR seriously. I guess they have a mentality of "where the fuck else are you gonna go, the node community is tightly coupled to our service"
- auggierose 9y ago
  They are Javascript programmers ...
  - atmartins 9y ago
    Not sure how that's relevant.
- 9y ago

lend000 9y ago

I think a major security improvement for npm and other package managers using almost exclusively open-source/ non-compiled code is to require the source code be linked from a popular open-source platform like Github and then to take the package directly from there, to ensure the code can be audited and nothing else can be snuck in.

a2tech 9y ago

Npm isn't ready for production use. If you're going to use it you either need to ship the modules you need or run it against a private repo of modules.

N_A_T_E 9y ago
I agree in principal about not relying on npm during deployment. For a novice, what is the best way to ship modules?
- md224 9y ago
  My first thought was “What about including node_modules in your version control?” Downside is that the repo would be much larger, but if it’s going to expand to that size on the client after running “npm install”, maybe that’s not a big deal? But it feels kinda wrong to do that...
  - acemarke 9y ago
    Committing your `node_modules` is a bad idea. The best answer is to either use an offline cache of package tarballs that have been committed to your repo, or an NPM caching proxy. Yarn has an "offline mirror" feature built in, and Shrinkpack is a tool that can create an offline mirror based on an npm-shrinkwrap.json file. I wrote about using Yarn's offline mirror here: http://blog.isquaredsoftware.com/2017/07/practical-redux-part-9-managing-dependencies/#managing-dependency-packages-for-offline-installation http://blog.isquaredsoftware.com/2017/07/practical-redux-par...
    - djsumdog 9y ago
      I know you can use JFrog/Artifactory to cache pulls of Jar files from maven repos, as well as Docker images. Does JFrog support node?
      - acemarke 9y ago
        I haven't actually used an NPM caching proxy myself. It looks like Artifactory does support acting as an NPM proxy. The other tools I've heard of are Sinopia, Verdaccio, and local-npm. Addy Osmani has a related article at https://addyosmani.com/blog/using-npm-offline/ https://addyosmani.com/blog/using-npm-offline/ .

deleted 9y ago

[deleted]

cjbprime 9y ago

The last paragraph is good! The other paragraphs are bad. The two main ways they are bad are: 1) A system that detects "spam" and then allows for the complete removal of packages as if they never existed, allowing anyone to replace them, should never be described in the neutral terms used in this post. This system appears to be an existential threat to the company and project, and one of its largest mistakes. It won't take a long investigation to figure that out; it should be obvious today. 2) The claim that the security and integrity of these hijacked npm package names was not jeopardized appears to be 100% pants-on-fire false. If it is not false, I think npm's users are owed an explanation today of why it's false, rather than a bare assertion. I know it's hard to be in the hot seat. No animosity to any of the humans involved.

alexkavon 9y ago

I think it's time to replace npm in default Node installs. They've shown a history of negligence for package handling and procedure. Why should Node continue putting the trust of such a substantial part of not only their ecosystem but the JS ecosystem as well into such unreliable hands?

sirius87 9y ago
the official node.js Docker image comes with yarn as well as npm. but this is an issue with the npm registry, not the package manager shipped with node. both yarn and npm use the npm registry. EDIT: it may be worth debating why npm (a binary produced by npm, Inc) is shipped with node.js at all, but that's upto the node foundation
- tatersolid 9y ago
  I’m pretty sure npm is shipped as Javascript source by node.js
  - sirius87 9y ago
    ah yes, you are correct. it isn't a binary

cobbzilla 9y ago

How difficult is it to run your own private npm repo? Looks like that's what security-conscious folks should be doing, given this response. Any pointers/gotchas/battle stories much appreciated.

gedy 9y ago
It's doable, and one very good option is Sonatype Nexus https://www.sonatype.com/download-oss-sonatype https://www.sonatype.com/download-oss-sonatype The one issue I've seen over the years is it seems npm Inc does not officially sanction on site npm repos (presumably to help monetize their private hosted repos), and finding info on how to set up a private repo is lacking.

gpsx 9y ago

I've used NPM only a little but this scares me. What are NPM alternatives?

nabeards 9y ago
This question is important: what are the alternatives to npm? Surely someone has something out there with code signing and single-use package names for node/JS?
Fr4nc3sc0NL 9y ago
There aren't any. Yarn is a better package manager than the npm client, but the problem is in the npm registry, and that's the same for both yarn and npm..
allover 9y ago
NPM shouldn't scare you. Simple fact is you should not be relying on ANY package registry at the time of deployment.
- reificator 9y ago
  > NPM shouldn't scare you. It absolutely should, just like any dependency on any other third party code or servers. Especially when they regularly have incidents like this. > Simple fact is you should not be relying on ANY package registry at the time of deployment. This is true, but doesn't prove the previous claim.
  - allover 9y ago
    > NPM shouldn't scare you. >> It absolutely should, just like any dependency on any other third party code or servers There's no need to be scared of any of those things if you understand the trade-offs and risks. >> Especially when they regularly have incidents like this. They don't. >> Simple fact is you should not be relying on ANY package registry at the time of deployment. > This is true, but doesn't prove the previous claim. The 'previous claim' follows from it. If you don't treat a package registry like an essential part of your own infrastructure, there's no need to be scared of it.

fencepost 9y ago

Independent of npm's issues now and in the past, how vulnerable are other package managers to similar problems? There were typosquatting issues with pip, replacement issues with Rubygems, probably others that I don't recall. What's the current state of things for the more commonly used languages?

_pmf_ 9y ago
Maven/Gradle don't allow un-namespaced packages; nobody knows for sure how deep up their arse npm's designers have had their ass when they made non-namespaced packages possible (even the only way in the beginning; I don't know if it's still the default). Additionally, there's the problem of npm's fine-grained dependencies (that have become a laughing stock in itself), in my Java large projects, while I don't know every single dependency, I know all the "vendors", i.e. trusted upstream namespaces I use, and their transitive dependencies. There's the occasional non-Eclipse, non-Google, non-Apache dependency (say, XStream), but I am aware of it and fix it to a specific version. npm with its thousands of dependencies by hundreds of pseudonymous developers for simple projects is an abomination.
- vbrandl 9y ago
  `Crates.io` (the Rust package ecosystem) doesn't have namespaces and operates after the first come first serve principle. They won't resolve namesquatting issues so something like the left-pad debacle won't happen. I think their policy[0] is quite sane. At least I'm not aware of any issues of that kind regarding crates.io. For typosquatting: I don't think that problem can be easily solved by package registries. They could warn you if there is a package with a similar name (and more votes, downloads, what ever) but aside from that, the developer should double check the dependency names and that's it. [0]: https://crates.io/policies https://crates.io/policies

dboreham 9y ago

Lesson : in a name space, never delete records. Mark them as unused, invalid, inactive, "deleted" but never actually delete.

knodi 9y ago

The pleasures of working with Node.js

glenjamin 9y ago

Lots of people are piling onto npm here. This reads to me like a fairly simple unintended consequence of what seemed like a good approach. Automated spam filters help to avoid dodgy packages. Spam filtering operates on heuristics so it’s sensible to not publicise how they work. The automated spam filter kills dodgy uploads, as these mostly happen on previously unused names a decision is made to not have the spam packages’ name remain taken. Among other things this stops the spammers from leaving all the good names blocked. The spam code gets a false positive and the above logic kicks in, leaving previously used names now available. This is noticed and corrected within a few hours. The npm team will likely improve their spam filtering heuristics and also ensure that formerly good packages that get spam flagged do not release their names - they have indicated roughly this on Twitter. This only just happened, and it’s the weekend, so I’d expect a full write up will be released during the week when they’ve had time to do a post-mortem and work through the salient points.

Lazare 9y ago
> This reads to me like a fairly simple unintended consequence of what seemed like a good approach. Yes, but you need to keep in mind: 1. This stuff is hideously important because getting it wrong is a incredibly serious security hole. Letting people publish a package with a name that other people are already requiring is a defcon 1 emergency. 2. They already had a major issue with this and said they had fixed it. You get, at most, one time when you can say "whups, we didn't realise how big an issue this was!". 3. It's their job to get it right. This is, literally, npm inc. It's not some hobbyists, or a service being provided by people in their spare time. So when you say: > The automated spam filter kills dodgy uploads, as these mostly happen on previously unused names a decision is made to not have the spam packages’ name remain taken. I have to disagree. They needed to check how many times the flagged package had been downloaded, and after the left-pad debacle, they knew (or should have known) that. They're not taking this seriously.
deleted 9y ago
[deleted]

Sephr 9y ago

> who believed that a malicious actor or security breach was to blame This was a security breach. Their anti-spam system should block repos by freezing the module name and returning blank files, not by deleting the entire module and subsequently allowing anyone to upload new modules. This is leftpad all over again. Update: Woah, so I was checking out my old NPM namespaces and apparently someone took control over https://www.npmjs.com/package/filesaver.js https://www.npmjs.com/package/filesaver.js

djsumdog 9y ago
It's not a "breach" technically. No malicious person caused the packages to be removed by circumventing authentication/authorization systems. It is a security issue though. A bug and serious security issue, yes, but not technically a breach.
- jeremyjh 9y ago
  A breach didn't cause the issue, but there absolutely was a breach in response to NPM's incompetence. https://news.ycombinator.com/item?id=16087126 https://news.ycombinator.com/item?id=16087126

perlgeek 9y ago

A manual review misjudged 106 packages at the same time? That seems... dubious. Unless they have a bulk removal based on meta data, instead of careful analysis. Which would also be pretty dubious.

jchw 9y ago

Very disheartening to see that NPM has not grown from the kik or left-pad incidents. Users should not be able to republish on top of old package names without some kind of intervention. Of course, NPM's response to the kik/left-pad problem was also pretty awful. Make it so users can't delete packages. Cool. For those of you using NPM's private offerings, this also applies to you, so hope you don't care about cleaning up your private NPM registry content. Getting pretty tired of this. Their open source operation seems to suffer from poor handling of community and technical issues, at least from a high level. Their private registry operation is very lean on features, and also suffers from very confusing limitations. I'm surprised at how long it took there to be read-only API keys. Until last year, you literally had to give your CI keys that could publish to your organization if you wanted access to private packages. And you had to pay for an extra user for the privilege of doing it wrong. It's fixed now but it still blows my mind that it took so long. Aside from faster installation, it's actually a lot better to just use private Git URLs instead of NPM's private offerings. NPM will never properly take the fall. They will take just enough blame to seem responsible and then shift the majority of the poor decision making onto other actors in the community in their explanations. Third time in recent memory.

emerged 9y ago

I left an Ethereum related project precisely because of the house of cards feel of it being built off JS and npm modules. They're a great example of how a project can be exploited by a malicious module which could proceed to extract all the tokens. If my hat is black, I'm writing a daemon which monitors relevant npm modules and uploads subtlely modified versions if and when the possibility to do so occurs again. Particularly, but not only, targeting crypto currencies.

strictnein 9y ago
> If my hat is black, I'm writing a daemon which monitors relevant npm modules and uploads subtlely modified versions ... I started sketching out one yesterday, and then thought "What am I going to do with this? Probably just get in trouble" so I stopped.

bennofs 9y ago

Why does a spam system delete packages that have already been in use for quite some time? I could understand if it blocked some newly updated ones, but it seems like it has deleted already existing packages that also were used by other packages?

flavio81 9y ago

It is really another embarrasment for NPM, considering that this is not the first time something similar happens (see: left-pad issue, 23 march 2016) So they learnt zero from it. Then, WHEN ARE THEY GOING TO SIGN THOSE PACKAGES? Zero, nothing. They didn't learn from mistakes, they don't listen. NPM is still open to all sorts of malicious use. Besides NPM problems, yesterday many packages won't work because package "pinkie-promise" wasn't available. This is the full effective source code of pinkie-promise: module.exports = typeof Promise === 'function' ? Promise : require('pinkie'); Not just a NPM fault, but a fault of the js community as a whole for accepting systems made from hundreds of one-line packages, a sort of spaghetti code for the modern era.

netsharc 9y ago
So the kiddies haven't learned, huh? The whole left-pad debacle was because of a package that has 1 single function: it pads your string to a certain length by prefixing characters. (Writing this I wonder who made right-pad). And it broke almost everything.
yoodenvranx 9y ago
Someone should grab the 100 most used packages which contain less than 30 lines of code, merge it into one package and call it 'npm standard library'.
- krapp 9y ago
  Guess I'll be that guy: https://xkcd.com/927/ https://xkcd.com/927/
  - i_am_nomad 9y ago
    I’ve gotten to the point where I recognize certain XKCD urls and no longer have to click on them to know which one it is. The ol’ 927, I see.
- t3rmi 9y ago
  The problem would be maintaining and versioning this standard library.

sumitgt 9y ago

> We don’t discuss all of our security processes and technologies in specific detail for what should be obvious reasons, but here is a high-level overview. Isn't that the opposite of good practice? You shouldn't rely on obscurity. It's better to have the security processes out in the open so that it can be audited and flaws pointed out.

siler 9y ago
Yeah, I found this an important aspect of the article. I don't know anything about the culture at NPM, but this feels related to the mismanagement of communication with consumers/the public about the incident itself. i.e., being secretive for no good reason.
xkcd-sucks 9y ago
I think it's > We don’t discuss all of our security processes and technologies [with each other internally, so dumb stuff keeps happening]
detaro 9y ago
Discussing anti-spam/abuse systems in specific detail would be fairly unusual, can you name any larger operation that does? There's a lot of questions open that are non-specific to that though, so I'd agree that they could be sharing more.

vl-y 9y ago

Such explanation looks misleading to me. Even having the module detected as potentially malicious, why was it put into a state that such module doesn't exist, letting others publish package with the same name. For me it looks like there is something wrong at the npm side internally with the processes.

julienfr112 9y ago

Is it related to a package that write logs in colors and send visa card numbers to hackers ?

slig 9y ago
No, that article was a warning that a package like that could exist in the wild and nobody would notice. It doesn't actually exists, as the author states in the blog post.

andersriutta 9y ago

I'm trying to figure out whether node2nix [0] could help avoid package name/version hijacking vulnerabilities. Node2nix can convert a package-lock.json file to something called a Nix expression [1], like this [2], where each dependency has a checksum. The purpose is to make the Nix expression deterministic so that each Nix package is reproducible. [0] https://github.com/svanderburg/node2nix https://github.com/svanderburg/node2nix [1] https://nixos.org/nix/about.html https://nixos.org/nix/about.html [2] https://github.com/svanderburg/node2nix/blob/22d0c19575ecd19208cbdfbb54e8c2228c553bf7/node-packages.nix#L13 https://github.com/svanderburg/node2nix/blob/22d0c19575ecd19... Edit: fix link formatting

indigodaddy 9y ago

I've been meaning to set up a blog with hexojs for a few weeks now, and was seriously all set to install node and npm (don't do too much with node...), and start tinkering yesterday, and then I checked hacker news, and was like 'well sheeyit'...

indigodaddy 9y ago
I guess this would be safe enough probably: https://github.com/billryan/docker-hexo https://github.com/billryan/docker-hexo

indigodaddy 9y ago

Looks like they don't allow comments on their blog posts??... I guess with good reason.

pvdebbe 9y ago
They use Tumblr as their blog platform, and Tumblr AFAIK doesn't use comments.
- dmitrygr 9y ago
  > They use Tumblr as their blog platform Wow! professionalism++

tzs 9y ago

From what I've read (I've not yet worked on anything that used npm), it appears that npm uses a single shared flat namespace for all packages. If I publish a package named "order_pizza" nobody else can name a package "order_pizza", unless mine gets deleted first, and then they can...and anyone who was using my "order_pizza" will now silently get the other developer's "order_pizza" the next time they build. Anyone know why they did it that way instead of giving each developer a subspace of the namespace that only that developer can publish under? I'm thinking of the way it works on Java, where my package would be net.tzs.order_pizza, or GitHub (and numerous others) where it would be tzs/order_pizza.

always_good 9y ago
I think it's annoying but also toxic to an ecosystem. You have to name your packages the same way you look for a domain name, people can squat on names, and it causes drama like how the 'kik' package was stolen from a developer and given to a company who never used it. Nobody would've cared it the package was "tzs/kik". For example, the Elm package system does this right. Another example of this failure is the Reddit subreddit system where a subreddit like /r/bitcoin makes it seem like the authority when it's a carefully moderated and biased subreddit. And nobody can really compete with it without getting a worse subreddit name like truebitcoin and bitcoin2 or something. Would be much more clear if it was /r/@theymos/bitcoin and /r/@spez/politics.
- MikkoFinell 9y ago
  The failure is yours alone if you actually consider Reddit an authority on anything.
  - MikkoFinell 9y ago
    Downvoters: I of course meant an authority on anything ELSE than memes and karmawhoring. Sorry for not being clear about that.
    - always_good 9y ago

noobermin 9y ago

With this and leftpad etc., I think it's time for the tech community to remember Java and Oracle, Sourceforge, etc., or look it up if you are too young to remember. It is beneficial to learn from history instead of making the same mistakes all over again. Centralized trusted authorities cannot be trusted if they are private companies who don't answer to you. If you aren't paying them and you don't have a management position in said company, things like this will happen.

swang 9y ago

maybe everyone should wait for the full report before jumping to conclusions? from what it appears, the reason someone else was allowed to register the same package name was because their database didn't pull in the entries for that package. an error? yes. an issue? yes. but it is not a repeat of the same incidence of left-pad. i'm not going to pass judgement until i read the full report. i think at the very least, whatever mechanism they were using to copy/backup the database seems to have an issue with deleting entries and needs to be addressed

dpwm 9y ago

Others may be interested to note this blog post from about a week ago by the CTO relating to npm's outage handling [0]. I'm not sure if I would classify this as an outage, but this seems to be the official narrative [1]. Given that status page was updated relatively quickly, I'm guessing that at least something like this procedure kicked in. I am a little shocked at the brevity of the procedure. I am guessing by the blog post pointing out the brevity that I am meant to be shocked, but if I turned out that 310 word procedure for downtime I'd be pretty confident it was not good enough. I'm not confident that I would be able to read that document and understand what to do if I was the one on pager duty. Maybe it would be different if I worked there and perhaps the procedures are more granular and each of the stages well explained in other procedures. Maybe. I'm finding it hard to understand what npm does that is so different to other languages' package managers that it can have so many problems and such bad public relations. Linux distros have repositories that handle packages with far greater sizes and quite a few are purely voluntary efforts. They also manage to include package signing as well. I've known of a few hiccups, but none that been handled quite so bad as to completely undermine my trust in the tool altogether. Is there any compelling technical reason why we couldn't have a tool that works more like go get for node? For those that haven't used it, you just point it at a git repository. [0] http://ceejbot.tumblr.com/post/169045932039/how-to-handle-an-outage http://ceejbot.tumblr.com/post/169045932039/how-to-handle-an... [1] https://status.npmjs.org/incidents/41zfb8qpvrdj https://status.npmjs.org/incidents/41zfb8qpvrdj

kaikai 9y ago

A bunch of folks here are complaining that the explanation isn't good enough, doesn't go into enough detail, etc. Literally the second sentence is: > Early this coming week, we will share a full analysis and technical explanation of the incident. I'm looking forward to reading the final post-mortem, but in the mean time I really appreciate their update.

dzonga 9y ago

noob question : would using yarn have mitigated any of the risks of missing/broken packages ?

therealasdf 9y ago
No, both Yarn and npm use the npm registry.

swsieber 9y ago

Everybody is mad/disappointed at/by the NPM for their handling... Does this have any real impact on the NPM package adoption though? There is no alternative node module package directory, is there? It seems like any action they could have taken would have had exactly the same effect (well, maybe some prospective products don't adopt JS backends, but you have all webpack projects locked in pretty tight, right?)

Lazare 9y ago
I've been using gitpkg (https://github.com/ramasilveyra/gitpkg https://github.com/ramasilveyra/gitpkg) lately; it works pretty well. Basically lets you use a git repo as a package repo, instead of npm. Very little about how I use npm requires it to be centralized; I don't ever search npm looking for a package to install; I find a package I want and then look to install it.
- dbbk 9y ago
  Couldn't you just use github IDs in your package.json directly? npm and yarn both have support for it.
  - Lazare 9y ago
    Short answer: Sure, but it works much better with gitpkg. Long answer: Sometimes you can do that, yes. But you'll run into issues with "modern" frontned toolchains and complicated packages where you want to write ES2017 code, transpile it with babel, bundle it with webpack, make a universal lib, etc., and push the result to your package repo, but not commit it to your git repo. Gitpkg handles all the wierd corner cases, and lets you keep build artifacts out of your git repo, and raw source out of your package repos. And if you use github you can even use the github IDs in your package.json, but the result is going to be a lot better. If you're just writing a plain one file package for use in node that doesn't need any processing, eh, no need for gitpkg.

whatever_dude 9y ago

Just when I started thinking NPM had turned the ship around after the previous incidents. Given the amount of people who depend on their service, this is extremely amateur.

throwaway66666 9y ago

Pay attention to the tags they themselves added to their report. Namely, "#SPAM IS WHY WE CAN'T HAVE NICE THINGS". Oh npm...

yunong 9y ago

Has the community discussed adding cryptographic signatures to npm packages? It's generally a good idea in any case, but this outage further highlights the security risks of a public registry where packages are not signed.

INTPenis 9y ago

Since everything uses HTTP these days I think they could use one of the 20x codes to signal when a package has been affected by spam management or suspension. That way the official clients would support it but you could easily ignore it if you wanted to.

z3t4 9y ago

idea: Peer reviewing packages with the ability to suggest a better name.