7 ms·
The Everything NPM Package
- navtoj 3y agoNPM still hasn't fixed the "*" package version bug on their end. https://www.youtube.com/watch?v=IzqtWTMFv9Y&t=465 https://www.youtube.com/watch?v=IzqtWTMFv9Y&t=465
- charcircuit 3y agoIt's intentional and is not a bug. "Fixing" it could cause someone's build to break by someone unpublishing a version.
- nosefurhairdo 3y agoIf you use wildcard to specify any version of a dependency you shouldn't be surprised if something breaks.
- charcircuit 3y agoThe same can be said about not pinning to a specific version as even some patch releases can break things or change performance characteristics.
- scubbo 3y agoThis, to me, is still one of the most astonishing things about the JavaScript ecosystem that everyone just accepts.
- Aeolun 3y agoWhat is your suggestion for improving it? I can accidentally publish a breaking bug in my patch release, and I might not notice.
- scubbo 3y agoI believe there is no process or tool that could reliably do so (see sibling comment[0]). Indeed, at some point you need to trust an author that what they are publishing is what they say they are publishing, and authors being fallible means that mistakes _might_ slip by. What I'm surprised by is the apparent cultural norm that this is just a regular everyday occurrence which entirely erodes any faith in the meaning of SemVer. Sure, we cannot 100% trust SemVer (because humans are fallible) - but there is a world of difference between trusting it ~99.9% and 0%. The JavaScript community (from the outside! I could be wrong!) seems to have simply accepted the 0% situation, and all the extra toil that goes along with it, rather than trying to raise the bar of its contributors to be better. [0] https://news.ycombinator.com/item?id=38906936 https://news.ycombinator.com/item?id=38906936
- Aeolun 3y agoI don’t think this is quite true. I can expect semver to work correctly in about 70% of all instances (working with JS/TS every day). Biggest issues are authors that keep their libraries at 0.x forever (every minor chance can be a breaking one) and the ones that release a new major version every other week. The times I do a minor update and something breaks are generally regarded as a bug by authors too.
- nosefurhairdo 3y agoNot a big deal, just specify exact dependency versions. Curious how you think other package managers handle this better than npm.
- scubbo 3y agoI'm not saying that other package managers handle it better - if authors wilfully misrepresent the state of their software, it is indeed not the remit of the package manager to correct them. If you started down that road, you'd probably end up with a library of tests (executed in the package manager's registry) to guarantee a non-breaking change, and at that point you have to trust the package author that the tests are indeed accurate, which is basically equivalent to trusting them to write the correct `version` string (unless you auto-generate the tests, which is an interesting idea but probably impractical). I'm saying that the fact that it is (apparently) the norm in JavaScript-world that authors will regularly publish breaking changes that are not advertised as such, and that that is just an acceptable everyday uncommentworthy inconvenience, is surprising to me. How do y'all get anything done if you can't even trust SemVer enough to automatically pull in minor/patch updates to dependencies!?
- bakkoting 3y agoIt's not common at all. It can happen, but it's very rare. And it's basically never intentional. In my experience the most common cause of breaking changes is accidentally breaking on older versions of the runtime, because the project is only running tests on the last version or two. Aside from that, the only notable example I can think of in the last year was a pretty subtle bug in what was supposed to be a pure performance optimization in a query language [1]. I think these are pretty representative, and not meaningfully worse than the experience in other languages. [1] https://github.com/estools/esquery/pull/138 https://github.com/estools/esquery/pull/138
- scubbo 3y agoHuh. I have got the wrong impression, then, from various blogs/articles which suggest never relying on SemVer because it's regarded as as-good-as-useless. Thanks for setting me straight!
- chatmasta 3y agoPinning to a specific version doesn't protect against the author unpublishing that version. The problem with the `*` bug is that it means you can stop anyone from unpublishing future versions of their package by simply creating a package that depends on it with a `*` identifier and publishing that to the registry.
- nosefurhairdo 3y ago> Pinning to a specific version doesn't protect against the author unpublishing that version. It does if your project is also in the npm public registry and the package you're dependent on is more than 72 hours old. https://docs.npmjs.com/policies/unpublish https://docs.npmjs.com/policies/unpublish
- sneak 3y agoGolang’s cryptographically verified module proxy cache solves this problem nicely.
- spion 3y agoA softer interpretation is possible. "*" should mean that the dependency cannot unpublish all packages and must have at least one version available.
- charcircuit 3y agoIf someone has an older version in the lock file this will break the build.
- spion 3y agoWhich they can fix by updating the lock file.
- int_19h 3y agoAs this package clearly demonstrates, it's a broken design.
- charcircuit 3y agoThe way unpublishing works is broken. It would be better if unpublish would just hide the version. Then it would not matter if someone unpublished something with dependencies.
- LelouBil 3y agoThat's how crates.io does it with yanked releases. It's removed from the index, and cargo will only download it using a pre-existing lock file
- francisduvivier 3y agoLol I can see trolls on reddit using this when people are asking for which npm npm package they should use. "Just install the everything package, then you will be sure to have the right package"
- troupo 3y agohttps://x.com/PatrickJS__/status/1743693931316191671 https://x.com/PatrickJS__/status/1743693931316191671 "accidentally broke NPM and all I got was this sweet permanent banner all over my Github (thats impossible to remove since they probably had to code it up last minute before removing the org/repo)"
- bsimpson 3y agoGitHub's support is surprisingly bad. When I was consulting for an R&D lab at eBay, we open sourced a bunch of our work in a GitHub org. It was sanctioned by eBay's OSPO; they even linked to it from their main open source directory. 7 years later, long after the team disbanded, someone in eBay's current legal team decided that the (now archival) org violated eBay's trademarks. For the last year+, every time I've opened GitHub, I've been met with the same undismissable banner. Since the only choice they give you is to contact support, I did. Unfortunately, their support team is not responsive, and has a completely separate notifications system. It took an inordinately long time for them to respond. (I have poor reception here so I can't check, but I think it was months.) Since I'm not in the habit of checking GitHub Support for new messages, when they eventually replied, I missed it. I had to start a whole new ticket. That too was months ago, and I still haven't heard back. So because I did some work for a skunkworks eBay team in 2015, the top 150px of my GitHub are unusable, and there's apparently nothing I can do about it until some call center decides to write me back.
- ruune 3y agoYou could probably code up a simple browser extension to hide the banner via CSS if it bothers you a lot. Still only a bad fix that shouldn't be necessary
- FireInsight 3y agoNo need to code anything up. uBo should be enough. And CSS could be injected using Stylus easily.
- philipwhiuk 3y ago
- Affric 3y agoWhat a dissatisfying non-apology of an apology. > First, just want to apologize about any difficulties this package has caused. No rationale. No shame. Just the word “apologize” in a sentence. Who downloaded it though? Surely as a dev if you download such a package it’s on you?
- Waterluvian 3y agoYeah. I’m honestly not sure where any of this “package chaos” actually exists. I mean… there’s incompetence everywhere in every language, so yes there. But I’ve yet to run into a friend of a friend who has a horror story about these dependencies.
- urbandw311er 3y agoDo you think he should be ashamed? Granted I may have overlooked something, but as far as I can tell it wasn’t an intentionally malicious act, it was a bit of a curious experiment. Seems rather inline with the HN values to me.
- Affric 3y agoShame is a spectrum. I don’t think he should flagellate himself until the end of time. I think that they should be a little bit embarrassed that they haven’t published what they believed the risk of the everything package was. Upon rereading the article I can see that the word “unintended” is actually not Patrick’s but the author of the recap’s word. Beyond that you seem to be ascribing benign intent. Reading it from the horses mouth [1] it doesn’t seems like they had any intent other than trying to find out if it could be done. In a world of worse is better creating the largest possible area of effect for your experiment seems to be a pretty easy way to amp up the consequences of your actions regardless of the risk. [1] https://uncenter.dev/posts/npm-install-everything/ https://uncenter.dev/posts/npm-install-everything/
- Retr0id 3y agoHe didn't kill someone's puppy, he just published some interesting data that others' code struggled to cope with. He was irresponsible perhaps, but I don't think he foresaw doing any real damage, and I don't think he needs to be especially sorry for it.
- superasn 3y agoI've seen a lot of people criticise npm and their policies but I've never come across a solution. Npm has its flaws and while there are such abuses like everything package, is-odd, left-pad, etc there are also many useful packages like vue, sortable, etc without which development will be a huge pain. So not asking rhetorically, if we had all the insight and knowledge we have now, how would you make it different?
- Bjartr 3y agoIIRC, the Maven crowd was criticizing npm's decisions from the get go because they chose to ignore many of the problems the Java community already solved a decade before.
- SahAssar 3y agoSo could you list those problems?
- Bjartr 3y agoOne issue is npm will allow arbitrary code to execute as part of an install script for a package, which allows a class of attacks that aren't possible in the maven world.
- richbell 3y agoNamespaces, for one.
- strken 3y agoNamespaces in Maven seem like they're clunky. The pseudo-DNS thing where the first section is an actual domain but the second is whatever you want is quite janky, as is not matching the namespace to the package. Plus domains are transferable themselves and it seems like a bad idea to use them as identifiers. Not to say that npm shouldn't have had namespaces by default, but I think there's good reason not to blindly do everything the way the Java community did.
- nubinetwork 3y ago[flagged]
- pvg 3y agoThese only make sense when there's significant discussion there otherwise they're just links to nowhere, e.g. https://news.ycombinator.com/item?id=38873944 https://news.ycombinator.com/item?id=38873944
- ruune 3y agoJust as a side note about the screenshot at the end. I think it's from this socket thing, but the supply chain security of a package that depends on literally anything on npm having a score of almost 50 really makes me think if that score is just artificially inflated on every other package. Can you even reach a score below 47?
- feross 3y agoFounder of socket here. npm has since unpublished the chunk packages that the 'everything' package depends on (or perhaps made them private), so those packages are no longer being taken into account in the package score. You're right that a package that depends on literally everything would absolutely have a score of 0 in our system.
- frabjoused 3y agoNPM as a soulless entity is easy to bash, while the creator of the package is a popular tech influencer so naturally has the support of the masses. If you’re going to complain about NPM, describe how you would solve it in their shoes.
- spion 3y agoSwitch to a "softer" interpretation of "*" that rather than blocking unpublishing of every version, allows unpublishing as long as one version is still available.
- dang 3y agoRecent and related: 'everything' blocks devs from removing their own NPM packages - https://news.ycombinator.com/item?id=38873944 https://news.ycombinator.com/item?id=38873944 - Jan 2024 (102 comments)
- ramesh31 3y agoAs an NPM user since NPM existed, all I can do is shake my head and laugh at this point.
- rubyissimo 3y agoDoes rubygems / go have "protections" against this? Is npm specifically vulnerable to this kind of thing? Or is it just a cultural elelemnt of npm that there are more micro-packages?
- klooney 3y agoGo is a little different here- originally it was totally decentralized, there was no central registry, just URLs. So you could depend on everything, no effect on other packages. No rules on publishing or unpublishing either, because you just get to run your repo the way you want. At some point, Russ Cox got the Fear about this, and now https://proxy.golang.org/ https://proxy.golang.org/ is an on by default, caching proxy in the middle. You can still delete your packages whenever you want to though.
- yreg 3y ago> The "everything" package, with its 5 sub-packages and thousands of dependencies, has essentially locked down the ability for authors to unpublish their packages. This situation is due to npm's policy shift following the infamous "left-pad" incident in 2016, where a popular package left-pad was removed, grinding development to a halt across much of the developer world. In response, npm tightened its rules around unpublishing, specifically preventing the unpublishing of any package that is used by another package. Has no one thought of that? It seems like it should have been obvious that such an absolute rule could be easily abused to troll the system at scale. Not sure if it's a problem though, perhaps all unpublishing requests should be reviewed by someone at the registry (and granted only when it makes sense).
- happens 3y agoMost articles say the page includes a Skyrim meme, but no one says what the meme is and I can't find anything relating to Skyrim on everything.npm.lol. This is very confusing to me.
- norskeld 3y ago[dead]
- SquidJack 3y agoThe article is totally misleading there is no storage space running out and system resource exhaustion. btw the total size is around 30MB or less than 50 The only thing is no one can unpublish the npm package because npm have policy if one package is depend on your package you can't unpublish it
- jasonjmcghee 3y agoI’m blown away by the reception of this article. It’s wildly low quality, generated SEO spam. > It was removed, but then reemerged under a different scope with over 33,000 sub-packages. It's like playing whack-a-mole with npm packages! > This whole saga is more than just a digital prank. It highlights the ongoing challenges in package management within the npm ecosystem. For developers, it's a reminder of the cascading effects of dependencies and the importance of mindful package creation, maintenance, and consumption. > As we navigate the open source world, incidents like the everything package remind us of the delicate balance between freedom and responsibility in open-source software.
- theshrike79 3y agoThe "delicate balance between X and Y" is an LLM tic[0]. Especially llama -based language models have a habit of ending any longer piece of text with a phrase like that. Source: have done a bunch of AI-assisted writing to develop my own skills and the tics and specific turns of phrases really pop out to me. [0] https://en.wikipedia.org/wiki/Tic https://en.wikipedia.org/wiki/Tic
- zbentley 3y agoMay well be. Ironically, the most common place I read the tic of ending a piece of persuasive with a deliberate, unconnected conclusion that doesn't persuade and instead equivocates or states a trivialism ... is in student papers or similarly graded-like-assignments rote work. Could be that there's a lot of that out there such that it's heavily represented in training data. Could just be a person doing a not-great writing job.
- leros 3y agoI don't see this package labeled as a dependent on my packages. Must not have gotten everything.