8 ms·

As an outsider to the npm ecosystem, reading this list of packages is astonishing. Why do js people import someone else's npm module for every little trivial th

As an outsider to the npm ecosystem, reading this list of packages is astonishing. Why do js people import someone else's npm module for every little trivial thing?

rglover 1y ago
You typically don't. But a lot of packages that you do install depend on smaller stuff like this under the hood (not necessarily good and obviously better handled with bespoke code in the package, but is is what it is).
- grishka 1y ago
  Then the question becomes, why do developers of larger libraries import someone else's module for every little trivial thing?
  - rglover 1y ago
    Because they don't have the slightest clue what they're doing.
    - zero_shift 1y ago
      It's not that either. There are a handful of important packages that are controlled by people who have consulting / commercial interests in OSS activity. These people have an incentive to inflate download numbers. There could be a collective push to move off these deps, but it takes effort and nobody has a strong incentive to be the first
  - SAI_Peregrinus 1y ago
    Sometimes it's not someone else's module, it's their own. They break up the big library into reusable components, and publish them all separately. Essentially taking DRY to an extreme: don't have private functions, make all your implementation details part of the public API & reuse them across projects.
paulddraper 1y ago
Which of these would you prefer to reimplement? Debug, chalk, ansi-styles? --- You can pretend like this is unique to JS ecosystem, but xz was compromised for 3 years.
- dsff3f3f3f 1y ago
  I wouldn't use debug or ansi-styles. They're not even remotely close to being worth adding a dependency. Obviously none of them are trustworthy now though.
  - skydhash 1y ago
    I wouldn’t even use chalk. Altering terminal output is easy. But it should be used sparingly.
    - dsff3f3f3f 1y ago
      You're right. I only looked at the source for debug and ansi-styles. After looking at chalk it's insanity to add that as a dependency as well.
      - kesor 1y ago
        And yet it has 300M weekly downloads. I am fairly sure that most of these are not because it is a direct dependency of people's projects, but rather it is a dependency of a dependency of a dependency.
        skydhash 1y ago
        I think expo and eas-cli (the expo build service) is using chalk. Never understood what those cli need colors for what can be easily done with proper spacing and some symbols.
- craftkiller 1y ago
  > You can pretend like this is unique to JS ecosystem, but xz was compromised for 3 years. Okay, but you're not suggesting that a compression algorithm is the same scale as "is-arrayish". I don't think everyone should need to reimplement LZMA but installing a library to determine if a value is an array is bordering on satire.
nine_k 1y ago
Having a module for every little trivial thing allows you to only bring these modules inside the JS bundle you serve to your client. If there's a problem in one trivial-thing function, other unrelated trivial things can still be used, because they are not bundled in the same package. A comprehensive library might offer a more neat DX, but you'd have to ship library code you don't use. (Yes, tree-shaking exists, but still is tricky and not widespread.)
- skydhash 1y ago
  Doesn’t the bundler already do tree shaking? Optimizing via dependency listing is very wrong.
  - tracker1 1y ago
    Tree shaking is less than reliable... for it to work well, all the dependencies need to be TS/ESModule imports/exports and even then may not shake out properly. It helps, but not as much as judicious imports. I've been using Deno more for my personal projects which does have a pretty good @std library, though I do think they should keep methods that simply pass through to the Deno runtime, and should probably support working in Node and Bun as well.
- palmfacehn 1y ago
  Things like this are good illustrations as to why many feel that the entire JS ecosystem is broken. Even if you have a standard lib included in a language, you wouldn't expect a bigger binary because of the standard lib. The JS solution is often more duct tape on top of a bad design. In this case tree shaking, which may or may not work as intended.
  - nine_k 1y ago
    This is because you cannot easily remove problematic stuff from the browser. It's actively being used by someone, so the vendors keep it, so it continues to be used. The process takes decades, literally. On the server side, of course, you can do whatever you like, see Node / Deno / Bun. But the code bundle size plays a minor role there.
thewebguyd 1y ago
Lack of a good batteries-included stdlib. You're either importing a ton of little dependencies (which then depend on other small libraries) or you end up writing a ton of really basic functionality yourself.
- rudedogg 1y ago
  This is the answer IMO. The number of targets and noise would be a lot less if JS had a decent stdlib or if we had access to a better language in the browser. I have no hope of this ever happening and am abandoning the web as a platform for interactive applications in my own projects. I’d rather build native applications using SDL3 or anything else.
  - imiric 1y ago
    To be fair, this is not a problem with the web itself, but with the Node ecosystem. It's perfectly possible to build web apps without relying on npm at all, or by being very selective and conservative about the packages you choose as your direct and transitive dependencies. If not by reviewing every line of code, then certainly by vendoring them. Yes, this is more inconvenient and labor intensive, but the alternative is far riskier and worse for users. The problem is with web developers themselves, who are often lazy, and prioritize their own development experience over their users'.
    - palmfacehn 1y ago
      I'm often surprised at the number of JS experts who struggle with the basics of the browser API. Instead of reasoning through the problem, many will reach for a framework or library.
      - skydhash 1y ago
        Especially with the MDN, an amazing resource.
      - PeterisP 1y ago
dist-epoch 1y ago
This is spreading everywhere, Rust, Python, ...
- grishka 1y ago
  Not Java, thankfully! Libraries containing 1-2 trivial classes do exist, but they're an exception rather than a rule. Might be that the process of publishing to Maven Central is just convoluted enough to deter the kinds of people who would publish such libraries.
  - Deukhoofd 1y ago
    Also because Java, .NET, etc. all have very expansive standard libraries. You don't need to import most stuff, as it's already built-in.
    - szatkus 1y ago
      I mean, Apache Commons are still widely used. But it's just a handful of libraries maintaned by one organisation.
    - tracker1 1y ago
      Very true... I'm more experienced with .Net, but usually when you bring in something, it's much more of a compositional library or framework for doing something... like a testing harness (XUnit), web framework (FastEndpoints), etc. No so much in terms of basic utilities, where the std library and extensions for LINQ cover a lot of ground, even if you aren't using LINQ expressions themselves.
    - kelvinjps10 1y ago
      But then you depend on Microsoft for everything. I prefer python where it's battery Included but you depend on a foundation
      - grishka 1y ago
        Hasn't .net been open-source for like 10 years?
jowea 1y ago
This conversation been a thing since at least the leftpad event. It's just how the js ecosystem works it seems. The default library is too small perhaps?
- raddan 1y ago
  Or the language is too braindead. `is-arrayish` should not even have to be a thing.
  - robrtsql 1y ago
    I agree that it doesn't need to exist, but as far as I can tell, almost no one depends on it directly. The only person using it is the author, who uses it in some other small libraries, which are then used in a larger, nontrivial library. I just created a Next.js app, saw that `is-arrayish` was in my node_modules, and tried to figure out how it got there and why. Here's the chain of dependencies: next > sharp > color > color-string > simple-swizzle > is-arrayish `next` uses `sharp` for image optimization. Seems reasonable. `sharp` uses `color` (https://www.npmjs.com/package/color https://www.npmjs.com/package/color) to convert and manipulate color strings. Again, that seems reasonable. This package is maintained by Qix-. Everything else in the chain (color-string > simple-swizzle > is-arrayish) is also maintained by Qix-. It's obnoxious to me that he feels it is necessary to have 80 different packages, but it would also be a substantial amount of effort for the other parties to stop relying on Qix-'s stuff entirely.
    - tkiolp4 1y ago
      That’s a tactic shitty maintainers do: write N dubious modules that no sane person would install. Write one or two valuable modules that import those N dubious modules.
felbane 1y ago
Extreme aversion to NIH syndrome, perhaps? I agree that it's weird. Sure, don't try to roll your own crypto library but the amount of `require('left-pad')` in the wild is egregious.
lukebechtel 1y ago
It's easier to find something frustrating in large code changes than in single line imports, even if the effective code being run is the same -- the PR review looks cleaner and safer to just import something that seems "trusted". I'm not saying it is safer, just to the tired grug brain it can feel safer.
zero_shift 1y ago
"JS people" don't, but certain key dependencies do, and there are social / OSS-political reasons why. Why do "Java people" depend on lowrie's itext? Remember the leftpad-esque incident he initiated in 2015?
austin-cheney 1y ago
I can provide you with some missing background as I was a prior full time JavaScript/TypeScript developer for 15 years. Most people writing JavaScript code for employment cannot really program. It is not a result of intellectual impairment, but appears to be more a training and cultural deficit in the work force. The result is extreme anxiety at the mere idea of writing original code, even when trivial in size and scope. The responses vary but often take the form of reused cliches of which some don't even directly apply. What's weird about this is that it is mostly limited to the employed workforce. Developers who are self-taught or spend as much time writing personal code on side projects don't have this anxiety. This is weird because the resulting hobby projects tend to be substantially more durable than products funded by employment that are otherwise better tested by paid QA staff. As a proof ask any JavaScript team at your employment to build their next project without a large framework and just observe how they respond both verbally and non-verbally.
- zero_shift 1y ago
  > The responses vary but often take the form of reused cliches of which some don't even directly apply. "It has been tested by a 1000 people before me" "What if there is an upstream optimisation?" "I'm just here to focus on Business Problems™" "It reduces cognitive load" --- Whilst I think you are exaggerating, I do recognise this phenomenon. For me, it was during the pandemic when I had to train / support a lot of bootcamp grads and new entrants to the career. They were anxious to perform in their new career and interpreted that as shipping tickets as fast as possible. These developers were not dumb but they had... like, no drive at all to engage with problems. Most programmers should enjoy problems, not develop a kind of bad feeling behind the eyes, or a tightness in their chest. But for these folks, a problem was a threat, of a bad status update at their daily Scrum. Dependencies are a socially condoned shortcut to that. You can use a library and look like a sensible and pragmatic engineer. When everyone around you appears to accept this as the norm, it's too easy to just go with the flow. I think it is a change in the psychological demographic too. This will sound fanciful. But tech used to select for very independent, stubborn, disagreeable people. Now, agreeableness is king. And what is more agreeable than using dependencies?
  - 1y ago
socalgal2 1y ago
Same reason they do in rust. The rust docs, a static site generator, pull in over 700 packages. Because it’s trivial and easy

As an outsider to the npm ecosystem, reading this list of packages is astonishing. Why do js people import someone else's npm module for every little trivial thing?

rglover 1y ago

You typically don't. But a lot of packages that you do install depend on smaller stuff like this under the hood (not necessarily good and obviously better handled with bespoke code in the package, but is is what it is).

grishka 1y ago
Then the question becomes, why do developers of larger libraries import someone else's module for every little trivial thing?
- rglover 1y ago
  Because they don't have the slightest clue what they're doing.
  - zero_shift 1y ago
    It's not that either. There are a handful of important packages that are controlled by people who have consulting / commercial interests in OSS activity. These people have an incentive to inflate download numbers. There could be a collective push to move off these deps, but it takes effort and nobody has a strong incentive to be the first
- SAI_Peregrinus 1y ago
  Sometimes it's not someone else's module, it's their own. They break up the big library into reusable components, and publish them all separately. Essentially taking DRY to an extreme: don't have private functions, make all your implementation details part of the public API & reuse them across projects.

paulddraper 1y ago

Which of these would you prefer to reimplement? Debug, chalk, ansi-styles? --- You can pretend like this is unique to JS ecosystem, but xz was compromised for 3 years.

dsff3f3f3f 1y ago
I wouldn't use debug or ansi-styles. They're not even remotely close to being worth adding a dependency. Obviously none of them are trustworthy now though.
- skydhash 1y ago
  I wouldn’t even use chalk. Altering terminal output is easy. But it should be used sparingly.
  - dsff3f3f3f 1y ago
    You're right. I only looked at the source for debug and ansi-styles. After looking at chalk it's insanity to add that as a dependency as well.
    - kesor 1y ago
      And yet it has 300M weekly downloads. I am fairly sure that most of these are not because it is a direct dependency of people's projects, but rather it is a dependency of a dependency of a dependency.
      - skydhash 1y ago
        I think expo and eas-cli (the expo build service) is using chalk. Never understood what those cli need colors for what can be easily done with proper spacing and some symbols.
craftkiller 1y ago
> You can pretend like this is unique to JS ecosystem, but xz was compromised for 3 years. Okay, but you're not suggesting that a compression algorithm is the same scale as "is-arrayish". I don't think everyone should need to reimplement LZMA but installing a library to determine if a value is an array is bordering on satire.

nine_k 1y ago

Having a module for every little trivial thing allows you to only bring these modules inside the JS bundle you serve to your client. If there's a problem in one trivial-thing function, other unrelated trivial things can still be used, because they are not bundled in the same package. A comprehensive library might offer a more neat DX, but you'd have to ship library code you don't use. (Yes, tree-shaking exists, but still is tricky and not widespread.)

skydhash 1y ago
Doesn’t the bundler already do tree shaking? Optimizing via dependency listing is very wrong.
- tracker1 1y ago
  Tree shaking is less than reliable... for it to work well, all the dependencies need to be TS/ESModule imports/exports and even then may not shake out properly. It helps, but not as much as judicious imports. I've been using Deno more for my personal projects which does have a pretty good @std library, though I do think they should keep methods that simply pass through to the Deno runtime, and should probably support working in Node and Bun as well.
palmfacehn 1y ago
Things like this are good illustrations as to why many feel that the entire JS ecosystem is broken. Even if you have a standard lib included in a language, you wouldn't expect a bigger binary because of the standard lib. The JS solution is often more duct tape on top of a bad design. In this case tree shaking, which may or may not work as intended.
- nine_k 1y ago
  This is because you cannot easily remove problematic stuff from the browser. It's actively being used by someone, so the vendors keep it, so it continues to be used. The process takes decades, literally. On the server side, of course, you can do whatever you like, see Node / Deno / Bun. But the code bundle size plays a minor role there.

thewebguyd 1y ago

Lack of a good batteries-included stdlib. You're either importing a ton of little dependencies (which then depend on other small libraries) or you end up writing a ton of really basic functionality yourself.

rudedogg 1y ago
This is the answer IMO. The number of targets and noise would be a lot less if JS had a decent stdlib or if we had access to a better language in the browser. I have no hope of this ever happening and am abandoning the web as a platform for interactive applications in my own projects. I’d rather build native applications using SDL3 or anything else.
- imiric 1y ago
  To be fair, this is not a problem with the web itself, but with the Node ecosystem. It's perfectly possible to build web apps without relying on npm at all, or by being very selective and conservative about the packages you choose as your direct and transitive dependencies. If not by reviewing every line of code, then certainly by vendoring them. Yes, this is more inconvenient and labor intensive, but the alternative is far riskier and worse for users. The problem is with web developers themselves, who are often lazy, and prioritize their own development experience over their users'.
  - palmfacehn 1y ago
    I'm often surprised at the number of JS experts who struggle with the basics of the browser API. Instead of reasoning through the problem, many will reach for a framework or library.
    - skydhash 1y ago
      Especially with the MDN, an amazing resource.
    - PeterisP 1y ago

dist-epoch 1y ago

This is spreading everywhere, Rust, Python, ...

grishka 1y ago
Not Java, thankfully! Libraries containing 1-2 trivial classes do exist, but they're an exception rather than a rule. Might be that the process of publishing to Maven Central is just convoluted enough to deter the kinds of people who would publish such libraries.
- Deukhoofd 1y ago
  Also because Java, .NET, etc. all have very expansive standard libraries. You don't need to import most stuff, as it's already built-in.
  - szatkus 1y ago
    I mean, Apache Commons are still widely used. But it's just a handful of libraries maintaned by one organisation.
  - tracker1 1y ago
    Very true... I'm more experienced with .Net, but usually when you bring in something, it's much more of a compositional library or framework for doing something... like a testing harness (XUnit), web framework (FastEndpoints), etc. No so much in terms of basic utilities, where the std library and extensions for LINQ cover a lot of ground, even if you aren't using LINQ expressions themselves.
  - kelvinjps10 1y ago
    But then you depend on Microsoft for everything. I prefer python where it's battery Included but you depend on a foundation
    - grishka 1y ago
      Hasn't .net been open-source for like 10 years?

jowea 1y ago

This conversation been a thing since at least the leftpad event. It's just how the js ecosystem works it seems. The default library is too small perhaps?

raddan 1y ago
Or the language is too braindead. `is-arrayish` should not even have to be a thing.
- robrtsql 1y ago
  I agree that it doesn't need to exist, but as far as I can tell, almost no one depends on it directly. The only person using it is the author, who uses it in some other small libraries, which are then used in a larger, nontrivial library. I just created a Next.js app, saw that `is-arrayish` was in my node_modules, and tried to figure out how it got there and why. Here's the chain of dependencies: next > sharp > color > color-string > simple-swizzle > is-arrayish `next` uses `sharp` for image optimization. Seems reasonable. `sharp` uses `color` (https://www.npmjs.com/package/color https://www.npmjs.com/package/color) to convert and manipulate color strings. Again, that seems reasonable. This package is maintained by Qix-. Everything else in the chain (color-string > simple-swizzle > is-arrayish) is also maintained by Qix-. It's obnoxious to me that he feels it is necessary to have 80 different packages, but it would also be a substantial amount of effort for the other parties to stop relying on Qix-'s stuff entirely.
  - tkiolp4 1y ago
    That’s a tactic shitty maintainers do: write N dubious modules that no sane person would install. Write one or two valuable modules that import those N dubious modules.

felbane 1y ago

Extreme aversion to NIH syndrome, perhaps? I agree that it's weird. Sure, don't try to roll your own crypto library but the amount of `require('left-pad')` in the wild is egregious.

lukebechtel 1y ago

It's easier to find something frustrating in large code changes than in single line imports, even if the effective code being run is the same -- the PR review looks cleaner and safer to just import something that seems "trusted". I'm not saying it is safer, just to the tired grug brain it can feel safer.

zero_shift 1y ago

"JS people" don't, but certain key dependencies do, and there are social / OSS-political reasons why. Why do "Java people" depend on lowrie's itext? Remember the leftpad-esque incident he initiated in 2015?

austin-cheney 1y ago

I can provide you with some missing background as I was a prior full time JavaScript/TypeScript developer for 15 years. Most people writing JavaScript code for employment cannot really program. It is not a result of intellectual impairment, but appears to be more a training and cultural deficit in the work force. The result is extreme anxiety at the mere idea of writing original code, even when trivial in size and scope. The responses vary but often take the form of reused cliches of which some don't even directly apply. What's weird about this is that it is mostly limited to the employed workforce. Developers who are self-taught or spend as much time writing personal code on side projects don't have this anxiety. This is weird because the resulting hobby projects tend to be substantially more durable than products funded by employment that are otherwise better tested by paid QA staff. As a proof ask any JavaScript team at your employment to build their next project without a large framework and just observe how they respond both verbally and non-verbally.

zero_shift 1y ago
> The responses vary but often take the form of reused cliches of which some don't even directly apply. "It has been tested by a 1000 people before me" "What if there is an upstream optimisation?" "I'm just here to focus on Business Problems™" "It reduces cognitive load" --- Whilst I think you are exaggerating, I do recognise this phenomenon. For me, it was during the pandemic when I had to train / support a lot of bootcamp grads and new entrants to the career. They were anxious to perform in their new career and interpreted that as shipping tickets as fast as possible. These developers were not dumb but they had... like, no drive at all to engage with problems. Most programmers should enjoy problems, not develop a kind of bad feeling behind the eyes, or a tightness in their chest. But for these folks, a problem was a threat, of a bad status update at their daily Scrum. Dependencies are a socially condoned shortcut to that. You can use a library and look like a sensible and pragmatic engineer. When everyone around you appears to accept this as the norm, it's too easy to just go with the flow. I think it is a change in the psychological demographic too. This will sound fanciful. But tech used to select for very independent, stubborn, disagreeable people. Now, agreeableness is king. And what is more agreeable than using dependencies?
- 1y ago

socalgal2 1y ago

Same reason they do in rust. The rust docs, a static site generator, pull in over 700 packages. Because it’s trivial and easy