8 ms·

Micro-libraries should never be used

180 points by nalgeon 2y ago

KaiserPro 2y ago
I suspect this might be seen as trolling, but why isn't there a standard lib of stuff like this? surely it can't be beyond the wit of programming kind to have a standard lib, or even layers of standard lib for Node? What is the argument for not having a standard lib, apart from download speed?
- qsort 2y ago
  I'm not taking an absolute position either way -- the devil is in the details -- but here's my steelman for the opposing view: When you put something in the standard library, it's harder to take it out, meaning that you're committing development resources to support the implementation. Furthermore things change: protocols and formats rise and fall in popularity and programming style evolves as the language changes (e.g. callbacks vs. promises in JS). Therefore the stdlib becomes where libraries go to die, and you'll always have a set of third party libraries that are "pseudo-standard", like NumPy in Python. Having a minimal stdlib lets you "free-market" the decision, letting the community effects take care of what is considered standard in the ecosystem, and lets you optimize its minimal surface, like what happened with C.
  - hinkley 2y ago
    But people don’t have to spend social capital to get it used at the next place they work.
- kwhitefoot 2y ago
  Why isn't there some kind of 'compiler/linker/stripper' that would collect the functions actually used and compile them into an application specific library? Yes I know that dynamic dispatch makes that difficult but the programmer does surely know which functions he wants to call. I sometimes hanker for a return to Fortran IV where every routine was separately compiled and the linker only put into the object code those that were referred to by something else.
  - mattlondon 2y ago
    Do you mean tree-shaking? I think basically all JavaScript "compilers" do this these days so that the code you serve is only the code you actually use.
  - samatman 2y ago
    Zig handles this with lazy compilation: code is parsed, but not even type checked, until the compiler reaches it. This can lead to the occasional rude surprise when finally reaching code I've been working on for awhile, but haven't yet connected to the rest of the project. But it means there's no need for tree shaking, because nothing gets in until it gets used. One of my favorite things about the language.
    - ArtixFox 2y ago
      its such a useful technique. tiny binaries ftw!!
- AdrianB1 2y ago
  There are many ways to look at this. Maybe the standard lib of such functions should be implemented as native functions in the language (1). Or as a standard external function library (2). Or people should copy-paste the functions they need and keep in their organization's function library (3). I am sure there are a few more options. I moved to option 3: in all my apps I include a function library that I build over the years, so I don't start from scratch every time. I deeply hate ("hate speech" example here) dependencies to libraries from all over the Internet due to security reasons, but I copy-paste code when needed to my library after I read, understand and check the code that I copy. The biggest advantage is that some of this code is better than what I could invent from scratch in a busy day and I save the time of doing it right. The disadvantage is there is no way to reward these authors that contribute to humankind. PS. My function library has functions mostly written by me, over 80%, but it includes code written by others. In my case, every time I need a function I check my existing library first, then analyze whether to write or copy.
  - richardwhiuk 2y ago
    I assume you correctly obey the licensing when you copy and paste it in....
    - AdrianB1 2y ago
      Yes. It is not worth not doing it.
- xmodem 2y ago
  Every non-trivial Java project I've worked on ends up depending on Google's Guava and some combination of Apache Commons libraries.
mkoubaa 2y ago
Why don't people just copypasta the library into their codebase with the license header intact and keep it out of npm
tptacek 2y ago
I think the JS library ecosystem is a debacle, but there's really only one point in this post that grabbed me: "Would future updates be useful? No. The library is so simple that any change to the logic would be breaking, and it is already clear that there are no bugs." Maybe what you want is a library ecosystem where things can be marked "this will never change". Something crazy happens and you actually need to update "is-number"? Rename it. Of course, you can simulate that with a single large omnibus dependency that everyone can trust that pulls all these silly micro-libraries in verbatim.
Lws803 2y ago
I would argue that "They should either be copy-pasted into your codebase" would cause more code liabilities and maintenance required further down the line. I've personally seen codebases with a ton of custom code, copy-pasted code, inspired implementations before and it was horrible to get them up to speed with the latest functionality / best practices. I agree that having too many micro-libraries might not be beneficial though, but perhaps look for larger, more well-established libraries that encompasses those functionalities :)
- quonn 2y ago
  What‘s an example for „latest functionality“ or „best practices“ that should or could possibly change for a function like leftPad and that would not automatically happen by virtue of being in the code base (such as formatting)?
- SOLAR_FIELDS 2y ago
  You don't even have to talk about hypotheticals when it comes to this "vendor everything instead" philosophy. This is basically how the C world works, a lot of which is driven by a general allergy to dependencies by embedded developers - partially out of necessity (space and overhead are MUCH more important constraints in embedded land) - but also partially out of cargo culting. I guess the opinion I'll share here is that I don't hear too many people arguing that the way embedded developers manage C libraries is at the forefront of how we should be handling and distributing code.
  - pton_xd 2y ago
    Many industries, like AAA game development, work this way. There's a good reason for it -- code quality and performance is important. Understanding what your code actually does and how it does it, is important. And as a result, AAA game software is able to push the boundaries of consumer compute performance. Micro-library-built software is generally just bloated crap that barely works. But it is fast to churn out, so there's that.
    - SOLAR_FIELDS 2y ago
gjsman-1000 2y ago
In my Laravel projects, there are a few packages of much more niche/hobbyist origin without corporate backing, some haven’t been updated for a while, and others are perfectly fine and don’t need much maintenance. Normally, packages are listed in my composer.json and stored in vendor/. For those packages, I created a separate folder called vendor_private/ which is part of my Git tree, put copies of these weird little packages in it, and set up my composer.json to consider that folder a repository. Works like a charm. My big important packages are still upstream. I can customize the little ones as needed to fit better, or have better code, and not worry about them going unmaintained. It’s also way quicker than copying the files individually out of the package and into the right places (along with updating Namespaces, configuration, etc.) Once in a while, I’ll go back and see if anything worthwhile has changed upstream - and so far, it never has.
AndyKelley 2y ago
Missing benefit: Libraries are a way to distribute and share the costs of labor, resulting in a more efficient ecosystem. This doesn't apply to micro-libraries, but it looks like that cost/benefit list is intended to cover libraries in general.
franciscop 2y ago
Seems a lot like the classic "I put only a couple of the strong advantages and enumerate everything I could think about as disadvantage". While I'm bias (I've done a bunch of these micro-libraries myself), there's more reasons I/OSS devs do them! To name other advantages (as a dev consuming them): - Documentation: they are usually well documented, at least a lot better than your average internal piece of code. - Portability: you learn it once and can use it in many projects, a lot easier than potentially copy/pasting a bunch of files from project to project (I used to do that and ugh what a nightmare it became!). - Semi-standard: everyone in the team is on the same page about how something works. This works on top of the previous two TBF, but is distinct as well e.g. if you use Axios, 50% of front-end devs will already know how to use it (edit: removed express since it's arguably not micro though). - Plugins: now with a single "source" other parties or yourself can also write plugins that will work well together. You don't need to do it all yourself. - Bugs! When there are bugs, now you have two distinct "entities" that have strong motivation to fix the bugs: you+your company, and the dev/company supporting the project. Linus's eyeballs and all (yes, this has a negative side, but those are also covered in the cons in the article already!). - Bugs 2: when you happen upon a bug, a 3rd party might've already found a bug and fixed it or offered an alternative solution! In fact I just did that today [1] That said, I do have some projects where I explicitly recommend to copy/paste the code straight into your project, e.g. https://www.npmjs.com/package/nocolor https://www.npmjs.com/package/nocolor (you can still install it though). [1] https://github.com/umami-software/node/issues/1#issuecomment-2295231274 https://github.com/umami-software/node/issues/1#issuecomment...
- pton_xd 2y ago
  Every team should eventually have some internal libraries of useful project-agnostic functionality. That addresses most of your points. Copy-paste the code into your internal library and maintain it yourself. Don't add a dependency on { "assert": "2.1.0" }. It probably doesn't do what you actually want, anyway. I think the more interesting point is that most projects don't know what they actually need and the code is disposable. In that scenario micro-libraries make some amount of sense. Just import random code and see how far you can get.
  - 2y ago
userbinator 2y ago
and because it updates fairly frequently I fail to comprehend how a single-function-library called "isNumber" even needs updating, much less "fairly frequently". The debate around third-party code vs. self-developed is eternal. IMHO if you think you can do better than existing solutions for your use-case, then self-developed is the obvious choice. If you don't, then use third-party. This of course says a lot about those who need to rely on trivial libraries.
- guestbest 2y ago
  I think the updates are more for bugfixes around edge cases than feature additions.
- foul 2y ago
  >I fail to comprehend how a single-function-library called "isNumber" even needs updating, much less "fairly frequently". If someone uses isNumber as a fundamental building block and surrogate for Elm or Typescript (a transpiler intermediate that would treat number more soundly I hope), this poor soul whom I deeply pity will encounter a lot of strange edge-cases (like that one stated in the article: NaN is a number or not?) and if they fear the burden of forking the library they will try to inflict this burden upstream, enabling feature or conf bloat. I insinuate that installation of isNumber is, like most of these basic microlibs, a symptom of incompetence in usage of the language. A worn JS dev would try isNaN(parseInt(num+'')) and sometime succeed.
  - flysand7 2y ago
    > [...] and sometime succeed Nothing is ever certain when you program in javascript.
- consteval 2y ago
  > I fail to comprehend how a single-function-library called "isNumber" even needs updating Never underestimate the complexity and footgunny nature of JS' type system.
ristos 2y ago
Micro-libraries are really good actually, they're highly modular, self-contained code, often making it really easy to understand what's going on. Another advantage is that because they're so minimal and self-contained, they're often "completed", because they achieved what they set out to do. So there's no need to continually patch it for security updates, or at least you need to do it less often, and it's less likely that you'll be dealing with breaking changes. The UNIX philosophy is also build on the idea of small programs, just like micro-libraries, of doing one thing and one thing well, and composing those things to make larger things. I would argue the problem is how dependencies in general are added to projects, which the blog author pointed out with left-pad. Copy-paste works, but I would argue the best way is to fork the libraries and add submodules to your project. Then if you want to pull a new version of the library, you can update the fork and review the changes. It's an explicit approach to managing it that can prevent a lot of pitfalls like malicious actors, breaking changes leading to bugs, etc.
- mattlondon 2y ago
  > Micro-libraries are really good actually, they're highly modular, self-contained code Well I think that is the point, they're not self-contained. You are adding mystery stuff and who knows how deep the chain of dependencies go. See the left-pad fiasco that broke so much stuff, because the chain of transitive dependencies ran deep and wide. NPM is a dumpster fire in this regard. I try to avoid it - is there a flag you can set to say "no downstream dependencies" or something when you add a dependency? At least that way you can be sure things really are self-contained.
  - IgorPartola 2y ago
    I remember adding a random date picker that pulled in a copy of React with it to a non-React project. NPM is a dumpster fire at a nuclear facility.
  - ristos 2y ago
    Yeah there's a way to do that, yarn and pnpm can flatten the dependency tree. You can add the fork directly too: yarn add <path/to/your/forked/micro-library.git> pnpm add <path/to/your/forked/micro-library.git>
qwerty456127 2y ago
> Micro-libraries should never be used. They should either be copy-pasted into your codebase, or not used at all. I would prefer them to be built straight in the languages.
- jdminhbg 2y ago
  Yes, everyone seems to take the wrong lesson from left-pad. The reason left-pad happened on NPM isn't that there's something uniquely wrong with how NPM was built, but that JS has a uniquely barren standard library. People aren't writing their own left-pad functions in Java or Go or Python, it's just in the stdlib.
  - _xiaz 2y ago
    At the same time Go is quite barren when it comes to list (slice) functions, but I largely agree with Java and Python
TacticalCoder 2y ago
Thought experiment: if a LLM can correctly produce the code for a micro-library like, say, leftpad... Should you call leftpad as a dependency or should you have the LLM generate that leftpad function for you? And if the LLM ain't good enough to write leftpad, how can I trust it to write anything at all?
joshmarinacci 2y ago
TL;DR most micro libraries should be gists
mirekrusin 2y ago
Personally I prefer sharing one level up from function to conceptual module, ie. instead of "left-pad" function, "string" module, ie. a bit like this [0] (`${authorshipDisclaimer}`). I'm also an advocate, against crowd, of qualified imports as they help with refactoring (renames are propagated, especially in monorepos), readability/reviews (functions are qualified, you know where they're coming from) and overall coding experience – qualified module name followed by dot gives good autocompletion, imports look neat in larger projects etc. The codebase written like this resembles extended standard library. It also helps with solving problems by encouraging first principle thinking, bottom up coding that produces auditable codebase with shallow external dependencies etc. [0] https://github.com/preludejs https://github.com/preludejs
xg15 2y ago
Micro libraries are worse than no libraries at all - but I maintain they are still better than gargantuan "frameworks" or everything-but-the-kitching-sink "util"/"commons" packages, where you end up only using a tiny fraction of the functionality but have to deal with the maintenance cost and attack surface of the whole thing. If you're particularly unlucky, the unused functionality pulls in transitive dependencies of its own - and you end up with libraries in your dependency tree that your code is literally not using at all. If you're even more unlucky, those "dead code" libraries will install their own event handlers or timers during load or will be picked up by some framework autodiscovery mechanism - and will actually execute some code at runtime, just not any code that provides anything useful to the project. I think an apt name for this would be "undead code". (The examples I have seem were from java frameworks like Spring and from webapps with too many autowired request filters, so I do hope that is no such an issue in JS yet)
- DonHopkins 2y ago
  Sometimes importing zombie "undead code" libraries can be beneficial! I just refactored a bunch of python computer vision code that used detectron2 and yolo (both of which indirectly use OpenCV and PyTorch and lots of other stuff), and in the process of cleaning up unused code, I threw out the old imports of the yolo modules that we weren't using any more. The yololess refactored code, which really didn't have any changes that should measurably affect the speed, ran a mortifying 10% slower, and I could not for the life of me figure out why! Benchmarking and comparing each version showed that the yololess version was spending a huge amount of time with multiple threads fighting over locks, which the yoloful code wasn't doing. But I hadn't changed anything relating to threads or locks in the refactoring -- I had just rearranged a few of the deck chairs on the Titanic and removed the unused yolo import, which seemed like a perfectly safe innocuous thing to do. Finally after questioning all of my implicit assumptions and running some really fundamental sanity checks and reality tests, I discovered that the 10% slow-down in detectron2 was caused by NOT importing the yolo module that we were not actually using. So I went over the yolo code I was originally importing line by line, and finally ran across a helpfully commented top-level call to fix an obscure performance problem: https://github.com/ultralytics/yolov5/blob/master/utils/general.py#L67 https://github.com/ultralytics/yolov5/blob/master/utils/gene... cv2.setNumThreads(0) # prevent OpenCV from multithreading (incompatible with PyTorch DataLoader) Even though we weren't actually using yolo, just importing it, executing that one line of code fixed a terrible multithreading performance problem with OpenCV and PyTorch DataLoader fighting behind the scenes over locks, even if you never called yolo itself. So I copied that magical incantation into my own detectron2 initialization function (not as top level code that got executed on import of course), wrote some triumphantly snarky comments to explain why I was doing that, and the performance problems went away! The regression wasn't yolo's or detectron2's fault per se, just an obscure invisible interaction of other modules they were both using, but yolo shouldn't have been doing anything globally systemic like that immediately when you import it without actually initializing it. But then I would have never discovered a simple way to speed up detectron2 by 10%! So if you're using detectron2 without also importing yolo, make sure you set the number of cv2 threads to zero or you'll be wasting a lot of money.
deleted 2y ago
[deleted]
qudat 2y ago
Partially disagree, JS has unique features that require small libraries: https://bower.sh/my-love-letter-to-front-end-web-development https://bower.sh/my-love-letter-to-front-end-web-development However, if I can inline a small function, I will, so in that sense I agree.
- crabmusket 2y ago
  > Javascript has a very unique set of challenges that differentiates itself from the rest of the programming world. The primary driving factor for its unique position is javascript is downloaded on the client's browser. Languages that run on the server, for example, don't need to send code that it runs onto client machines, they merely respond to requests made from the browser. This is profoundly true. JavaScript written for the frontend has different "physics" to backend code. It's not only code size that is significant. It's the fact that when you ship code over the wire to a client, you don't know what browser or even JS engine version will be interpreting it. Platform incompatibility has been a huge driver of issues in the JS/NPM ecosystem and has caused JS's culture to develop the way it has. I wrote more about this, link in a top level comment.
PhilipRoman 2y ago
Micro libraries are fine (well... not really), the problem starts when each of those depends on 10 more "micro" libraries and so forth. The branching factor quickly leads to bloat. Libraries have a duty to minimize their footprint in ways that applications do not.
rc_kas 2y ago
the entire nodejs ecosystem needs to die. You all are just keeping it alive.
- edwinjm 2y ago
  You can start by ignoring it
- crabmusket 2y ago
  Have fun convincing all major browser vendors to no longer support JS scripts :) (At this point Nodejs is the defacto tooling ecosystem for even JS destined to run in a browser. You can't separate the two.)
edfletcher_t137 2y ago
Where do you draw the line: is 10 lines "micro"? 50? 100? It is never quantified within, yet the very click-bait-y title relies on the term. How many bugs can hide in 50 lines or 100? And you really want to copy-paste that code at a static point in time?!
- IshKebab 2y ago
  I don't think they need to draw an exact line for us to know what they're talking about.
  - edfletcher_t137 2y ago
    The fact that we're even debating it shows that they do. And I didn't ask for "an exact line". I asked for anything. What is your qualification for a "micro" library? I'm betting it's different from 9 other respondents. That's a problem. Furthermore, that's not even the main contention I was highlighting. Without a proper definition, the advice of "just copy/paste these" is dangerous. Someone will draw their line at something too large, copy/paste that in and inherit bugs/vulnerabilities they never fix. That's a big problem.
edwinjm 2y ago
So, everybody can contribute an npm package. The advantage: - everybody can contribute an npm package The disadvantage: - everybody can contribute an npm package
unstable 2y ago
> You can write isNumber(foo) instead of typeof foo === "number". Indeed you can, but it depends what isNumber does. This is more like what it should do IMO: function isNumber( foo ) { return ( (typeof foo === "number") && (foo == foo)) || ((typeof foo === 'object') && (foo instanceof Number) ); } And that is I think the value of micro libs, at least in JS, you don't want to think about all the edge cases when you only want to check if something is a Number.
- IshKebab 2y ago
  Doesn't that exclude NaN (which you probably want despite the name)? I think this really highlights that you probably do want to think about those edge cases... In any case this is a bad example because Typescript exists.
  - unstable 2y ago
    Accepting NaN as a number can potentially crash your app, that's why I reject it in isNumber. I only tried to highlight some edge cases that I personally don't like to spend energy on, trying to get it right, when writing code. Btw, isNumber is a dynamic call in the example and unrelated to TypeScript. TypeScript doesn't exist at runtime.
- layer8 2y ago
  This is an argument for having a library that provides that function, but it is not an argument that it should be a micro-library.
- ristos 2y ago
  Reminds me of the 2ality blog post on that: https://2ality.com/2017/08/type-right.html https://2ality.com/2017/08/type-right.html
- crabmusket 2y ago
  This library is a hilarious example of a huge problem with this kind of package. "Number" is in the eye of the beholder. A string containing numeric characters is, in my view, in no useful way "a number". A package that treats it as such just perpetuates weakly-typed nonsense.* But the broader point is, you can't outsource understanding to a package. There will be places in your code where NaN is a perfectly valid number, or Infinity. And other places where you absolutely need to be sure neither of the above make their way in. By pretending that a package can capture the universal essence of "numberless", and that this will broadly apply across the entire JS ecosystem (see reported benefits like "different libraries can all rely on is-number instead of rewriting duplicated helper functions!") is naive. I wrote more about this in a post linked in a top level comment. The is-promise library is another great example. * Personal pet theory is that the package author would have been embarrassed to publish a 1-line package, so included "numeric strings are numbers" as a fig leaf to justify the package's existence. They should have instead created two new packages, is-actual-number and is-numeric-string, so the implementation of is-number could be nice and clean: module.exports = function(n) { return require('is-actual-number')(n) || require('is-numeric-string')(n); } I can feel the power of webscale coursing through me
replete 2y ago
Not checking dependency updates need to die already. Not choosing better dependencies needs to die already.
stevebmark 2y ago
[flagged]
- johnnyanmac 2y ago
  >This discussion is a complete waste of oxygen and bits. There is no point discussing this. Why?
crabmusket 2y ago
While I mainly agree with the author's substantive point, though I find some of the ways it's presented in this post not entirely convincing or fair, I am interested that someone else has identified this: > I have talked a lot about the costs of libraries, and I do hope people are more cautious about them. But there’s one factor I left out from my previous discussion. I think there’s one more reason why people use libraries: fear. > Programmers are afraid of causing bugs. Afraid of making mistakes. Afraid of missing edge cases. Afraid that they won’t be able to understand how things work. In their fear they fall back on libraries. “Thank goodness someone else has solved the problem; surely I never would have been able to.” I think this is true, but why does the JS ecosystem seem to have "more fear" than for example the Python ecosystem? I wrote about this a while ago. I think that actually JS does (or did) cause more fear in its developers than other programming languages. I described it as paranoia, a more insidious uncertainty. Quoting myself[1]: > There are probably many contributing factors that have shaped NPM into what it is today. However, I assert that the underlying reason for the bizarre profusion of tiny, absurd-seeming one-liner packages on NPM is paranoia, caused by a unique combination of factors. > Three factors have caused a widespread cultural paranoia among JavaScript developers. This has been inculcated over years. These factors are: JavaScript's weak dynamic type system; the diversity of runtimes JavaScript targets; and the physics of deploying software on the web. ... > Over the years there has been rapid evolution in both frontend frameworks and backend JavaScript, high turnover in bundlers and best-practises. This has metastasized into a culture of uncertainty, an air of paranoia, and an extreme profusion of small packages. Reinventing the wheel can sometimes be good - but would you really bother doing it if you had to learn all the arcane bullshit of browser evolution, IE8 compatibility, implementation bugs, etc. ad infinitum? > And it's not just that you don't understand how things work now, or how they used to work - but that they'll change in the future! [1] https://listed.to/@crabmusket/14061/javascript-s-ecosystem-is-uniquely-paranoid https://listed.to/@crabmusket/14061/javascript-s-ecosystem-i...
- nerdbert 2y ago
  > why does the JS ecosystem seem to have "more fear" than for example the Python ecosystem? Perhaps it's because so many JS developers - quite rightfully - suffer from impostor syndrome? It's the language with the largest proportion of people who didn't set out to be programmers but somehow got mission-crept into becoming one.
DonHopkins 2y ago
>Micro-libraries should never be used Passive voice. WHO should never use micro-libraries?
- bvisness 2y ago
  You, specifically.
oftenwrong 2y ago
The primary cause of the left-pad incident was that left-pad was removed from the npm registry. Many libraries depended on left-pad. The same could have occurred with any popular library, whether micro or not. To reformulate the statement made in the intro of this post: "maybe it’s not a great idea to outsource _any critical_ functionality to random people on the internet." It has long been a standard, best practice in software engineering to ensure dependencies are stored in and made available from first-party sources. For example, this could mean maintaining an internal registry mirror that permanently stores any dependencies that are fetched. It could also be done by vendoring dependencies. The main point is to take proactive steps to ensure your dependencies will always be there when you need them, and to not blindly trust a third-party to always be there to give your dependencies to you.
- Brian_K_White 2y ago
  The problem with micro is 100 micros is 100x more surface area and chances than 1.
- klabb3 2y ago
  > To reformulate the statement made in the intro of this post: "maybe it’s not a great idea to outsource _any critical_ functionality to random people on the internet." Well everything is critical in the sense that a syntax error could break many builds and CI systems. This is what lock files are for. If used properly, and the registry is available, there are no massive issues. This is how things are supposed work – all the tooling is made this way. In short, I think the lessons from the leftpad debacle are (1) people don’t use existing versioning tooling, (2) there is a surprising amount of vendors involved if you look at dep trees for completely normal functionality and (3) the JS ecosystem is particularly fragmented with poor API discipline and non-existent stdlib. EDIT: Just read up on it again and I misremembered. The author removed leftpad from NPM due to a dispute with the company regarding an unrelated package. That’s more of a mismanaged registry situation. You can’t mutate and remove published code without breaking things. Thus NPM wasn’t a good steward of their registry. If there’s a need to unpublish or mutate anything, there needs to be leeway and a path to migrate.
  - 2y ago
edwinjm 2y ago
If your only examples are leftPad and isNumber, I can’t take this article seriously. There are so many really useful micro libraries.
- hinkley 2y ago
  p-limit
  - Sammi 2y ago
    P-limit is amazing. So good it kinda is a proof that most libs aren't as good as it.
- jrpelkonen 2y ago
  There are plenty of other one-liners: is-odd, is-promise, etc.
Cheezmeister 2y ago
At no point does this article attempt to define the term "micro-library". Perhaps we should start there.
stuaxo 2y ago
The bigger principle is that atomising everything into small parts separated from their context obfuscated them and the bigger picture.
morningsam 2y ago
These arguments aren't very convincing IMHO. Using SNS as an example when it's neither micro nor a library but a service (and a huge abstraction over native push notifications, whereas most micro-libraries provide simple utilities that aren't very abstract), saying that complex libraries are harder to audit and hence a security risk (which should be a point in favor of micro-libraries that are small enough to audit in minutes), saying libraries might have large footprints (which is surely another reason to go for micro-libraries over all-you-could-possibly-need-libraries), saying transitive dependencies are bad, (yet again, this points towards an advantage of micro-libraries, which are less likely to have many dependencies), ... I don't know.
wakawaka28 2y ago
There's nothing especially wrong with small libraries if you carefully manage them and don't allow for supply chain attacks. I don't think updates are a serious concern compared to not using a library, because your own code could easily have vulnerabilities too. It is harder to update lots of small libraries versus one big library, but you pick your battle.
gerdesj 2y ago
"The library may be a bad fit for your problem." How is this the fault of the library? You chose the wrong one! "This often cancels out the primary benefit of libraries. No, you don’t have to write the code, but you do have to adapt your problem to fit the library" You evaluated the library, found is unsuitable and yet, it is somehow their fault. Why on earth would you project your own failures on to someone else's code? You do you!
- bvisness 2y ago
  If we had perfect knowledge, we could choose perfect libraries. But we don't. Maybe someday.
flysand7 2y ago
I was about to jump into the comment section and say something along the lines of "but no one really thinks they're actually good, right?", only to see the top comment arguing they're good.
- _xiaz 2y ago
  Astounding that this is as polarizing of a take as it seems to be
shiroiushi 2y ago
Until I read the comments here, I thought from the title that this was about those small neighborhood "libraries" that are basically a box the size of a very large birdhouse, mounted on a post, with a bunch of donated books inside that passersby are free to borrow. I was really wondering why someone would have a problem with these, unless they work for a book publisher.
statictype 2y ago
I have found that micro libraries are being replaced by ChatGPT generated functions.
nalgeon 2y ago
The title of the article is "Micro-libraries need to die already". Renaming the submission to "Micro-libraries should never be used" is pathetic, Daniel. I'm not surprised though.
Joker_vD 2y ago
> One breaking change simply upgraded the minimum supported Node version from 0.10.0 to 0.12.0 and changed nothing else. Well, that's a proper use of SemVer, not sure why you put it against the library's author. I've personally been burned enough times by libraries that for some reason think that literally being unable to compile them is somehow a backwards-compatible change, so it's refreshing to see that some people actually understand that.
ptman 2y ago
A little copying is better than a little dependency.
kazinator 2y ago
This seems better articulated like this: Applications should never have trivial, tiny libraries as moving-target external dependencies. If you must use a small library, bring it into the program.
LordHeini 2y ago
I think discussions like this mostly miss the point. Obviously you want basic, stable and well documented functionality in your programming language. But JavaScript does simply not have it. So how do you solve this dilemma? 1) the everything is an import way: use NPM and create a dependency hell from hell (requires Satan) made by Lucifer (same as Satan but different) using lava with fire (requires node v <= 9.42.0815) and heat (deprecated) requiring brimstone (only node v > 10.23) with a cyclic dependency on the Devil (incompatible with Satan). 2) the Golang way: copy paste ALL the things, only for your co worker to copy paste all the things again, only for your co worker to copy paste all the tings again, only for your... Way 1 wastes your time when it breaks (sooner than later) but is necessary for non trivial functionality. Way 2 works only for trivial packages so choose your poison. JavaScript (apart from not being a good programming language in general) is sorely missing a std lib. One could argue that having a bad std lib is even even worse (PHP anyone?) but it is really hard to decide. Sadly JavaScript is just unfit for the purpose it is being used for.
n0tank3sh 2y ago
It really depends on the case. Some folks use the left-pad library for aligning, which can be done in 10 minutes. In C++, we have header libraries for thread pools, etc. I don't think implementing a fully functional thread pool with waiting and other features is an easy task. In conclusion, it really depends on the situation.
- Brian_K_White 2y ago
  No it doesn't. The answer to needing a string manipulation function would be to use a string manipulation library that include that function, not one that is nothing but that finction. If you don't need anything else, and having the linker not include unused code isn't good enough, then just vendor the single function. There could still be some special case but that will need a lot of explaining to justify and will be such an exception that it is silly to talk about. There are legitimate one time freak exceptions to every principle. It means nothing.
arlojacek1 2y ago
[dead]

180 points by nalgeon 2y ago

I suspect this might be seen as trolling, but why isn't there a standard lib of stuff like this? surely it can't be beyond the wit of programming kind to have a standard lib, or even layers of standard lib for Node? What is the argument for not having a standard lib, apart from download speed?

qsort 2y ago
I'm not taking an absolute position either way -- the devil is in the details -- but here's my steelman for the opposing view: When you put something in the standard library, it's harder to take it out, meaning that you're committing development resources to support the implementation. Furthermore things change: protocols and formats rise and fall in popularity and programming style evolves as the language changes (e.g. callbacks vs. promises in JS). Therefore the stdlib becomes where libraries go to die, and you'll always have a set of third party libraries that are "pseudo-standard", like NumPy in Python. Having a minimal stdlib lets you "free-market" the decision, letting the community effects take care of what is considered standard in the ecosystem, and lets you optimize its minimal surface, like what happened with C.
- hinkley 2y ago
  But people don’t have to spend social capital to get it used at the next place they work.
kwhitefoot 2y ago
Why isn't there some kind of 'compiler/linker/stripper' that would collect the functions actually used and compile them into an application specific library? Yes I know that dynamic dispatch makes that difficult but the programmer does surely know which functions he wants to call. I sometimes hanker for a return to Fortran IV where every routine was separately compiled and the linker only put into the object code those that were referred to by something else.
- mattlondon 2y ago
  Do you mean tree-shaking? I think basically all JavaScript "compilers" do this these days so that the code you serve is only the code you actually use.
- samatman 2y ago
  Zig handles this with lazy compilation: code is parsed, but not even type checked, until the compiler reaches it. This can lead to the occasional rude surprise when finally reaching code I've been working on for awhile, but haven't yet connected to the rest of the project. But it means there's no need for tree shaking, because nothing gets in until it gets used. One of my favorite things about the language.
  - ArtixFox 2y ago
    its such a useful technique. tiny binaries ftw!!
AdrianB1 2y ago
There are many ways to look at this. Maybe the standard lib of such functions should be implemented as native functions in the language (1). Or as a standard external function library (2). Or people should copy-paste the functions they need and keep in their organization's function library (3). I am sure there are a few more options. I moved to option 3: in all my apps I include a function library that I build over the years, so I don't start from scratch every time. I deeply hate ("hate speech" example here) dependencies to libraries from all over the Internet due to security reasons, but I copy-paste code when needed to my library after I read, understand and check the code that I copy. The biggest advantage is that some of this code is better than what I could invent from scratch in a busy day and I save the time of doing it right. The disadvantage is there is no way to reward these authors that contribute to humankind. PS. My function library has functions mostly written by me, over 80%, but it includes code written by others. In my case, every time I need a function I check my existing library first, then analyze whether to write or copy.
- richardwhiuk 2y ago
  I assume you correctly obey the licensing when you copy and paste it in....
  - AdrianB1 2y ago
    Yes. It is not worth not doing it.
xmodem 2y ago
Every non-trivial Java project I've worked on ends up depending on Google's Guava and some combination of Apache Commons libraries.

mkoubaa 2y ago

Why don't people just copypasta the library into their codebase with the license header intact and keep it out of npm

tptacek 2y ago

I think the JS library ecosystem is a debacle, but there's really only one point in this post that grabbed me: "Would future updates be useful? No. The library is so simple that any change to the logic would be breaking, and it is already clear that there are no bugs." Maybe what you want is a library ecosystem where things can be marked "this will never change". Something crazy happens and you actually need to update "is-number"? Rename it. Of course, you can simulate that with a single large omnibus dependency that everyone can trust that pulls all these silly micro-libraries in verbatim.

Lws803 2y ago

I would argue that "They should either be copy-pasted into your codebase" would cause more code liabilities and maintenance required further down the line. I've personally seen codebases with a ton of custom code, copy-pasted code, inspired implementations before and it was horrible to get them up to speed with the latest functionality / best practices. I agree that having too many micro-libraries might not be beneficial though, but perhaps look for larger, more well-established libraries that encompasses those functionalities :)

quonn 2y ago
What‘s an example for „latest functionality“ or „best practices“ that should or could possibly change for a function like leftPad and that would not automatically happen by virtue of being in the code base (such as formatting)?
SOLAR_FIELDS 2y ago
You don't even have to talk about hypotheticals when it comes to this "vendor everything instead" philosophy. This is basically how the C world works, a lot of which is driven by a general allergy to dependencies by embedded developers - partially out of necessity (space and overhead are MUCH more important constraints in embedded land) - but also partially out of cargo culting. I guess the opinion I'll share here is that I don't hear too many people arguing that the way embedded developers manage C libraries is at the forefront of how we should be handling and distributing code.
- pton_xd 2y ago
  Many industries, like AAA game development, work this way. There's a good reason for it -- code quality and performance is important. Understanding what your code actually does and how it does it, is important. And as a result, AAA game software is able to push the boundaries of consumer compute performance. Micro-library-built software is generally just bloated crap that barely works. But it is fast to churn out, so there's that.
  - SOLAR_FIELDS 2y ago

gjsman-1000 2y ago

In my Laravel projects, there are a few packages of much more niche/hobbyist origin without corporate backing, some haven’t been updated for a while, and others are perfectly fine and don’t need much maintenance. Normally, packages are listed in my composer.json and stored in vendor/. For those packages, I created a separate folder called vendor_private/ which is part of my Git tree, put copies of these weird little packages in it, and set up my composer.json to consider that folder a repository. Works like a charm. My big important packages are still upstream. I can customize the little ones as needed to fit better, or have better code, and not worry about them going unmaintained. It’s also way quicker than copying the files individually out of the package and into the right places (along with updating Namespaces, configuration, etc.) Once in a while, I’ll go back and see if anything worthwhile has changed upstream - and so far, it never has.

AndyKelley 2y ago

Missing benefit: Libraries are a way to distribute and share the costs of labor, resulting in a more efficient ecosystem. This doesn't apply to micro-libraries, but it looks like that cost/benefit list is intended to cover libraries in general.

franciscop 2y ago

Seems a lot like the classic "I put only a couple of the strong advantages and enumerate everything I could think about as disadvantage". While I'm bias (I've done a bunch of these micro-libraries myself), there's more reasons I/OSS devs do them! To name other advantages (as a dev consuming them): - Documentation: they are usually well documented, at least a lot better than your average internal piece of code. - Portability: you learn it once and can use it in many projects, a lot easier than potentially copy/pasting a bunch of files from project to project (I used to do that and ugh what a nightmare it became!). - Semi-standard: everyone in the team is on the same page about how something works. This works on top of the previous two TBF, but is distinct as well e.g. if you use Axios, 50% of front-end devs will already know how to use it (edit: removed express since it's arguably not micro though). - Plugins: now with a single "source" other parties or yourself can also write plugins that will work well together. You don't need to do it all yourself. - Bugs! When there are bugs, now you have two distinct "entities" that have strong motivation to fix the bugs: you+your company, and the dev/company supporting the project. Linus's eyeballs and all (yes, this has a negative side, but those are also covered in the cons in the article already!). - Bugs 2: when you happen upon a bug, a 3rd party might've already found a bug and fixed it or offered an alternative solution! In fact I just did that today [1] That said, I do have some projects where I explicitly recommend to copy/paste the code straight into your project, e.g. https://www.npmjs.com/package/nocolor https://www.npmjs.com/package/nocolor (you can still install it though). [1] https://github.com/umami-software/node/issues/1#issuecomment-2295231274 https://github.com/umami-software/node/issues/1#issuecomment...

pton_xd 2y ago
Every team should eventually have some internal libraries of useful project-agnostic functionality. That addresses most of your points. Copy-paste the code into your internal library and maintain it yourself. Don't add a dependency on { "assert": "2.1.0" }. It probably doesn't do what you actually want, anyway. I think the more interesting point is that most projects don't know what they actually need and the code is disposable. In that scenario micro-libraries make some amount of sense. Just import random code and see how far you can get.
- 2y ago

userbinator 2y ago

and because it updates fairly frequently I fail to comprehend how a single-function-library called "isNumber" even needs updating, much less "fairly frequently". The debate around third-party code vs. self-developed is eternal. IMHO if you think you can do better than existing solutions for your use-case, then self-developed is the obvious choice. If you don't, then use third-party. This of course says a lot about those who need to rely on trivial libraries.

guestbest 2y ago
I think the updates are more for bugfixes around edge cases than feature additions.
foul 2y ago
>I fail to comprehend how a single-function-library called "isNumber" even needs updating, much less "fairly frequently". If someone uses isNumber as a fundamental building block and surrogate for Elm or Typescript (a transpiler intermediate that would treat number more soundly I hope), this poor soul whom I deeply pity will encounter a lot of strange edge-cases (like that one stated in the article: NaN is a number or not?) and if they fear the burden of forking the library they will try to inflict this burden upstream, enabling feature or conf bloat. I insinuate that installation of isNumber is, like most of these basic microlibs, a symptom of incompetence in usage of the language. A worn JS dev would try isNaN(parseInt(num+'')) and sometime succeed.
- flysand7 2y ago
  > [...] and sometime succeed Nothing is ever certain when you program in javascript.
consteval 2y ago
> I fail to comprehend how a single-function-library called "isNumber" even needs updating Never underestimate the complexity and footgunny nature of JS' type system.

ristos 2y ago

Micro-libraries are really good actually, they're highly modular, self-contained code, often making it really easy to understand what's going on. Another advantage is that because they're so minimal and self-contained, they're often "completed", because they achieved what they set out to do. So there's no need to continually patch it for security updates, or at least you need to do it less often, and it's less likely that you'll be dealing with breaking changes. The UNIX philosophy is also build on the idea of small programs, just like micro-libraries, of doing one thing and one thing well, and composing those things to make larger things. I would argue the problem is how dependencies in general are added to projects, which the blog author pointed out with left-pad. Copy-paste works, but I would argue the best way is to fork the libraries and add submodules to your project. Then if you want to pull a new version of the library, you can update the fork and review the changes. It's an explicit approach to managing it that can prevent a lot of pitfalls like malicious actors, breaking changes leading to bugs, etc.

mattlondon 2y ago
> Micro-libraries are really good actually, they're highly modular, self-contained code Well I think that is the point, they're not self-contained. You are adding mystery stuff and who knows how deep the chain of dependencies go. See the left-pad fiasco that broke so much stuff, because the chain of transitive dependencies ran deep and wide. NPM is a dumpster fire in this regard. I try to avoid it - is there a flag you can set to say "no downstream dependencies" or something when you add a dependency? At least that way you can be sure things really are self-contained.
- IgorPartola 2y ago
  I remember adding a random date picker that pulled in a copy of React with it to a non-React project. NPM is a dumpster fire at a nuclear facility.
- ristos 2y ago
  Yeah there's a way to do that, yarn and pnpm can flatten the dependency tree. You can add the fork directly too: yarn add <path/to/your/forked/micro-library.git> pnpm add <path/to/your/forked/micro-library.git>

qwerty456127 2y ago

> Micro-libraries should never be used. They should either be copy-pasted into your codebase, or not used at all. I would prefer them to be built straight in the languages.

jdminhbg 2y ago
Yes, everyone seems to take the wrong lesson from left-pad. The reason left-pad happened on NPM isn't that there's something uniquely wrong with how NPM was built, but that JS has a uniquely barren standard library. People aren't writing their own left-pad functions in Java or Go or Python, it's just in the stdlib.
- _xiaz 2y ago
  At the same time Go is quite barren when it comes to list (slice) functions, but I largely agree with Java and Python

TacticalCoder 2y ago

Thought experiment: if a LLM can correctly produce the code for a micro-library like, say, leftpad... Should you call leftpad as a dependency or should you have the LLM generate that leftpad function for you? And if the LLM ain't good enough to write leftpad, how can I trust it to write anything at all?

joshmarinacci 2y ago

TL;DR most micro libraries should be gists

mirekrusin 2y ago

Personally I prefer sharing one level up from function to conceptual module, ie. instead of "left-pad" function, "string" module, ie. a bit like this [0] (`${authorshipDisclaimer}`). I'm also an advocate, against crowd, of qualified imports as they help with refactoring (renames are propagated, especially in monorepos), readability/reviews (functions are qualified, you know where they're coming from) and overall coding experience – qualified module name followed by dot gives good autocompletion, imports look neat in larger projects etc. The codebase written like this resembles extended standard library. It also helps with solving problems by encouraging first principle thinking, bottom up coding that produces auditable codebase with shallow external dependencies etc. [0] https://github.com/preludejs https://github.com/preludejs

xg15 2y ago

Micro libraries are worse than no libraries at all - but I maintain they are still better than gargantuan "frameworks" or everything-but-the-kitching-sink "util"/"commons" packages, where you end up only using a tiny fraction of the functionality but have to deal with the maintenance cost and attack surface of the whole thing. If you're particularly unlucky, the unused functionality pulls in transitive dependencies of its own - and you end up with libraries in your dependency tree that your code is literally not using at all. If you're even more unlucky, those "dead code" libraries will install their own event handlers or timers during load or will be picked up by some framework autodiscovery mechanism - and will actually execute some code at runtime, just not any code that provides anything useful to the project. I think an apt name for this would be "undead code". (The examples I have seem were from java frameworks like Spring and from webapps with too many autowired request filters, so I do hope that is no such an issue in JS yet)

DonHopkins 2y ago
Sometimes importing zombie "undead code" libraries can be beneficial! I just refactored a bunch of python computer vision code that used detectron2 and yolo (both of which indirectly use OpenCV and PyTorch and lots of other stuff), and in the process of cleaning up unused code, I threw out the old imports of the yolo modules that we weren't using any more. The yololess refactored code, which really didn't have any changes that should measurably affect the speed, ran a mortifying 10% slower, and I could not for the life of me figure out why! Benchmarking and comparing each version showed that the yololess version was spending a huge amount of time with multiple threads fighting over locks, which the yoloful code wasn't doing. But I hadn't changed anything relating to threads or locks in the refactoring -- I had just rearranged a few of the deck chairs on the Titanic and removed the unused yolo import, which seemed like a perfectly safe innocuous thing to do. Finally after questioning all of my implicit assumptions and running some really fundamental sanity checks and reality tests, I discovered that the 10% slow-down in detectron2 was caused by NOT importing the yolo module that we were not actually using. So I went over the yolo code I was originally importing line by line, and finally ran across a helpfully commented top-level call to fix an obscure performance problem: https://github.com/ultralytics/yolov5/blob/master/utils/general.py#L67 https://github.com/ultralytics/yolov5/blob/master/utils/gene... cv2.setNumThreads(0) # prevent OpenCV from multithreading (incompatible with PyTorch DataLoader) Even though we weren't actually using yolo, just importing it, executing that one line of code fixed a terrible multithreading performance problem with OpenCV and PyTorch DataLoader fighting behind the scenes over locks, even if you never called yolo itself. So I copied that magical incantation into my own detectron2 initialization function (not as top level code that got executed on import of course), wrote some triumphantly snarky comments to explain why I was doing that, and the performance problems went away! The regression wasn't yolo's or detectron2's fault per se, just an obscure invisible interaction of other modules they were both using, but yolo shouldn't have been doing anything globally systemic like that immediately when you import it without actually initializing it. But then I would have never discovered a simple way to speed up detectron2 by 10%! So if you're using detectron2 without also importing yolo, make sure you set the number of cv2 threads to zero or you'll be wasting a lot of money.

deleted 2y ago

[deleted]

qudat 2y ago

Partially disagree, JS has unique features that require small libraries: https://bower.sh/my-love-letter-to-front-end-web-development https://bower.sh/my-love-letter-to-front-end-web-development However, if I can inline a small function, I will, so in that sense I agree.

crabmusket 2y ago
> Javascript has a very unique set of challenges that differentiates itself from the rest of the programming world. The primary driving factor for its unique position is javascript is downloaded on the client's browser. Languages that run on the server, for example, don't need to send code that it runs onto client machines, they merely respond to requests made from the browser. This is profoundly true. JavaScript written for the frontend has different "physics" to backend code. It's not only code size that is significant. It's the fact that when you ship code over the wire to a client, you don't know what browser or even JS engine version will be interpreting it. Platform incompatibility has been a huge driver of issues in the JS/NPM ecosystem and has caused JS's culture to develop the way it has. I wrote more about this, link in a top level comment.

PhilipRoman 2y ago

Micro libraries are fine (well... not really), the problem starts when each of those depends on 10 more "micro" libraries and so forth. The branching factor quickly leads to bloat. Libraries have a duty to minimize their footprint in ways that applications do not.

rc_kas 2y ago

the entire nodejs ecosystem needs to die. You all are just keeping it alive.

edwinjm 2y ago
You can start by ignoring it
crabmusket 2y ago
Have fun convincing all major browser vendors to no longer support JS scripts :) (At this point Nodejs is the defacto tooling ecosystem for even JS destined to run in a browser. You can't separate the two.)

edfletcher_t137 2y ago

Where do you draw the line: is 10 lines "micro"? 50? 100? It is never quantified within, yet the very click-bait-y title relies on the term. How many bugs can hide in 50 lines or 100? And you really want to copy-paste that code at a static point in time?!

IshKebab 2y ago
I don't think they need to draw an exact line for us to know what they're talking about.
- edfletcher_t137 2y ago
  The fact that we're even debating it shows that they do. And I didn't ask for "an exact line". I asked for anything. What is your qualification for a "micro" library? I'm betting it's different from 9 other respondents. That's a problem. Furthermore, that's not even the main contention I was highlighting. Without a proper definition, the advice of "just copy/paste these" is dangerous. Someone will draw their line at something too large, copy/paste that in and inherit bugs/vulnerabilities they never fix. That's a big problem.

edwinjm 2y ago

So, everybody can contribute an npm package. The advantage: - everybody can contribute an npm package The disadvantage: - everybody can contribute an npm package

unstable 2y ago

> You can write isNumber(foo) instead of typeof foo === "number". Indeed you can, but it depends what isNumber does. This is more like what it should do IMO: function isNumber( foo ) { return ( (typeof foo === "number") && (foo == foo)) || ((typeof foo === 'object') && (foo instanceof Number) ); } And that is I think the value of micro libs, at least in JS, you don't want to think about all the edge cases when you only want to check if something is a Number.

IshKebab 2y ago
Doesn't that exclude NaN (which you probably want despite the name)? I think this really highlights that you probably do want to think about those edge cases... In any case this is a bad example because Typescript exists.
- unstable 2y ago
  Accepting NaN as a number can potentially crash your app, that's why I reject it in isNumber. I only tried to highlight some edge cases that I personally don't like to spend energy on, trying to get it right, when writing code. Btw, isNumber is a dynamic call in the example and unrelated to TypeScript. TypeScript doesn't exist at runtime.
layer8 2y ago
This is an argument for having a library that provides that function, but it is not an argument that it should be a micro-library.
ristos 2y ago
Reminds me of the 2ality blog post on that: https://2ality.com/2017/08/type-right.html https://2ality.com/2017/08/type-right.html
crabmusket 2y ago
This library is a hilarious example of a huge problem with this kind of package. "Number" is in the eye of the beholder. A string containing numeric characters is, in my view, in no useful way "a number". A package that treats it as such just perpetuates weakly-typed nonsense.* But the broader point is, you can't outsource understanding to a package. There will be places in your code where NaN is a perfectly valid number, or Infinity. And other places where you absolutely need to be sure neither of the above make their way in. By pretending that a package can capture the universal essence of "numberless", and that this will broadly apply across the entire JS ecosystem (see reported benefits like "different libraries can all rely on is-number instead of rewriting duplicated helper functions!") is naive. I wrote more about this in a post linked in a top level comment. The is-promise library is another great example. * Personal pet theory is that the package author would have been embarrassed to publish a 1-line package, so included "numeric strings are numbers" as a fig leaf to justify the package's existence. They should have instead created two new packages, is-actual-number and is-numeric-string, so the implementation of is-number could be nice and clean: module.exports = function(n) { return require('is-actual-number')(n) || require('is-numeric-string')(n); } I can feel the power of webscale coursing through me

replete 2y ago

Not checking dependency updates need to die already. Not choosing better dependencies needs to die already.

stevebmark 2y ago

[flagged]

johnnyanmac 2y ago
>This discussion is a complete waste of oxygen and bits. There is no point discussing this. Why?

crabmusket 2y ago

While I mainly agree with the author's substantive point, though I find some of the ways it's presented in this post not entirely convincing or fair, I am interested that someone else has identified this: > I have talked a lot about the costs of libraries, and I do hope people are more cautious about them. But there’s one factor I left out from my previous discussion. I think there’s one more reason why people use libraries: fear. > Programmers are afraid of causing bugs. Afraid of making mistakes. Afraid of missing edge cases. Afraid that they won’t be able to understand how things work. In their fear they fall back on libraries. “Thank goodness someone else has solved the problem; surely I never would have been able to.” I think this is true, but why does the JS ecosystem seem to have "more fear" than for example the Python ecosystem? I wrote about this a while ago. I think that actually JS does (or did) cause more fear in its developers than other programming languages. I described it as paranoia, a more insidious uncertainty. Quoting myself[1]: > There are probably many contributing factors that have shaped NPM into what it is today. However, I assert that the underlying reason for the bizarre profusion of tiny, absurd-seeming one-liner packages on NPM is paranoia, caused by a unique combination of factors. > Three factors have caused a widespread cultural paranoia among JavaScript developers. This has been inculcated over years. These factors are: JavaScript's weak dynamic type system; the diversity of runtimes JavaScript targets; and the physics of deploying software on the web. ... > Over the years there has been rapid evolution in both frontend frameworks and backend JavaScript, high turnover in bundlers and best-practises. This has metastasized into a culture of uncertainty, an air of paranoia, and an extreme profusion of small packages. Reinventing the wheel can sometimes be good - but would you really bother doing it if you had to learn all the arcane bullshit of browser evolution, IE8 compatibility, implementation bugs, etc. ad infinitum? > And it's not just that you don't understand how things work now, or how they used to work - but that they'll change in the future! [1] https://listed.to/@crabmusket/14061/javascript-s-ecosystem-is-uniquely-paranoid https://listed.to/@crabmusket/14061/javascript-s-ecosystem-i...

nerdbert 2y ago
> why does the JS ecosystem seem to have "more fear" than for example the Python ecosystem? Perhaps it's because so many JS developers - quite rightfully - suffer from impostor syndrome? It's the language with the largest proportion of people who didn't set out to be programmers but somehow got mission-crept into becoming one.

DonHopkins 2y ago

>Micro-libraries should never be used Passive voice. WHO should never use micro-libraries?

bvisness 2y ago
You, specifically.

oftenwrong 2y ago

The primary cause of the left-pad incident was that left-pad was removed from the npm registry. Many libraries depended on left-pad. The same could have occurred with any popular library, whether micro or not. To reformulate the statement made in the intro of this post: "maybe it’s not a great idea to outsource _any critical_ functionality to random people on the internet." It has long been a standard, best practice in software engineering to ensure dependencies are stored in and made available from first-party sources. For example, this could mean maintaining an internal registry mirror that permanently stores any dependencies that are fetched. It could also be done by vendoring dependencies. The main point is to take proactive steps to ensure your dependencies will always be there when you need them, and to not blindly trust a third-party to always be there to give your dependencies to you.

Brian_K_White 2y ago
The problem with micro is 100 micros is 100x more surface area and chances than 1.
klabb3 2y ago
> To reformulate the statement made in the intro of this post: "maybe it’s not a great idea to outsource _any critical_ functionality to random people on the internet." Well everything is critical in the sense that a syntax error could break many builds and CI systems. This is what lock files are for. If used properly, and the registry is available, there are no massive issues. This is how things are supposed work – all the tooling is made this way. In short, I think the lessons from the leftpad debacle are (1) people don’t use existing versioning tooling, (2) there is a surprising amount of vendors involved if you look at dep trees for completely normal functionality and (3) the JS ecosystem is particularly fragmented with poor API discipline and non-existent stdlib. EDIT: Just read up on it again and I misremembered. The author removed leftpad from NPM due to a dispute with the company regarding an unrelated package. That’s more of a mismanaged registry situation. You can’t mutate and remove published code without breaking things. Thus NPM wasn’t a good steward of their registry. If there’s a need to unpublish or mutate anything, there needs to be leeway and a path to migrate.
- 2y ago

edwinjm 2y ago

If your only examples are leftPad and isNumber, I can’t take this article seriously. There are so many really useful micro libraries.

hinkley 2y ago
p-limit
- Sammi 2y ago
  P-limit is amazing. So good it kinda is a proof that most libs aren't as good as it.
jrpelkonen 2y ago
There are plenty of other one-liners: is-odd, is-promise, etc.

Cheezmeister 2y ago

At no point does this article attempt to define the term "micro-library". Perhaps we should start there.

stuaxo 2y ago

The bigger principle is that atomising everything into small parts separated from their context obfuscated them and the bigger picture.

morningsam 2y ago

These arguments aren't very convincing IMHO. Using SNS as an example when it's neither micro nor a library but a service (and a huge abstraction over native push notifications, whereas most micro-libraries provide simple utilities that aren't very abstract), saying that complex libraries are harder to audit and hence a security risk (which should be a point in favor of micro-libraries that are small enough to audit in minutes), saying libraries might have large footprints (which is surely another reason to go for micro-libraries over all-you-could-possibly-need-libraries), saying transitive dependencies are bad, (yet again, this points towards an advantage of micro-libraries, which are less likely to have many dependencies), ... I don't know.

wakawaka28 2y ago

There's nothing especially wrong with small libraries if you carefully manage them and don't allow for supply chain attacks. I don't think updates are a serious concern compared to not using a library, because your own code could easily have vulnerabilities too. It is harder to update lots of small libraries versus one big library, but you pick your battle.

gerdesj 2y ago

"The library may be a bad fit for your problem." How is this the fault of the library? You chose the wrong one! "This often cancels out the primary benefit of libraries. No, you don’t have to write the code, but you do have to adapt your problem to fit the library" You evaluated the library, found is unsuitable and yet, it is somehow their fault. Why on earth would you project your own failures on to someone else's code? You do you!

bvisness 2y ago
If we had perfect knowledge, we could choose perfect libraries. But we don't. Maybe someday.

flysand7 2y ago

I was about to jump into the comment section and say something along the lines of "but no one really thinks they're actually good, right?", only to see the top comment arguing they're good.

_xiaz 2y ago
Astounding that this is as polarizing of a take as it seems to be

shiroiushi 2y ago

Until I read the comments here, I thought from the title that this was about those small neighborhood "libraries" that are basically a box the size of a very large birdhouse, mounted on a post, with a bunch of donated books inside that passersby are free to borrow. I was really wondering why someone would have a problem with these, unless they work for a book publisher.

statictype 2y ago

I have found that micro libraries are being replaced by ChatGPT generated functions.

nalgeon 2y ago

The title of the article is "Micro-libraries need to die already". Renaming the submission to "Micro-libraries should never be used" is pathetic, Daniel. I'm not surprised though.

Joker_vD 2y ago

> One breaking change simply upgraded the minimum supported Node version from 0.10.0 to 0.12.0 and changed nothing else. Well, that's a proper use of SemVer, not sure why you put it against the library's author. I've personally been burned enough times by libraries that for some reason think that literally being unable to compile them is somehow a backwards-compatible change, so it's refreshing to see that some people actually understand that.

ptman 2y ago

A little copying is better than a little dependency.

kazinator 2y ago

This seems better articulated like this: Applications should never have trivial, tiny libraries as moving-target external dependencies. If you must use a small library, bring it into the program.

LordHeini 2y ago

I think discussions like this mostly miss the point. Obviously you want basic, stable and well documented functionality in your programming language. But JavaScript does simply not have it. So how do you solve this dilemma? 1) the everything is an import way: use NPM and create a dependency hell from hell (requires Satan) made by Lucifer (same as Satan but different) using lava with fire (requires node v <= 9.42.0815) and heat (deprecated) requiring brimstone (only node v > 10.23) with a cyclic dependency on the Devil (incompatible with Satan). 2) the Golang way: copy paste ALL the things, only for your co worker to copy paste all the things again, only for your co worker to copy paste all the tings again, only for your... Way 1 wastes your time when it breaks (sooner than later) but is necessary for non trivial functionality. Way 2 works only for trivial packages so choose your poison. JavaScript (apart from not being a good programming language in general) is sorely missing a std lib. One could argue that having a bad std lib is even even worse (PHP anyone?) but it is really hard to decide. Sadly JavaScript is just unfit for the purpose it is being used for.

n0tank3sh 2y ago

It really depends on the case. Some folks use the left-pad library for aligning, which can be done in 10 minutes. In C++, we have header libraries for thread pools, etc. I don't think implementing a fully functional thread pool with waiting and other features is an easy task. In conclusion, it really depends on the situation.

Brian_K_White 2y ago
No it doesn't. The answer to needing a string manipulation function would be to use a string manipulation library that include that function, not one that is nothing but that finction. If you don't need anything else, and having the linker not include unused code isn't good enough, then just vendor the single function. There could still be some special case but that will need a lot of explaining to justify and will be such an exception that it is silly to talk about. There are legitimate one time freak exceptions to every principle. It means nothing.

arlojacek1 2y ago

[dead]