8 ms·

We Are Forking dotenvy into dotenv-ng

46 points by linggen 1mo ago

hinkley 1mo ago
> But migrating away from .env starts with reading it correctly. We had a system that mediated all access to the static config through an in-house library. Because nobody touched the environment-specific data directly, it was a convenient spot to layer on reloadable configuration from a Raft source, and fold in environment-specific secrets coming through an old 12 Factor solution, although we could have also integrated some other secrets management as well. Or, left it as a separate API. That tends to work for tired and distracted people well. Having security related stuff use a unique code pattern to access it does, I find, help keep people from treating it all the same way they treat other data, which is to throw it around profligately and occasionally write it to the logs. The moral equivalent to the guard over the missile launch switch versus the machine gun trigger. One of those has much more forgiving failure modes than the other. It's the same problem 'capability' based systems frequently run into - capabilities end up meaning that if you can see something you're entitled to use it. And when your coworkers add new features to the system, you end up accidentally being able to see things you shouldn't be able to see. Java for instance ran into this rather frequently with theirs. Someone would add an object to the global state and not notice that some peer added a reference to a reference to that object with proprietary data the client code wasn't meant to have at all. Information wants to be 'free'.
- SOLAR_FIELDS 1mo ago
  I think there are two separate concerns here. The idea that long lived secrets survive in plaintext on a disk somewhere is a bad idea that should go away, but also tangential to that people have to have good hygiene around not printing evil things to logs. Latter is a lot harder to catch imo, and is a separate problem
  - hinkley 1mo ago
    Yeah I ended up finishing a Lava Flow Antipattern, where someone started wrapping of stdout and never finished it. We replaced every value for a key of 'token', 'password', 'secret' with 8 stars in the logs. We were using Splunk and if you're handing third parties your secrets, you're gonna have a bad time. Sooner or later they'll have a disgruntled employee or a breach.
- fragmede 1mo ago
  > the guard over the missile launch switch That's called a molly guard, but also a Debian package that asks you before you run shutdown if you're on the server you intended to shut down.
  - hinkley 1mo ago
    I have on a number of occasions wondered if there's some way to make something like the 2 key system for nuclear launch protocols work for making destructive changes in production. I request an action, and it only takes effect if someone else repeats it. The problem though is priming, and a two key system only fixes that the first couple times. In a year you're right back where you started. Because once I've suggested a course of action to you, the Primacy Effect makes you key your counterproposals to my initial bid, and if I'm enough off the mark we end up off the mark together. Add Recency Effect on top of it - if I haven't been wrong the last ten times I asked you to spot me, you slowly stop assuming I'm wrong this time. When the production outage happens, it comes out that your thought process was that I know what I'm doing and you didn't 'need' to be there for the last five times so you've zoned out and didn't catch my mistake. And if I start my ask by sending you a URL, if I cut and pasted the wrong URL you're stuck looking at the wrong thing with me and we both sail off into the canyon together. Still, if I'm careful to give you only the high-level request, and we derive the same solution from that request, then maybe a tool like that would still be useful. But the most important part of that interaction is still outside of the scope of any tool that could sanity check us. It's just process.
    - fragmede 29d ago
      But the process is important. I've used Shamir secret sharing, ssss, to have it so that the root account on an important server somewhere is only accessible when two of seven people come together and break glass to get access.
eterm 1mo ago
I love AI development, but I wish product announcements were self-authored.
- paxys 1mo ago
  It’s an open source project, so you don’t have to wish it. “Hey dotenvy team, I’m a fan of your project and wanted to help out. I noticed that some of your announcements are AI generated, and would like to help bring back the human voice. I’ll be happy to proof read and/or write your blog posts and release announcements in the future. Let me know if this is something you could use.”
yieldcrv 1mo ago
doppler all the way
drdexebtjl 1mo ago
Can’t you just escape the dollar sign? If you really believe .env was a mistake (it was), just let it die. You are the ones keeping it alive now.
- haswell 1mo ago
  .env is self sustaining at this point. The functionality described in the post exists to help move people off of it. It’ll have a better chance of dying when fewer people use it and that starts with making it easy to move away from it.
  - drdexebtjl 1mo ago
    How exactly changing a default that users rely on is going to make it easy to move away from it? Instead of migrating to dotenv-ng, these users could be putting effort into migrating away from .env files entirely. edit: I understand SecretSpec needs to parse .env files exactly like Node.js’s dotenv library, which doesn’t support interpolation, to make it easier for these projects to use SecretSpec. I’m saying making this a standalone library is bad. Rust projects will pick this up and use it instead of SecretSpec, and users of the unmaintained Rust crate will migrate to it.
- Martinussen 29d ago
  Or even simpler: why on earth are you using double quotes for a string if you don't want double-quote behaviour? Anything like this should just be in 'single quotes'... Any developer getting to "its parser changed a secret while reading it" and "By the time the bcrypt bug blocked SecretSpec," from basic shell semantics is frankly astonishing. I mean this on an organisational/team level, not as a personal attack on anyone, but the hubris is incredible to me. This feels like the exact type of solution a badly prompted and poorly piloted AI tool manically follows to an absurd conclusion if you don't step in and tell it to do it the obvious and correct way instead of re-inventing the wheel. (I would also argue most of the point of a .env file being what it is that you can source it in a shell? Does the author here have any sane argument for not using a yaml/json/whatever file and XKCD-927ing it instead? Is this not just introducing a flaw when `source .env` stops behaving like expected?)
a1o 1mo ago
What does the ng at the end means? I have seen a few products/projects that added it and no explanation of the meaning.
- egil 1mo ago
  Next generation I guess.
- sphars 1mo ago
  I came in and thought it had something to do with Angular since `ng` is a popular pre/post-fix for Angular libraries.
- zufallsheld 1mo ago
  Probably next generation.
- jdhdbdbd 1mo ago
  It means new generation. Using ng suffix was the thing to do in the 2000s just like calling something beta or 2.0 in the years following
threecheese 1mo ago
Am I weird or old-fashioned for relying on direnv over .env? These days my projects use a lot of different tech, and it just feels weird to me to have them all parsing .env files and hoping everything works. Especially with inference provider tokens basically connecting my checking account to the internet.
- afiori 1mo ago
  Most uses of .env files are for apps or containers imho, so you could just have a .env file in each folder and configure docker compose /node / whatever to pick it up
- wrenky 1mo ago
  direnv + nix flakes solved so many problems for me, I have no idea how other people work on multiple projects at the same time without getting mixed up.
- stock_toaster 1mo ago
  I also used to use direnv (and likewise eschewed .env), but more recently I moved env definitions to mise. Been working out nicely.
- paulddraper 1mo ago
  Direnv's first stable release was in 2013, .env was 2012 (adopted by Heroku). So no, if anything direnv makes you hip and edgy.
- LawnGnome 1mo ago
  No, I also use direnv. It does everything I want, nothing I don't, and has been stable for me for years.
- PufPufPuf 1mo ago
  I use Mise for env. I primarily use it for installing tools and running tasks, having env support is just a cherry on top.
- dolni 1mo ago
  Dotenv is definitely something that should not exist. Direnv is the right way to go if you want a tool for this. Otherwise dotenv's implementation in bash is essentially `set -a; source .env`. Look ma, no custom parser! Seems really silly to have a library do what two lines of shell can reasonably accomplish.
Garlef 1mo ago
I think mise offers a better solution for the problem.
- jdxcode 1mo ago
  well mise uses dotenvy right now (for env._file anyhow), it may be worth switching to dotenv-ng but I haven't looked into it yet https://github.com/jdx/mise/blob/fc441787fe52b46be1245046758fe3d4bd36c9ed/Cargo.toml#L110 https://github.com/jdx/mise/blob/fc441787fe52b46be1245046758... I assume you just mean normal [env] in mise.toml which is different, but this still is relevant to mise
  - Garlef 1mo ago
    Yes - I think .env files are not a good idea (works for one person but even then it's not great)
asveikau 1mo ago
I am an old man yelling at cloud, but why is a library that does something this simple something that needs maintenance? Like if your jaw is dropping because an environment variable parser hasn't seen commits since 2023 ... ... How old is /usr/bin/env ? Lots of code churn on that one?
whalesalad 1mo ago
The beauty of the environment is that it can be manipulated prior to running the program. I don't understand the desire to have runtime .env loaders when that env can just be established before the program is run. Every language can read env without a library. The churn in this space is too high... it's a smell. I have a shell function for this in my zsh config: function lenv() { export $(cat .env | xargs) } It doesn't get any more unix than this. I do not allow my team to build software that autoloads environment files using libraries. It's such a mess. That is a concern of the shell.
- gwynforthewyn 1mo ago
  You're not wrong, but there's a small detail that your shell function doesn't solve for: unloading the environment variable once it's no longer needed. If you maintain one shell per process-that-needs-a-secret-in-the-env then it doesn't matter, but if you're using long-running shells then direnv can load and unload env vars. It also gives you a nice little heads-up that it loaded/unloaded variables.
  - drdexebtjl 1mo ago
    Use this wrapper instead: #!/bin/sh set -a . ./.env exec "$@"
    - jolmg 1mo ago
      or just `exec $SHELL` to unload.
      - gwynforthewyn 29d ago
        Sort of? It definitely does what's needed, but it also removes any additional environment variables that were set, unless they're in your shell initialisation files. I'm not trying to persuade that these tools are essential or anything, just highlighting that there's a few small details some take care of that're fairly nice.
- 1mo ago
yoyohello13 1mo ago
Maybe this is a dumb question but what does -ng mean? I see a lot of forks seem to follow the <project>-ng naming convention.
- deleted 1mo ago
  [deleted]
- deleted 1mo ago
  [deleted]
- drdexebtjl 1mo ago
  Next generation. It’s not a great convention.
  - yoyohello13 1mo ago
    Ah, ok. That makes sense.
panzi 1mo ago
.env is another "simple" format that turns out to be actually a whole family of languages where each handles corner cases differently. And usually implementations quietly do nonsense on malformed input. Some even have command injection vulnerabilities. Or at least this was the case back when I looked at a few such implementations and wrote up this: https://github.com/panzi/punktum https://github.com/panzi/punktum

46 points by linggen 1mo ago

> But migrating away from .env starts with reading it correctly. We had a system that mediated all access to the static config through an in-house library. Because nobody touched the environment-specific data directly, it was a convenient spot to layer on reloadable configuration from a Raft source, and fold in environment-specific secrets coming through an old 12 Factor solution, although we could have also integrated some other secrets management as well. Or, left it as a separate API. That tends to work for tired and distracted people well. Having security related stuff use a unique code pattern to access it does, I find, help keep people from treating it all the same way they treat other data, which is to throw it around profligately and occasionally write it to the logs. The moral equivalent to the guard over the missile launch switch versus the machine gun trigger. One of those has much more forgiving failure modes than the other. It's the same problem 'capability' based systems frequently run into - capabilities end up meaning that if you can see something you're entitled to use it. And when your coworkers add new features to the system, you end up accidentally being able to see things you shouldn't be able to see. Java for instance ran into this rather frequently with theirs. Someone would add an object to the global state and not notice that some peer added a reference to a reference to that object with proprietary data the client code wasn't meant to have at all. Information wants to be 'free'.

SOLAR_FIELDS 1mo ago
I think there are two separate concerns here. The idea that long lived secrets survive in plaintext on a disk somewhere is a bad idea that should go away, but also tangential to that people have to have good hygiene around not printing evil things to logs. Latter is a lot harder to catch imo, and is a separate problem
- hinkley 1mo ago
  Yeah I ended up finishing a Lava Flow Antipattern, where someone started wrapping of stdout and never finished it. We replaced every value for a key of 'token', 'password', 'secret' with 8 stars in the logs. We were using Splunk and if you're handing third parties your secrets, you're gonna have a bad time. Sooner or later they'll have a disgruntled employee or a breach.
fragmede 1mo ago
> the guard over the missile launch switch That's called a molly guard, but also a Debian package that asks you before you run shutdown if you're on the server you intended to shut down.
- hinkley 1mo ago
  I have on a number of occasions wondered if there's some way to make something like the 2 key system for nuclear launch protocols work for making destructive changes in production. I request an action, and it only takes effect if someone else repeats it. The problem though is priming, and a two key system only fixes that the first couple times. In a year you're right back where you started. Because once I've suggested a course of action to you, the Primacy Effect makes you key your counterproposals to my initial bid, and if I'm enough off the mark we end up off the mark together. Add Recency Effect on top of it - if I haven't been wrong the last ten times I asked you to spot me, you slowly stop assuming I'm wrong this time. When the production outage happens, it comes out that your thought process was that I know what I'm doing and you didn't 'need' to be there for the last five times so you've zoned out and didn't catch my mistake. And if I start my ask by sending you a URL, if I cut and pasted the wrong URL you're stuck looking at the wrong thing with me and we both sail off into the canyon together. Still, if I'm careful to give you only the high-level request, and we derive the same solution from that request, then maybe a tool like that would still be useful. But the most important part of that interaction is still outside of the scope of any tool that could sanity check us. It's just process.
  - fragmede 29d ago
    But the process is important. I've used Shamir secret sharing, ssss, to have it so that the root account on an important server somewhere is only accessible when two of seven people come together and break glass to get access.

eterm 1mo ago

I love AI development, but I wish product announcements were self-authored.

paxys 1mo ago
It’s an open source project, so you don’t have to wish it. “Hey dotenvy team, I’m a fan of your project and wanted to help out. I noticed that some of your announcements are AI generated, and would like to help bring back the human voice. I’ll be happy to proof read and/or write your blog posts and release announcements in the future. Let me know if this is something you could use.”

yieldcrv 1mo ago

doppler all the way

drdexebtjl 1mo ago

Can’t you just escape the dollar sign? If you really believe .env was a mistake (it was), just let it die. You are the ones keeping it alive now.

haswell 1mo ago
.env is self sustaining at this point. The functionality described in the post exists to help move people off of it. It’ll have a better chance of dying when fewer people use it and that starts with making it easy to move away from it.
- drdexebtjl 1mo ago
  How exactly changing a default that users rely on is going to make it easy to move away from it? Instead of migrating to dotenv-ng, these users could be putting effort into migrating away from .env files entirely. edit: I understand SecretSpec needs to parse .env files exactly like Node.js’s dotenv library, which doesn’t support interpolation, to make it easier for these projects to use SecretSpec. I’m saying making this a standalone library is bad. Rust projects will pick this up and use it instead of SecretSpec, and users of the unmaintained Rust crate will migrate to it.
Martinussen 29d ago
Or even simpler: why on earth are you using double quotes for a string if you don't want double-quote behaviour? Anything like this should just be in 'single quotes'... Any developer getting to "its parser changed a secret while reading it" and "By the time the bcrypt bug blocked SecretSpec," from basic shell semantics is frankly astonishing. I mean this on an organisational/team level, not as a personal attack on anyone, but the hubris is incredible to me. This feels like the exact type of solution a badly prompted and poorly piloted AI tool manically follows to an absurd conclusion if you don't step in and tell it to do it the obvious and correct way instead of re-inventing the wheel. (I would also argue most of the point of a .env file being what it is that you can source it in a shell? Does the author here have any sane argument for not using a yaml/json/whatever file and XKCD-927ing it instead? Is this not just introducing a flaw when `source .env` stops behaving like expected?)

a1o 1mo ago

What does the ng at the end means? I have seen a few products/projects that added it and no explanation of the meaning.

egil 1mo ago
Next generation I guess.
sphars 1mo ago
I came in and thought it had something to do with Angular since `ng` is a popular pre/post-fix for Angular libraries.
zufallsheld 1mo ago
Probably next generation.
jdhdbdbd 1mo ago
It means new generation. Using ng suffix was the thing to do in the 2000s just like calling something beta or 2.0 in the years following

threecheese 1mo ago

Am I weird or old-fashioned for relying on direnv over .env? These days my projects use a lot of different tech, and it just feels weird to me to have them all parsing .env files and hoping everything works. Especially with inference provider tokens basically connecting my checking account to the internet.

afiori 1mo ago
Most uses of .env files are for apps or containers imho, so you could just have a .env file in each folder and configure docker compose /node / whatever to pick it up
wrenky 1mo ago
direnv + nix flakes solved so many problems for me, I have no idea how other people work on multiple projects at the same time without getting mixed up.
stock_toaster 1mo ago
I also used to use direnv (and likewise eschewed .env), but more recently I moved env definitions to mise. Been working out nicely.
paulddraper 1mo ago
Direnv's first stable release was in 2013, .env was 2012 (adopted by Heroku). So no, if anything direnv makes you hip and edgy.
LawnGnome 1mo ago
No, I also use direnv. It does everything I want, nothing I don't, and has been stable for me for years.
PufPufPuf 1mo ago
I use Mise for env. I primarily use it for installing tools and running tasks, having env support is just a cherry on top.
dolni 1mo ago
Dotenv is definitely something that should not exist. Direnv is the right way to go if you want a tool for this. Otherwise dotenv's implementation in bash is essentially `set -a; source .env`. Look ma, no custom parser! Seems really silly to have a library do what two lines of shell can reasonably accomplish.

Garlef 1mo ago

I think mise offers a better solution for the problem.

jdxcode 1mo ago
well mise uses dotenvy right now (for env._file anyhow), it may be worth switching to dotenv-ng but I haven't looked into it yet https://github.com/jdx/mise/blob/fc441787fe52b46be1245046758fe3d4bd36c9ed/Cargo.toml#L110 https://github.com/jdx/mise/blob/fc441787fe52b46be1245046758... I assume you just mean normal [env] in mise.toml which is different, but this still is relevant to mise
- Garlef 1mo ago
  Yes - I think .env files are not a good idea (works for one person but even then it's not great)

asveikau 1mo ago

I am an old man yelling at cloud, but why is a library that does something this simple something that needs maintenance? Like if your jaw is dropping because an environment variable parser hasn't seen commits since 2023 ... ... How old is /usr/bin/env ? Lots of code churn on that one?

whalesalad 1mo ago

The beauty of the environment is that it can be manipulated prior to running the program. I don't understand the desire to have runtime .env loaders when that env can just be established before the program is run. Every language can read env without a library. The churn in this space is too high... it's a smell. I have a shell function for this in my zsh config: function lenv() { export $(cat .env | xargs) } It doesn't get any more unix than this. I do not allow my team to build software that autoloads environment files using libraries. It's such a mess. That is a concern of the shell.

gwynforthewyn 1mo ago
You're not wrong, but there's a small detail that your shell function doesn't solve for: unloading the environment variable once it's no longer needed. If you maintain one shell per process-that-needs-a-secret-in-the-env then it doesn't matter, but if you're using long-running shells then direnv can load and unload env vars. It also gives you a nice little heads-up that it loaded/unloaded variables.
- drdexebtjl 1mo ago
  Use this wrapper instead: #!/bin/sh set -a . ./.env exec "$@"
  - jolmg 1mo ago
    or just `exec $SHELL` to unload.
    - gwynforthewyn 29d ago
      Sort of? It definitely does what's needed, but it also removes any additional environment variables that were set, unless they're in your shell initialisation files. I'm not trying to persuade that these tools are essential or anything, just highlighting that there's a few small details some take care of that're fairly nice.
1mo ago

yoyohello13 1mo ago

Maybe this is a dumb question but what does -ng mean? I see a lot of forks seem to follow the <project>-ng naming convention.

deleted 1mo ago
[deleted]
deleted 1mo ago
[deleted]
drdexebtjl 1mo ago
Next generation. It’s not a great convention.
- yoyohello13 1mo ago
  Ah, ok. That makes sense.

panzi 1mo ago

.env is another "simple" format that turns out to be actually a whole family of languages where each handles corner cases differently. And usually implementations quietly do nonsense on malformed input. Some even have command injection vulnerabilities. Or at least this was the case back when I looked at a few such implementations and wrote up this: https://github.com/panzi/punktum https://github.com/panzi/punktum