12 ms·

Avoiding Complexity with Systemd

291 points by irl_ 5y ago

whateveracct 5y ago
Use NixOS and you'll love systemd. You'll be defining your own systemd units with ease. systemd to you will be journalctl and systemctl. So pretty good.
- atoav 5y ago
  I use mostly Ubuntu and Debian and defining systemd units just means you have to spit the right text into a .service file placed at the right spot. How does NixOS make that easier?
  - nix23 5y ago
    https://nixos.wiki/wiki/NixOS:extend_NixOS https://nixos.wiki/wiki/NixOS:extend_NixOS
  - siraben 5y ago
    In NixOS it's trivial to use and define NixOS modules, which handle restarting/starting/stopping systemd units. See the wiki example[0] on how to define and use a service that greets the user with GNU Hello. Also, since you have access to Nixpkgs, you can make the ExecStart as complicated as you want with whatever dependencies you desire, and trivially share it with others. [0] https://nixos.wiki/wiki/Module#Example https://nixos.wiki/wiki/Module#Example
    - drran 5y ago
      It looks like NixWay of doing NixThings to solve NixProblems. Why I need NixModules in the first place? ELI5, please.
      - leotaku 5y ago
        It's basically just a nice way of composing the different parts of your system. E.g. you could write a NixOS module to manage your web app. You declaratively configure that you want to run Nginx, Postgres, etc. open some ports, connect to your VPN and basically everything else you might want. There are a lot of existing modules, so in most cases you have to write very little code yourself. If you want to scale your system to run on multiple systems or containers, you can do that with relative ease. You also have a singular source of truth for all of your information, down to your application binaries. Your configuration says you are using Postgres version X.Y.Z with compilation options A, B and C, so that's exactly what is running on your systems.
      - diragon 5y ago
        It's an academically more impressive version of Ansible that's deeply intertwined with the distro's package manager.
  - goodpoint 5y ago
    It does not.
  - nh2 5y ago
    You can override NixOS's predefined systemd settings from _outside_ using NixOS module options. This allows you to change default settings that are not optimal for your use case, without having to patch NixOS itself, or write your own unit config. For example, systemd by default permanently gives up restarting services after a few number of tries (e.g. 5), even if you have set `Restart=always`. This is suboptimal for web servers that should recover by themselves after arbitrarily long failures (e.g. network downtimes outside of your control). On NixOS, you can, from your machine config, set: systemd.services.nginx.unitConfig.StartLimitIntervalSec = 0; This sets/overrides just that specific systemd option for the existing nginx module. On other distros, you often have to resort to global mutation in `/etc` that does not compose well. We use NixOS for our infra (having used Ansible before), and this ability to override anything cleanly and keeping defaults otherwise made for much easier to maintain infra code and less ugly/surprising compromises.
    - Arnavion 5y ago
      >On other distros, you often have to resort to global mutation in `/etc` that does not compose well. Why does this "not compose well" ? You don't have to override the whole unit as /etc/systemd/system/nginx.service , which would have problems if two things wanted to override different parts of the original unit. Just drop an override file in /etc/systemd/system/nginx.service.d/90-restart-always.conf with that one specific config you want to override.
      - nh2 5y ago
        > Why does this "not compose well" ? Because you cannot easily write libraries/components that do this. In NixOS, other modules can override the options of other modules. For example, a a web app can set the nginx options that it needs, instead of requiring you (the admin) to "drop a file" in /etc. This is one of the reasons why on Ansible Galaxy (community repository of Ansible roles) there are 527 nginx roles [1], and in NixOS there is 1 nginx module that everybody code-reuses. [1]: https://galaxy.ansible.com/search?deprecated=false&tags=web&keywords=nginx&order_by=-relevance&page=1 https://galaxy.ansible.com/search?deprecated=false&tags=web&...
- iso1210 5y ago
  If systemd was just a replacement for starting scripts that would be one thing. when you have to run "systemctl disable systemd-timesyncd systemd-resolved systemd-networkd" to start to get back to sanity, that's not init
- GuidoW 5y ago
  > Use NixOS and you'll love systemd. I use NixOS but certainly not love systemd. Instead, I've created a way to replace it with s6.[1] 1: https://sr.ht/~guido/nixos-init-freedom/ https://sr.ht/~guido/nixos-init-freedom/ Cheers, Guido.
  - forgotpwd16 5y ago
    That's nice and it showcases how Nix can create a declarative process management atop a script-based imperative manager. How is your experience with it? Also note that there's https://github.com/svanderburg/nix-processmgmt https://github.com/svanderburg/nix-processmgmt, a manager agnostic processes management framework supporting s6 among others, but your way seems a bit more straightforward.
    - GuidoW 5y ago
      Thanks, My use case is to get my VPS run a web and mail service on Nixos but without the bloated binary logging of journalcontrol. The indexes in these log files change heavily between snapshots so they take up way more disk space than append-only text logs, that snapshots very well on ZFS. My experiences while building: - It's easy to use the config.system.services tree of the user-services (ssh, bind, caddy, etc) to create s6-services; - Sometimes it needs a change to make it work [1] on s6; - That same change seems oblivious to systemd. I guess it just starts the process and never bothers to monitor liveness. So much for a process management system ;-) - Nixos packagers seems overworked, as my pull requests seem to get stuck ;-( - Nixos use of systemd leaves a lot of decisions to resolve at boot time, decisions that I want to make at build time with s6; - However, I cannot create a s6 dependency tree specification at build time, that's still at run time; - Because S6 uses the service-directory to store state-files in the same directory (no /etc-/var split). 1: https://github.com/NixOS/nixpkgs/pull/122844 https://github.com/NixOS/nixpkgs/pull/122844
m4r35n357 5y ago
Sick of trying to work out what language, why not mention it?
folex 5y ago
I just learned about S6 recently. It simplified daemon supervision for me a lot.
- cpach 5y ago
  s6 is very cool. Just out of curiosity, have you found any specific use-cases for it yet?
djhworld 5y ago
Really enjoyed reading this article. The LoadCredentials thing reminds me of configmaps in K8S, is there a more general thing in systemd e.g LoadConfig
- mhitza 5y ago
  Disclaimer, no idea what LoadConfig does. A more generic approach than LoadCredentials I think is the EnvironmentFile= directive if you want to pass along multiple env variables to your process without individual Environment= directives
Dah00n 5y ago
I read that as meaning "avoid systemd's complexity" as opposed to "avoid complexity by doing so and so using systemd". Oh well.
Dinux 5y ago
I'm glad to see more people sway to systemd. Systemd is 10 years in the making and it was met with skepticism right from the first day. Some of that is now slowly changing with systemd being accepted in more and more distributions. Service and runlevel management wasn't any better in the sysv era, nor were any of the multitude of custom start and boot scripts. I remember when it took multiple days testing the configuration on different distributions, editions and versions to just get a single daemon process to start without failure. Then do the whole thing over again because the Debian based distros did not use the RedHat startup tools, different (or no) runlevel configurations, different service binders, different NTP daemon, different terminal deamon, etc.. And of course the French customers wanted the service to work with Mandriva, the German customers want SUSE support with extended security options like dropping permissions after boot. Just like the article mentions you can define a portable service model with security and failure handling built in. There wasn't even anything that came close back in the day. Systemd may not have been built with the Unix philosophy in mind, but at some point that becomes a secondary concern. Systemd unifies all systemd resources in units which work anywhere, its expandable and extendable, user friendly, allows for remote monitoring etc.
- zozbot234 5y ago
  There's a lot of poorly-understood incidental complexity in the systemd codebase, and this can bite users even when doing basic service and runlevel management. The systemd approach is to try and make it 100% declarative based on simple .ini files, but the semantics of this seemingly "declarative" configuration was never properly specified. Even many systemd fans seem to be quite aware of this, and there seems to be a common understanding that some ground-up reimplementation of these ideas based on a clearer underlying "philosophy" will be needed at some point. Systemd has been a successful experiment in many ways, but relying on throwaway experimental code for one's basic production needs is not a good idea.
  - southerntofu 5y ago
    I hope you're right and we get a better, simpler declarative format in the future. systemd isn't bad but i've found when writing service files that some stuff is not as obvious as it should be.
turminal 5y ago
The title is an oxymoron. Putting a huge complex piece of software between yourself and "complexity" doesn't make the system less complex.
- smitty1e 5y ago
  Systemd seems to do a good job of moving the complexity of managing the privileged/unprivileged divide into a standardized service. I sympathize with the "transition sucks" sentiments elsewhere on this post. Having a bunch of working scripts turned into instant technical debt cannot be pleasant. But, as with python3, systemd seems to be the way things are headed.
  - deleted 5y ago
    [deleted]
  - birdyrooster 5y ago
    Why are we talking like this? We’ve been using systemd for over 5 years. It’s weird. Time loops.
    - boublepop 5y ago
      It took 10 years before the “python2 will never die”-crowd finally accepted they where not going to win at that python3 was here to stay. People spend a couple of years getting used to a stack in their early carrier and then spend decades arguing that it should never change.
      - kaba0 5y ago
        But python programs are much more complex than init scripts systemd replace. It is more like people hate change and are lazy to learn a new thing.
        smitty1e 5y ago
        Mamagement has 99 problems. Holding a status quo is about minimizing the backlog more than hating change as such. Even when we can show am improvement in security and usability, and lower training cost because of consistency, it's still another mouth to feed.
bitwize 5y ago
You incur complexity with systemd. You avoid complexity with runit.
- cpach 5y ago
  IMO runit is abandonware at this point. No release since 2014. Have you looked at s6? It’s a compelling alternative.
  - ori_b 5y ago
    Runit is finished at this point. It does what it says, and I haven't run into bugs. Churn isn't a virtue.
    - h1x 5y ago
      Exactly this. Runit was designed to be simple and in simple software, at some point, there is just not much to improve.
  - aerique 5y ago
    Void Linux, "the BSD of Linux", uses runit and it has been fine since forever. If there were any issues they would be worked on.
    - cpach 5y ago
      I’m glad to hear runit works fine for them. I’m not so sure there isn’t any room for improvement though. In the related s6 project there is lot of discussions about adding new features that are beneficial for supporting a modern Linux distro.
      - imiric 5y ago
        As a datapoint of one, I've been using Void Linux for near a decade. I haven't once been in a situation where I thought systemd would help. Granted, I don't run many custom services, but for what it does, runit does a great job. Something a bit more user friendly like s6 would be nice, but otherwise it stays out of my way and I don't think about it. NixOS/Guix would be worthwhile switching to, but Void is comfortably simple.
imiric 5y ago
Apologies, I was also triggered by the title. :) > systemd provides ways to restrict the parts of the filesystem the service can see. So like chroot and namespaces? Why do I have to depend on systemd when these are native features provided by Linux? So systemd provides a friendlier abstraction of these concepts. Great, but so do Docker and Podman and many other tools that can actually be installed without taking over the rest of the system. Having your application actually use systemd libraries further increases this dependency and makes it no longer usable but on a subset of Linux machines. This would be fine for some controlled production deployment, but is awful for usability and adoption.
- viraptor 5y ago
  > So like chroot and namespaces? Why do I have to depend on systemd when these are native features provided by Linux? Not like namespaces - using namespaces. And for the same reason we use other high level abstractions and high level languages rather than handcrafted assembly. You don't have to depend on it either - you can still use chroot instead of you want, but it's more work that way. > Great, but so do Docker and Podman and many other tools that can actually be installed without taking over the rest of the system. Docker installs a service which takes over lifecycle management, restarts, and traffic proxying for apps. It injects and managed multiple firewall chains. It pretty much takes over network management. And it's still stuck on the old cgroups format so it forces that on your system. It really doesn't win this comparison. > Having your application actually use systemd libraries You don't need them. Everything from the post is defined in simple environment variables. For example socket activation is maybe 3 extra lines when done from scratch.
  - imiric 5y ago
    > You don't have to depend on it either - you can still use chroot instead of you want, but it's more work that way. How so? If I need filesystem isolation, I'll use the simplest tool that provides it. In this case probably a container runtime. Note that none of your criticism about Docker applies to Podman. Why would I ever want to use containers with a tool that forces (OK, strongly suggests...) me to use it as an init system, logging system, network manager, DNS resolver, and whatever other aspect of my Linux system its authors think it should manage? I apologize for retreading the same discussion on this topic, but like others mentioned, adopting an incredibly complex tool doesn't mean you're simplifying. You're just working at a higher level of abstraction, which can be comforting, but simplifying would be to use the underlying systems directly or using a tool that only focuses on a single aspect of what you need (i.e. containerization). > You don't need them. Everything from the post is defined in simple environment variables. For example socket activation is maybe 3 extra lines when done from scratch. Great, then the article shouldn't import systemd bindings... My point is that the program is now tied to systemd systems. Containers don't impose such restrictions.
tbrock 5y ago
This should be part of the manual. Ive tried multiple times to understand systemd deeper than a service restart here and looking at logs for a unit there to no avail.
- proactivesvcs 5y ago
  I've found systemd to be quite well documented. Here's a few of the resources I frequent: Systemd docs: https://www.freedesktop.org/software/systemd/man/index.html https://www.freedesktop.org/software/systemd/man/index.html List of directives: https://www.freedesktop.org/software/systemd/man/systemd.directives.html https://www.freedesktop.org/software/systemd/man/systemd.dir... Unit-specific configuration: https://www.freedesktop.org/software/systemd/man/systemd.unit.html https://www.freedesktop.org/software/systemd/man/systemd.uni... Service-specific configuration: https://www.freedesktop.org/software/systemd/man/systemd.service.html https://www.freedesktop.org/software/systemd/man/systemd.ser... Timer-specific configuration: https://www.freedesktop.org/software/systemd/man/systemd.timer.html https://www.freedesktop.org/software/systemd/man/systemd.tim...
kkirsche 5y ago
As someone who wants to learn systemd, even just to understand what I’m stuck with and why it was made, this is the best introduction I’ve found. Does anyone have other resources to help beginners learn how to use systemd efficiently and it’s relationship to Docker?
- Ballas 5y ago
  You have probably already seen this, but I found this an interesting explanation to the why question: https://www.youtube.com/watch?v=o_AIw9bGogo https://www.youtube.com/watch?v=o_AIw9bGogo
- proactivesvcs 5y ago
  As a complete systemd newbie, and being pretty new to Linux, I found the documentation well-written, organised and pretty easy to understand. See https://news.ycombinator.com/item?id=27653283 https://news.ycombinator.com/item?id=27653283
tyingq 5y ago
My frustration with systemd is that it creeps out over time into other functionality, and that it's difficult to find the right documentation. Like when logind was changed to kill background processes when you log out, by making KillUserProcesses=yes the default. Some Linux distros left that as is, others overrode it in /etc/systemd/logind.conf. So, figuring out what was happening, and how to fix it, was confusing. I had no idea it would have been logind doing that. Similar for changes introduced with systemd-resolved.
- iso1210 5y ago
  Ouch, what on earth does "log out" even mean? What a terrible way to run any computer other than maybe a single user laptop.
  - kaba0 5y ago
    Not sure if sarcastic, but the systemd default makes sense. Why should random processes linger in a background, with an ill defined “not answering a signal” hack? There is no distinction between a frozen process and tmux. They can easily register a user service and continue running there.
    - tyingq 5y ago
      It wasn't respecting things that have been considered normal for a while. Even calling setsid(), which tmux does, didn't save it from being killed. Same for nohup. When the systemd people say session, they mean you would have to specifically do some kind of dbus call I'm not familiar with.
    - e12e 5y ago
      > They can easily register a user service and continue running there. Typically this was an issue on B) workstations, and B) servers, where you might log in as a regular user without super user access. And this[1] (might?) now work as advertised in the documentation - but the discussion around the bug doesn't induce great confidence... https://github.com/systemd/systemd/issues/3388 https://github.com/systemd/systemd/issues/3388 [1] example no 5 here: https://www.man7.org/linux/man-pages/man1/systemd-run.1.html#EXAMPLES https://www.man7.org/linux/man-pages/man1/systemd-run.1.html...
TruthWillHurt 5y ago
I'd suggest you stear away from using systemd and a server to launch your startup. While this is a good writeup, and you end up with a service, you still need to manage a machine with all risks involved - server reboots, updates, networking etc. AWS Fargate, or the new App Runner will manage a container almost hassle-free
stefanha 5y ago
Recently I have been wondering if systemd solves problems that are becoming less and less relevant to developers. New services are often deployed as containers. While systemd has a bunch of container-related functionality, it does not integrate well into the Kubernetes or even Docker workflow. It's used very little in those environments. If you are building CoreOS or NixOS system images, or traditional Linux system services, then systemd matters. But I think way more services are being built for the container world where these problems are solved differently. For example, the TLS configuration can be handled with common container patterns. The author's startup example would translate more easily to a full-blown Kubernetes environment once the VC funding hits their bank account if they had used containers from the start instead of first writing the service for systemd. It's a shame because systemd is very powerful and I've enjoyed using it.
- danny_sf45 5y ago
  As a developer I prefer using systemd instead of containers to deploy Golang applications. Without (Docker) containers it is: - build Go binary and install it in production server - write and enable the systemd unit file With (Docker) containers it is: - write Dockerfile - install Docker in production server - build Docker image and deploy container in production server I get the appealing of containers when one production server is used for multiple applications (e.g., you have a Golang app and a redis cache), but the example above I think containers a bit of an overkill.
  - mercora 5y ago
    i feel the same way, systemd also has some comprehensive sandboxing capabilities built-in.... i have my gripe with systemd too though. mostly with journald because it is slow, likely due to its on disk format, really could have used sqlite for this...
  - christophilus 5y ago
    Same. I deployed a fleet of transcoding servers with the worker logic being a simple Go program. It was super simple with systemd.
  - 5y ago
knorker 5y ago
My problem with systemd is that it's just so poorly written. The ideas are not inherently bad. But they're not thought through, and the implementation is pure garbage. Like taking the most stable software in the world[1], and going "nah, I'll just replace it with proof-of-concept code, leaving a TODO for error handling. It'll be fine.". And then the "awesomeness" of placing configuration files whereever the fuck you want. Like /lib. Yes, "lib" sounds like "this is where you place settings file in Unix. At least there's no other central place to put settings". [1] Yes, slight hyperbole. But I've not had Linux init crash since the mid-90s, pre-systemd
- throw-8462682 5y ago
  Care to elaborate? Off the bat, your comment comes across as a cynical rant due to its high use of strong words (garbage, fuck) and lack of examples. And even if you have anecdotes, to be convincing, it would have compare something like bug density to the software projects that collectively replaces. As written, your statement is unlikely to convince anyone that isn’t yet already.
  - j16sdiz 5y ago
    Not OP. I my experience, systemd config is simple because it handles all the complexity. Inside it’s guts, it is much more complicated than a sysv system — naturally so because it can do so much more. Those folks loves using the latest and greatest kernel function for all its glory. All works well - until it don’t. When something is broken, suddenly you have to understand all the interdependent components to debug. Back in the days, these were not so uncommon, because bugs or simply unimplemented features………
    - knorker 5y ago
      Agree with that. It breaks catastrophically. But it's also overengineered. Like starting a daemon on first connect is "neat trick", but should never have gone beyond that. Like: oh, I want to restart this daemon, because I want it to re-run its init code (possibly with new settings), but you CAN'T, because some idiot decided that it'll only actually start when someone connects to its unix socket, so running "restart" is a no-op.
kemotep 5y ago
Whenever these discussions of systemd come up, I am reminded of this talk[0]. It will be interesting to see if one day a replacement for systemd comes along and people who once championed systemd will begin to use the arguments the people who do not prefer systemd use to defend their choices for not wanting to use the next init system manager. [0]: https://youtu.be/o_AIw9bGogo https://youtu.be/o_AIw9bGogo
- einpoklum 5y ago
  > see if one day a replacement for systemd comes along Part of the critique of systemd is the basic architectural choice of having this monolithic layer between regular user apps and the kernel. So, in a sense, the idea is _not_ to replace systemd with a better-written systemd, but to do things differently.
  - kemotep 5y ago
    Yes but in doing things differently with the new one people will defend systemd the same way people defend sysvinit was my point. They will claim the new way is too complicated or tries to replace too much. I might not be getting the point of the talk but I really appreciate the argument that Benno Rice presents.
deleted 5y ago
[deleted]
baybal2 5y ago
I want to mention connman. Does everything you expect of single programmlet to manage your NTP, resolve.conf, DNS caching, mDNS, network devices, and etc. Importantly, it weights only 1/100 of SystemD
- drran 5y ago
  How to kick out SystemD first, to try connman?
  - viraptor 5y ago
    systemctl disable systemd-timesyncd systemd-resolved systemd-networkd For the services listed, that's all you need.
- SahAssar 5y ago
  Seems like that is mostly for the network-related stuff and not services, mounts, isolation/namespacing/containers, init, logging and so on. Aren't you comparing a basket of apples to an apple-slice?
  - baybal2 5y ago
    Yes, I do, but that's what I believe people look for when looking for running a server without 10 daemons to handle network configuration.
    - SahAssar 5y ago
      I don't think systemd requires 10 daemons for network config, isn't it only systemd-resolvd for DNS, systemd-networkd for actual network, and systemd-timesyncd for NTP? If you only mean to say that you replace 3 bits of systemd then it would be good to say that, and to say which bits of systemd it replaces. A lot of distros do not even use those systemd services, for example ubuntu does not use systemd-networkd by default. I read "it weights only 1/100 of SystemD" as if it compared systemd to connman directly.
candiddevmike 5y ago
Only complaint I have about systemd is their docs aren't versioned, so it's difficult to see if the functionality you're looking for is available in the version you're running. Most of the time I have to Ctrl F through the NEWS doc in their repo.
fidesomnes 5y ago
The most ironic title in a while.
WesolyKubeczek 5y ago
For me, systemd is the best thing since sliced bread. As a programmer, I now don't need to care about dropping privileges, managing logging, daemonization (the moment I need to do the double-fork dance again, chairs will be flying, I swear), dropping into a chroot, and do half-arsed guesses "is it up and running yet?" from a convoluted mess of shell code that looks like a bunch of hair stuck down a drain for a month. I just write an easy-to-debug program which I can launch from command line and see it run, and when I'm satisfied, a systemd unit from a cookie-cutter template is going to make it run. Service dependencies are now a breeze, too. If I need to limit resources, I can just declare the limits in the unit. If I want a custom networking/mount namespace, it's taken care of.
- einpoklum 5y ago
  1. This can, and should, all be done without systemd, and not with idiosyncratic shell scripts and guesses. 2. Some of the systemd logging is binary, so good luck with that if there's a problem. 3. Have you tried non-systemd init systems other than sysvinit? 4. Yes, it is convenient when everything below your development is cenrtalized by a single entity. It can easily provide a consistently useful underpinning. But there's a price - overly strong coupling of the init system, the kernel and part of the user-space, centralized control of, well, almost all of how things work on the system, and stagnation of the ecosystem due to there being only one game in town.
  - geofft 5y ago
    I really don't understand the scenario where binary logging is a problem. journalctl is a command just like, I dunno, gzip, and people are fine with gzipped logs. If something goes horribly wrong with your system, you're not looking at logs with an oscilloscope, you're looking at logs by mounting the disk on some other working OS - whether it's the initramfs, or a live CD, or whatever. You can run gunzip < /mnt/brokensystem/var/log/messages.1.gz from that other working OS to read text logs; you can also run journalctl --root /mnt/brokensystem to read journald logs. (Not to mention UNIX has log files that have been in a binary format since time immemorial, like utmp and wtmp.)
sgt 5y ago
If you are looking to run your containers in a very light weight way and also easily understood, you can use systemd for this and use tips from this article. You will have to do the orchestration yourself so I think it would have to be more suitable for very simple deployments with small teams and / or part time projects.
- leotaku 5y ago
  I've used systemd containers to run badly behaved GUI apps, e.g. Steam and MS Teams on my personal computers. Surprisingly good experience, the systemd manual pages are extremely comprehensive and, in my opinion, much easier to understand than people make them out to be.
einpoklum 5y ago
The title is quite triggering. As I see it, systemd itself is very complex. So, one might say "Guaranteeing complexity with systemd", though possibly being able to ignore the complexity, usually.
teknopaul 5y ago
Trying to avoid complexity by being dependent on something horribly complex is not going to work.
- ketzu 5y ago
  > Trying to avoid complexity by being dependent on something horribly complex is not going to work. It is probably unavoidable, looking at how complex modern compilers, processors and kernels are. They sure do make a lot of things simpler, though.
  - lanstin 5y ago
    Yeah if it is simple to use and has all the hooks I might want eventually, I don’t mind a complex implementation. Better one complex but popular implementation behind simple config than a bunch of people DIYing it. As long as turning on some high detail debug/tracing is possible so I can debug the complexity if needed
PaulHoule 5y ago
I don’t see it as a systemd-ism but that the UNIX way is maturing. Passing an arbitrary fd or socket from one process to another solves many problems and we are in the habit of doing it now.
CGamesPlay 5y ago
In the TLS example, how does systemd handle certificate rotation? Are the certificate files symlinks, hard links, or copies? If hard links or copies, presumably my service will need to be signaled or restarted to get the new certificate; does systemd do this?
robertlagrant 5y ago
To;Dr: good lord it's hard to avoid complexity when you're not using Docker to run your app.

291 points by irl_ 5y ago

Use NixOS and you'll love systemd. You'll be defining your own systemd units with ease. systemd to you will be journalctl and systemctl. So pretty good.

atoav 5y ago
I use mostly Ubuntu and Debian and defining systemd units just means you have to spit the right text into a .service file placed at the right spot. How does NixOS make that easier?
- nix23 5y ago
  https://nixos.wiki/wiki/NixOS:extend_NixOS https://nixos.wiki/wiki/NixOS:extend_NixOS
- siraben 5y ago
  In NixOS it's trivial to use and define NixOS modules, which handle restarting/starting/stopping systemd units. See the wiki example[0] on how to define and use a service that greets the user with GNU Hello. Also, since you have access to Nixpkgs, you can make the ExecStart as complicated as you want with whatever dependencies you desire, and trivially share it with others. [0] https://nixos.wiki/wiki/Module#Example https://nixos.wiki/wiki/Module#Example
  - drran 5y ago
    It looks like NixWay of doing NixThings to solve NixProblems. Why I need NixModules in the first place? ELI5, please.
    - leotaku 5y ago
      It's basically just a nice way of composing the different parts of your system. E.g. you could write a NixOS module to manage your web app. You declaratively configure that you want to run Nginx, Postgres, etc. open some ports, connect to your VPN and basically everything else you might want. There are a lot of existing modules, so in most cases you have to write very little code yourself. If you want to scale your system to run on multiple systems or containers, you can do that with relative ease. You also have a singular source of truth for all of your information, down to your application binaries. Your configuration says you are using Postgres version X.Y.Z with compilation options A, B and C, so that's exactly what is running on your systems.
    - diragon 5y ago
      It's an academically more impressive version of Ansible that's deeply intertwined with the distro's package manager.
- goodpoint 5y ago
  It does not.
- nh2 5y ago
  You can override NixOS's predefined systemd settings from _outside_ using NixOS module options. This allows you to change default settings that are not optimal for your use case, without having to patch NixOS itself, or write your own unit config. For example, systemd by default permanently gives up restarting services after a few number of tries (e.g. 5), even if you have set `Restart=always`. This is suboptimal for web servers that should recover by themselves after arbitrarily long failures (e.g. network downtimes outside of your control). On NixOS, you can, from your machine config, set: systemd.services.nginx.unitConfig.StartLimitIntervalSec = 0; This sets/overrides just that specific systemd option for the existing nginx module. On other distros, you often have to resort to global mutation in `/etc` that does not compose well. We use NixOS for our infra (having used Ansible before), and this ability to override anything cleanly and keeping defaults otherwise made for much easier to maintain infra code and less ugly/surprising compromises.
  - Arnavion 5y ago
    >On other distros, you often have to resort to global mutation in `/etc` that does not compose well. Why does this "not compose well" ? You don't have to override the whole unit as /etc/systemd/system/nginx.service , which would have problems if two things wanted to override different parts of the original unit. Just drop an override file in /etc/systemd/system/nginx.service.d/90-restart-always.conf with that one specific config you want to override.
    - nh2 5y ago
      > Why does this "not compose well" ? Because you cannot easily write libraries/components that do this. In NixOS, other modules can override the options of other modules. For example, a a web app can set the nginx options that it needs, instead of requiring you (the admin) to "drop a file" in /etc. This is one of the reasons why on Ansible Galaxy (community repository of Ansible roles) there are 527 nginx roles [1], and in NixOS there is 1 nginx module that everybody code-reuses. [1]: https://galaxy.ansible.com/search?deprecated=false&tags=web&keywords=nginx&order_by=-relevance&page=1 https://galaxy.ansible.com/search?deprecated=false&tags=web&...
iso1210 5y ago
If systemd was just a replacement for starting scripts that would be one thing. when you have to run "systemctl disable systemd-timesyncd systemd-resolved systemd-networkd" to start to get back to sanity, that's not init
GuidoW 5y ago
> Use NixOS and you'll love systemd. I use NixOS but certainly not love systemd. Instead, I've created a way to replace it with s6.[1] 1: https://sr.ht/~guido/nixos-init-freedom/ https://sr.ht/~guido/nixos-init-freedom/ Cheers, Guido.
- forgotpwd16 5y ago
  That's nice and it showcases how Nix can create a declarative process management atop a script-based imperative manager. How is your experience with it? Also note that there's https://github.com/svanderburg/nix-processmgmt https://github.com/svanderburg/nix-processmgmt, a manager agnostic processes management framework supporting s6 among others, but your way seems a bit more straightforward.
  - GuidoW 5y ago
    Thanks, My use case is to get my VPS run a web and mail service on Nixos but without the bloated binary logging of journalcontrol. The indexes in these log files change heavily between snapshots so they take up way more disk space than append-only text logs, that snapshots very well on ZFS. My experiences while building: - It's easy to use the config.system.services tree of the user-services (ssh, bind, caddy, etc) to create s6-services; - Sometimes it needs a change to make it work [1] on s6; - That same change seems oblivious to systemd. I guess it just starts the process and never bothers to monitor liveness. So much for a process management system ;-) - Nixos packagers seems overworked, as my pull requests seem to get stuck ;-( - Nixos use of systemd leaves a lot of decisions to resolve at boot time, decisions that I want to make at build time with s6; - However, I cannot create a s6 dependency tree specification at build time, that's still at run time; - Because S6 uses the service-directory to store state-files in the same directory (no /etc-/var split). 1: https://github.com/NixOS/nixpkgs/pull/122844 https://github.com/NixOS/nixpkgs/pull/122844

m4r35n357 5y ago

Sick of trying to work out what language, why not mention it?

folex 5y ago

I just learned about S6 recently. It simplified daemon supervision for me a lot.

cpach 5y ago
s6 is very cool. Just out of curiosity, have you found any specific use-cases for it yet?

djhworld 5y ago

Really enjoyed reading this article. The LoadCredentials thing reminds me of configmaps in K8S, is there a more general thing in systemd e.g LoadConfig

mhitza 5y ago
Disclaimer, no idea what LoadConfig does. A more generic approach than LoadCredentials I think is the EnvironmentFile= directive if you want to pass along multiple env variables to your process without individual Environment= directives

Dah00n 5y ago

I read that as meaning "avoid systemd's complexity" as opposed to "avoid complexity by doing so and so using systemd". Oh well.

Dinux 5y ago

I'm glad to see more people sway to systemd. Systemd is 10 years in the making and it was met with skepticism right from the first day. Some of that is now slowly changing with systemd being accepted in more and more distributions. Service and runlevel management wasn't any better in the sysv era, nor were any of the multitude of custom start and boot scripts. I remember when it took multiple days testing the configuration on different distributions, editions and versions to just get a single daemon process to start without failure. Then do the whole thing over again because the Debian based distros did not use the RedHat startup tools, different (or no) runlevel configurations, different service binders, different NTP daemon, different terminal deamon, etc.. And of course the French customers wanted the service to work with Mandriva, the German customers want SUSE support with extended security options like dropping permissions after boot. Just like the article mentions you can define a portable service model with security and failure handling built in. There wasn't even anything that came close back in the day. Systemd may not have been built with the Unix philosophy in mind, but at some point that becomes a secondary concern. Systemd unifies all systemd resources in units which work anywhere, its expandable and extendable, user friendly, allows for remote monitoring etc.

zozbot234 5y ago
There's a lot of poorly-understood incidental complexity in the systemd codebase, and this can bite users even when doing basic service and runlevel management. The systemd approach is to try and make it 100% declarative based on simple .ini files, but the semantics of this seemingly "declarative" configuration was never properly specified. Even many systemd fans seem to be quite aware of this, and there seems to be a common understanding that some ground-up reimplementation of these ideas based on a clearer underlying "philosophy" will be needed at some point. Systemd has been a successful experiment in many ways, but relying on throwaway experimental code for one's basic production needs is not a good idea.
- southerntofu 5y ago
  I hope you're right and we get a better, simpler declarative format in the future. systemd isn't bad but i've found when writing service files that some stuff is not as obvious as it should be.

turminal 5y ago

The title is an oxymoron. Putting a huge complex piece of software between yourself and "complexity" doesn't make the system less complex.

smitty1e 5y ago
Systemd seems to do a good job of moving the complexity of managing the privileged/unprivileged divide into a standardized service. I sympathize with the "transition sucks" sentiments elsewhere on this post. Having a bunch of working scripts turned into instant technical debt cannot be pleasant. But, as with python3, systemd seems to be the way things are headed.
- deleted 5y ago
  [deleted]
- birdyrooster 5y ago
  Why are we talking like this? We’ve been using systemd for over 5 years. It’s weird. Time loops.
  - boublepop 5y ago
    It took 10 years before the “python2 will never die”-crowd finally accepted they where not going to win at that python3 was here to stay. People spend a couple of years getting used to a stack in their early carrier and then spend decades arguing that it should never change.
    - kaba0 5y ago
      But python programs are much more complex than init scripts systemd replace. It is more like people hate change and are lazy to learn a new thing.
      - smitty1e 5y ago
        Mamagement has 99 problems. Holding a status quo is about minimizing the backlog more than hating change as such. Even when we can show am improvement in security and usability, and lower training cost because of consistency, it's still another mouth to feed.

bitwize 5y ago

You incur complexity with systemd. You avoid complexity with runit.

cpach 5y ago
IMO runit is abandonware at this point. No release since 2014. Have you looked at s6? It’s a compelling alternative.
- ori_b 5y ago
  Runit is finished at this point. It does what it says, and I haven't run into bugs. Churn isn't a virtue.
  - h1x 5y ago
    Exactly this. Runit was designed to be simple and in simple software, at some point, there is just not much to improve.
- aerique 5y ago
  Void Linux, "the BSD of Linux", uses runit and it has been fine since forever. If there were any issues they would be worked on.
  - cpach 5y ago
    I’m glad to hear runit works fine for them. I’m not so sure there isn’t any room for improvement though. In the related s6 project there is lot of discussions about adding new features that are beneficial for supporting a modern Linux distro.
    - imiric 5y ago
      As a datapoint of one, I've been using Void Linux for near a decade. I haven't once been in a situation where I thought systemd would help. Granted, I don't run many custom services, but for what it does, runit does a great job. Something a bit more user friendly like s6 would be nice, but otherwise it stays out of my way and I don't think about it. NixOS/Guix would be worthwhile switching to, but Void is comfortably simple.

imiric 5y ago

Apologies, I was also triggered by the title. :) > systemd provides ways to restrict the parts of the filesystem the service can see. So like chroot and namespaces? Why do I have to depend on systemd when these are native features provided by Linux? So systemd provides a friendlier abstraction of these concepts. Great, but so do Docker and Podman and many other tools that can actually be installed without taking over the rest of the system. Having your application actually use systemd libraries further increases this dependency and makes it no longer usable but on a subset of Linux machines. This would be fine for some controlled production deployment, but is awful for usability and adoption.

viraptor 5y ago
> So like chroot and namespaces? Why do I have to depend on systemd when these are native features provided by Linux? Not like namespaces - using namespaces. And for the same reason we use other high level abstractions and high level languages rather than handcrafted assembly. You don't have to depend on it either - you can still use chroot instead of you want, but it's more work that way. > Great, but so do Docker and Podman and many other tools that can actually be installed without taking over the rest of the system. Docker installs a service which takes over lifecycle management, restarts, and traffic proxying for apps. It injects and managed multiple firewall chains. It pretty much takes over network management. And it's still stuck on the old cgroups format so it forces that on your system. It really doesn't win this comparison. > Having your application actually use systemd libraries You don't need them. Everything from the post is defined in simple environment variables. For example socket activation is maybe 3 extra lines when done from scratch.
- imiric 5y ago
  > You don't have to depend on it either - you can still use chroot instead of you want, but it's more work that way. How so? If I need filesystem isolation, I'll use the simplest tool that provides it. In this case probably a container runtime. Note that none of your criticism about Docker applies to Podman. Why would I ever want to use containers with a tool that forces (OK, strongly suggests...) me to use it as an init system, logging system, network manager, DNS resolver, and whatever other aspect of my Linux system its authors think it should manage? I apologize for retreading the same discussion on this topic, but like others mentioned, adopting an incredibly complex tool doesn't mean you're simplifying. You're just working at a higher level of abstraction, which can be comforting, but simplifying would be to use the underlying systems directly or using a tool that only focuses on a single aspect of what you need (i.e. containerization). > You don't need them. Everything from the post is defined in simple environment variables. For example socket activation is maybe 3 extra lines when done from scratch. Great, then the article shouldn't import systemd bindings... My point is that the program is now tied to systemd systems. Containers don't impose such restrictions.

tbrock 5y ago

This should be part of the manual. Ive tried multiple times to understand systemd deeper than a service restart here and looking at logs for a unit there to no avail.

proactivesvcs 5y ago
I've found systemd to be quite well documented. Here's a few of the resources I frequent: Systemd docs: https://www.freedesktop.org/software/systemd/man/index.html https://www.freedesktop.org/software/systemd/man/index.html List of directives: https://www.freedesktop.org/software/systemd/man/systemd.directives.html https://www.freedesktop.org/software/systemd/man/systemd.dir... Unit-specific configuration: https://www.freedesktop.org/software/systemd/man/systemd.unit.html https://www.freedesktop.org/software/systemd/man/systemd.uni... Service-specific configuration: https://www.freedesktop.org/software/systemd/man/systemd.service.html https://www.freedesktop.org/software/systemd/man/systemd.ser... Timer-specific configuration: https://www.freedesktop.org/software/systemd/man/systemd.timer.html https://www.freedesktop.org/software/systemd/man/systemd.tim...

kkirsche 5y ago

As someone who wants to learn systemd, even just to understand what I’m stuck with and why it was made, this is the best introduction I’ve found. Does anyone have other resources to help beginners learn how to use systemd efficiently and it’s relationship to Docker?

Ballas 5y ago
You have probably already seen this, but I found this an interesting explanation to the why question: https://www.youtube.com/watch?v=o_AIw9bGogo https://www.youtube.com/watch?v=o_AIw9bGogo
proactivesvcs 5y ago
As a complete systemd newbie, and being pretty new to Linux, I found the documentation well-written, organised and pretty easy to understand. See https://news.ycombinator.com/item?id=27653283 https://news.ycombinator.com/item?id=27653283

tyingq 5y ago

My frustration with systemd is that it creeps out over time into other functionality, and that it's difficult to find the right documentation. Like when logind was changed to kill background processes when you log out, by making KillUserProcesses=yes the default. Some Linux distros left that as is, others overrode it in /etc/systemd/logind.conf. So, figuring out what was happening, and how to fix it, was confusing. I had no idea it would have been logind doing that. Similar for changes introduced with systemd-resolved.

iso1210 5y ago
Ouch, what on earth does "log out" even mean? What a terrible way to run any computer other than maybe a single user laptop.
- kaba0 5y ago
  Not sure if sarcastic, but the systemd default makes sense. Why should random processes linger in a background, with an ill defined “not answering a signal” hack? There is no distinction between a frozen process and tmux. They can easily register a user service and continue running there.
  - tyingq 5y ago
    It wasn't respecting things that have been considered normal for a while. Even calling setsid(), which tmux does, didn't save it from being killed. Same for nohup. When the systemd people say session, they mean you would have to specifically do some kind of dbus call I'm not familiar with.
  - e12e 5y ago
    > They can easily register a user service and continue running there. Typically this was an issue on B) workstations, and B) servers, where you might log in as a regular user without super user access. And this[1] (might?) now work as advertised in the documentation - but the discussion around the bug doesn't induce great confidence... https://github.com/systemd/systemd/issues/3388 https://github.com/systemd/systemd/issues/3388 [1] example no 5 here: https://www.man7.org/linux/man-pages/man1/systemd-run.1.html#EXAMPLES https://www.man7.org/linux/man-pages/man1/systemd-run.1.html...

TruthWillHurt 5y ago

I'd suggest you stear away from using systemd and a server to launch your startup. While this is a good writeup, and you end up with a service, you still need to manage a machine with all risks involved - server reboots, updates, networking etc. AWS Fargate, or the new App Runner will manage a container almost hassle-free

stefanha 5y ago

Recently I have been wondering if systemd solves problems that are becoming less and less relevant to developers. New services are often deployed as containers. While systemd has a bunch of container-related functionality, it does not integrate well into the Kubernetes or even Docker workflow. It's used very little in those environments. If you are building CoreOS or NixOS system images, or traditional Linux system services, then systemd matters. But I think way more services are being built for the container world where these problems are solved differently. For example, the TLS configuration can be handled with common container patterns. The author's startup example would translate more easily to a full-blown Kubernetes environment once the VC funding hits their bank account if they had used containers from the start instead of first writing the service for systemd. It's a shame because systemd is very powerful and I've enjoyed using it.

danny_sf45 5y ago
As a developer I prefer using systemd instead of containers to deploy Golang applications. Without (Docker) containers it is: - build Go binary and install it in production server - write and enable the systemd unit file With (Docker) containers it is: - write Dockerfile - install Docker in production server - build Docker image and deploy container in production server I get the appealing of containers when one production server is used for multiple applications (e.g., you have a Golang app and a redis cache), but the example above I think containers a bit of an overkill.
- mercora 5y ago
  i feel the same way, systemd also has some comprehensive sandboxing capabilities built-in.... i have my gripe with systemd too though. mostly with journald because it is slow, likely due to its on disk format, really could have used sqlite for this...
- christophilus 5y ago
  Same. I deployed a fleet of transcoding servers with the worker logic being a simple Go program. It was super simple with systemd.
- 5y ago

knorker 5y ago

My problem with systemd is that it's just so poorly written. The ideas are not inherently bad. But they're not thought through, and the implementation is pure garbage. Like taking the most stable software in the world[1], and going "nah, I'll just replace it with proof-of-concept code, leaving a TODO for error handling. It'll be fine.". And then the "awesomeness" of placing configuration files whereever the fuck you want. Like /lib. Yes, "lib" sounds like "this is where you place settings file in Unix. At least there's no other central place to put settings". [1] Yes, slight hyperbole. But I've not had Linux init crash since the mid-90s, pre-systemd

throw-8462682 5y ago
Care to elaborate? Off the bat, your comment comes across as a cynical rant due to its high use of strong words (garbage, fuck) and lack of examples. And even if you have anecdotes, to be convincing, it would have compare something like bug density to the software projects that collectively replaces. As written, your statement is unlikely to convince anyone that isn’t yet already.
- j16sdiz 5y ago
  Not OP. I my experience, systemd config is simple because it handles all the complexity. Inside it’s guts, it is much more complicated than a sysv system — naturally so because it can do so much more. Those folks loves using the latest and greatest kernel function for all its glory. All works well - until it don’t. When something is broken, suddenly you have to understand all the interdependent components to debug. Back in the days, these were not so uncommon, because bugs or simply unimplemented features………
  - knorker 5y ago
    Agree with that. It breaks catastrophically. But it's also overengineered. Like starting a daemon on first connect is "neat trick", but should never have gone beyond that. Like: oh, I want to restart this daemon, because I want it to re-run its init code (possibly with new settings), but you CAN'T, because some idiot decided that it'll only actually start when someone connects to its unix socket, so running "restart" is a no-op.

kemotep 5y ago

Whenever these discussions of systemd come up, I am reminded of this talk[0]. It will be interesting to see if one day a replacement for systemd comes along and people who once championed systemd will begin to use the arguments the people who do not prefer systemd use to defend their choices for not wanting to use the next init system manager. [0]: https://youtu.be/o_AIw9bGogo https://youtu.be/o_AIw9bGogo

einpoklum 5y ago
> see if one day a replacement for systemd comes along Part of the critique of systemd is the basic architectural choice of having this monolithic layer between regular user apps and the kernel. So, in a sense, the idea is _not_ to replace systemd with a better-written systemd, but to do things differently.
- kemotep 5y ago
  Yes but in doing things differently with the new one people will defend systemd the same way people defend sysvinit was my point. They will claim the new way is too complicated or tries to replace too much. I might not be getting the point of the talk but I really appreciate the argument that Benno Rice presents.

deleted 5y ago

[deleted]

baybal2 5y ago

I want to mention connman. Does everything you expect of single programmlet to manage your NTP, resolve.conf, DNS caching, mDNS, network devices, and etc. Importantly, it weights only 1/100 of SystemD

drran 5y ago
How to kick out SystemD first, to try connman?
- viraptor 5y ago
  systemctl disable systemd-timesyncd systemd-resolved systemd-networkd For the services listed, that's all you need.
SahAssar 5y ago
Seems like that is mostly for the network-related stuff and not services, mounts, isolation/namespacing/containers, init, logging and so on. Aren't you comparing a basket of apples to an apple-slice?
- baybal2 5y ago
  Yes, I do, but that's what I believe people look for when looking for running a server without 10 daemons to handle network configuration.
  - SahAssar 5y ago
    I don't think systemd requires 10 daemons for network config, isn't it only systemd-resolvd for DNS, systemd-networkd for actual network, and systemd-timesyncd for NTP? If you only mean to say that you replace 3 bits of systemd then it would be good to say that, and to say which bits of systemd it replaces. A lot of distros do not even use those systemd services, for example ubuntu does not use systemd-networkd by default. I read "it weights only 1/100 of SystemD" as if it compared systemd to connman directly.

candiddevmike 5y ago

Only complaint I have about systemd is their docs aren't versioned, so it's difficult to see if the functionality you're looking for is available in the version you're running. Most of the time I have to Ctrl F through the NEWS doc in their repo.

fidesomnes 5y ago

The most ironic title in a while.

WesolyKubeczek 5y ago

For me, systemd is the best thing since sliced bread. As a programmer, I now don't need to care about dropping privileges, managing logging, daemonization (the moment I need to do the double-fork dance again, chairs will be flying, I swear), dropping into a chroot, and do half-arsed guesses "is it up and running yet?" from a convoluted mess of shell code that looks like a bunch of hair stuck down a drain for a month. I just write an easy-to-debug program which I can launch from command line and see it run, and when I'm satisfied, a systemd unit from a cookie-cutter template is going to make it run. Service dependencies are now a breeze, too. If I need to limit resources, I can just declare the limits in the unit. If I want a custom networking/mount namespace, it's taken care of.

einpoklum 5y ago
1. This can, and should, all be done without systemd, and not with idiosyncratic shell scripts and guesses. 2. Some of the systemd logging is binary, so good luck with that if there's a problem. 3. Have you tried non-systemd init systems other than sysvinit? 4. Yes, it is convenient when everything below your development is cenrtalized by a single entity. It can easily provide a consistently useful underpinning. But there's a price - overly strong coupling of the init system, the kernel and part of the user-space, centralized control of, well, almost all of how things work on the system, and stagnation of the ecosystem due to there being only one game in town.
- geofft 5y ago
  I really don't understand the scenario where binary logging is a problem. journalctl is a command just like, I dunno, gzip, and people are fine with gzipped logs. If something goes horribly wrong with your system, you're not looking at logs with an oscilloscope, you're looking at logs by mounting the disk on some other working OS - whether it's the initramfs, or a live CD, or whatever. You can run gunzip < /mnt/brokensystem/var/log/messages.1.gz from that other working OS to read text logs; you can also run journalctl --root /mnt/brokensystem to read journald logs. (Not to mention UNIX has log files that have been in a binary format since time immemorial, like utmp and wtmp.)

sgt 5y ago

If you are looking to run your containers in a very light weight way and also easily understood, you can use systemd for this and use tips from this article. You will have to do the orchestration yourself so I think it would have to be more suitable for very simple deployments with small teams and / or part time projects.

leotaku 5y ago
I've used systemd containers to run badly behaved GUI apps, e.g. Steam and MS Teams on my personal computers. Surprisingly good experience, the systemd manual pages are extremely comprehensive and, in my opinion, much easier to understand than people make them out to be.

einpoklum 5y ago

The title is quite triggering. As I see it, systemd itself is very complex. So, one might say "Guaranteeing complexity with systemd", though possibly being able to ignore the complexity, usually.

teknopaul 5y ago

Trying to avoid complexity by being dependent on something horribly complex is not going to work.

ketzu 5y ago
> Trying to avoid complexity by being dependent on something horribly complex is not going to work. It is probably unavoidable, looking at how complex modern compilers, processors and kernels are. They sure do make a lot of things simpler, though.
- lanstin 5y ago
  Yeah if it is simple to use and has all the hooks I might want eventually, I don’t mind a complex implementation. Better one complex but popular implementation behind simple config than a bunch of people DIYing it. As long as turning on some high detail debug/tracing is possible so I can debug the complexity if needed

PaulHoule 5y ago

I don’t see it as a systemd-ism but that the UNIX way is maturing. Passing an arbitrary fd or socket from one process to another solves many problems and we are in the habit of doing it now.

CGamesPlay 5y ago

In the TLS example, how does systemd handle certificate rotation? Are the certificate files symlinks, hard links, or copies? If hard links or copies, presumably my service will need to be signaled or restarted to get the new certificate; does systemd do this?

robertlagrant 5y ago

To;Dr: good lord it's hard to avoid complexity when you're not using Docker to run your app.