12 ms·

Runsit – A process manager in Go

57 points by armenb 12y ago

armenb 12y ago
Just an introduction: Currently I'm using runsit as an alternative for supervisord and I'm happy with it so far. Stdout and stderror of the processes can be queried with a very simple HTTP interface. Runsit watches a config directory for any changes and applies them immediately. Config files are in json format.
- fcoury 12y ago
  And the JSON parser is awesome, it gives user friendly error messages when the file has the wrong format. Very cool.
  - armenb 12y ago
    Yes it is.
- elithrar 12y ago
  Mind sharing a (lightly commented) config? I've been using Supervisor to run my Go services for a long while (the built-in log rotation is one of the big attractions) but keen to try alternatives. I never found mmonit and other alternatives to be as comprehensive as Supervisor.
  - armenb 12y ago
    For instance I use following config (nginx.json) for nginx: { "user": ["_env", "${USER}"], "cwd": "/var/www", "standardEnv": true, "numFiles": 1024, "binary": "/usr/sbin/nginx" } And php-fpm.json: { "user": ["_env", "${USER}"], "cwd": "/usr/sbin", "standardEnv": true, "numFiles": 1024, "binary": "php-fpm", "args": [ "-F" ] } Hope that helps
    - armenb 12y ago
      Forgot to tell you that nginx shouldn't be daemonized: echo "daemon off;" >> /etc/nginx/nginx.conf
akerl_ 12y ago
Is there a site for this or some nature of docs? I'm still searching for a process manager that I can love for use with Docker (I've played with runit and am currently playing with s6), but the total lack of readme or docs makes this link fairly unhelpful.
- armenb 12y ago
  Unfortunately there isn't much doc, but it's not that hard to set up. Take a look at run.sh file and config directory. Andrew Gerrand has a init script[1] in his fork for it which might be useful. [1] https://github.com/nf/runsit/tree/master/doc/initd https://github.com/nf/runsit/tree/master/doc/initd
- stormbrew 12y ago
  Huh. I have somehow never heard of s6 but it looks very promising. What did you find when using it for this purpose?
  - akerl_ 12y ago
    My initial thoughts were less than positive, just because building it is a less-than-stellar process. Part of that is because it has a couple of deps that must also be compiled (skalibs and execline), and part is because it follows the slashpackage conventions (http://cr.yp.to/slashpackage.html http://cr.yp.to/slashpackage.html). Now that I've got it build, the next hurdle is making init scripts in execline. It's a fairly simple language, and I enjoy the premise behind it, where scripts should be clear and deterministic. Overall, s6 provides a lot more helper tools for daemon management than runit did, so it looks like it's gonna be great for my use case. I've got an automated build set up to handle making a Docker image with s6 prepared: https://registry.hub.docker.com/u/dock0/service/ https://registry.hub.docker.com/u/dock0/service/
    - srean 12y ago
      Seems you are knowledgeable about these matters, I am not. So could you put in perspective why would runsit be interesting. I have run into several names that claim to handle this problem, you have mentioned s6 and runit, then there is monit, launchd, supervisor... Are there some glaring lacks that needs to be filled. Do they all do the same thing or do they have specialized niches ? I guess I am asking for a lot, no need to indulge, but if you do, much appreciated. @akerl_19 Thanks. I have but two upvotes to give.
chubot 12y ago
FWIW I also started writing a init-like server in Go. One thing I ran into was that Go's APIs sort of coerce you into having an extra thread per process. runsit also has this issue. See line 489 of runsit.go, pasted below. You can do it non-portably in Go by using os.ForkExec and Wait4(-1). The portable exec package assumes you will call Wait(pid), and not Wait(-1), which basically implies using a thread per process. Go's runtime isn't magic -- if you call libc/syscall wait(), an entire thread will be blocked, and the runtime can't use it for anything else. In this case this is the lifetime of an entire process, which is forever for server processes. I'm pretty sure nobody would use a real PID 1 that burned a thread per process (systemd, upstart, etc.). But yes, for most use cases, in the grand scheme of things, it's probably not a big deal. I suppose Linux has an O(1) scheduler, although I'm not quite sure how this affects scheduling (interested in any comments). But this goes to show that portable APIs are awkward and obscure for low level code. Better to use raw Unix APIs for something like an init server. Python and Java have similar problems. IMO all interesting code nowadays is POSIX-like, so we should drop the pretension of portability and simplify our lives. Unix works. // run in its own goroutine func (in *TaskInstance) awaitDeath() { in.waitErr = in.cmd.Wait() // ties up an OS thread for the lifetime of a process ... }
- zemo 12y ago
  A goroutine is not a thread. That only ties up that one goroutine.
  - chubot 12y ago
    Your first statement is true; the second isn't. Re-read what I wrote. If that doesn't convince you, then download and run the code. Run "pstree" on it and observe how many child processes and threads there are. You'll learn something useful about the relationship of the Go runtime to the OS.
  - zimbatm 12y ago
    A thread is created for blocking syscalls. I don't know if it's a syscall under the hood here but he might be talking about that.
gwoo 12y ago
I built https://github.com/gwoo/goforever https://github.com/gwoo/goforever which has similar goals. I definitely like some of the ideas in runsit like automatic config watching. I still need to handle log rotation with something like https://github.com/natefinch/lumberjack https://github.com/natefinch/lumberjack Thanks for putting this out there.
aktau 12y ago
I'm a big fan of Go and have a few Go binaries running on hundreds of clients (and a few servers) right now. They are being managed by Runit though. Since the names are so similar, did you get inspiration from Runit? And if so, what would be the main differences/advantages besides being portable to more platforms (Windows I presume)? @chubot mentions the thread-per-process overhead. Runit does process-per-process so in that aspect Runsit should be a bit lighter. Then again, Runit is extremely tiny, its statically compiled binaries taking up next to nothing. The wait(-1) trick sounds good, but there must be a reason why for example Runit doesn't use it, since afaik Runit only runs on POSIX systems. Keep up the good work! EDIT: a web interface, that's pretty spiffy! (though I've made something similar work for Runit by querying the status of a service and serving up a dashboard, with a Go webserver of course).
kylered 12y ago
In case anyone is interested, we open sourced a process manager and web interface a few months ago. https://github.com/VividCortex/pm https://github.com/VividCortex/pm https://github.com/VividCortex/pm-web https://github.com/VividCortex/pm-web

57 points by armenb 12y ago

Just an introduction: Currently I'm using runsit as an alternative for supervisord and I'm happy with it so far. Stdout and stderror of the processes can be queried with a very simple HTTP interface. Runsit watches a config directory for any changes and applies them immediately. Config files are in json format.

fcoury 12y ago
And the JSON parser is awesome, it gives user friendly error messages when the file has the wrong format. Very cool.
- armenb 12y ago
  Yes it is.
elithrar 12y ago
Mind sharing a (lightly commented) config? I've been using Supervisor to run my Go services for a long while (the built-in log rotation is one of the big attractions) but keen to try alternatives. I never found mmonit and other alternatives to be as comprehensive as Supervisor.
- armenb 12y ago
  For instance I use following config (nginx.json) for nginx: { "user": ["_env", "${USER}"], "cwd": "/var/www", "standardEnv": true, "numFiles": 1024, "binary": "/usr/sbin/nginx" } And php-fpm.json: { "user": ["_env", "${USER}"], "cwd": "/usr/sbin", "standardEnv": true, "numFiles": 1024, "binary": "php-fpm", "args": [ "-F" ] } Hope that helps
  - armenb 12y ago
    Forgot to tell you that nginx shouldn't be daemonized: echo "daemon off;" >> /etc/nginx/nginx.conf

akerl_ 12y ago

Is there a site for this or some nature of docs? I'm still searching for a process manager that I can love for use with Docker (I've played with runit and am currently playing with s6), but the total lack of readme or docs makes this link fairly unhelpful.

armenb 12y ago
Unfortunately there isn't much doc, but it's not that hard to set up. Take a look at run.sh file and config directory. Andrew Gerrand has a init script[1] in his fork for it which might be useful. [1] https://github.com/nf/runsit/tree/master/doc/initd https://github.com/nf/runsit/tree/master/doc/initd
stormbrew 12y ago
Huh. I have somehow never heard of s6 but it looks very promising. What did you find when using it for this purpose?
- akerl_ 12y ago
  My initial thoughts were less than positive, just because building it is a less-than-stellar process. Part of that is because it has a couple of deps that must also be compiled (skalibs and execline), and part is because it follows the slashpackage conventions (http://cr.yp.to/slashpackage.html http://cr.yp.to/slashpackage.html). Now that I've got it build, the next hurdle is making init scripts in execline. It's a fairly simple language, and I enjoy the premise behind it, where scripts should be clear and deterministic. Overall, s6 provides a lot more helper tools for daemon management than runit did, so it looks like it's gonna be great for my use case. I've got an automated build set up to handle making a Docker image with s6 prepared: https://registry.hub.docker.com/u/dock0/service/ https://registry.hub.docker.com/u/dock0/service/
  - srean 12y ago
    Seems you are knowledgeable about these matters, I am not. So could you put in perspective why would runsit be interesting. I have run into several names that claim to handle this problem, you have mentioned s6 and runit, then there is monit, launchd, supervisor... Are there some glaring lacks that needs to be filled. Do they all do the same thing or do they have specialized niches ? I guess I am asking for a lot, no need to indulge, but if you do, much appreciated. @akerl_19 Thanks. I have but two upvotes to give.

chubot 12y ago

FWIW I also started writing a init-like server in Go. One thing I ran into was that Go's APIs sort of coerce you into having an extra thread per process. runsit also has this issue. See line 489 of runsit.go, pasted below. You can do it non-portably in Go by using os.ForkExec and Wait4(-1). The portable exec package assumes you will call Wait(pid), and not Wait(-1), which basically implies using a thread per process. Go's runtime isn't magic -- if you call libc/syscall wait(), an entire thread will be blocked, and the runtime can't use it for anything else. In this case this is the lifetime of an entire process, which is forever for server processes. I'm pretty sure nobody would use a real PID 1 that burned a thread per process (systemd, upstart, etc.). But yes, for most use cases, in the grand scheme of things, it's probably not a big deal. I suppose Linux has an O(1) scheduler, although I'm not quite sure how this affects scheduling (interested in any comments). But this goes to show that portable APIs are awkward and obscure for low level code. Better to use raw Unix APIs for something like an init server. Python and Java have similar problems. IMO all interesting code nowadays is POSIX-like, so we should drop the pretension of portability and simplify our lives. Unix works. // run in its own goroutine func (in *TaskInstance) awaitDeath() { in.waitErr = in.cmd.Wait() // ties up an OS thread for the lifetime of a process ... }

zemo 12y ago
A goroutine is not a thread. That only ties up that one goroutine.
- chubot 12y ago
  Your first statement is true; the second isn't. Re-read what I wrote. If that doesn't convince you, then download and run the code. Run "pstree" on it and observe how many child processes and threads there are. You'll learn something useful about the relationship of the Go runtime to the OS.
- zimbatm 12y ago
  A thread is created for blocking syscalls. I don't know if it's a syscall under the hood here but he might be talking about that.

gwoo 12y ago

I built https://github.com/gwoo/goforever https://github.com/gwoo/goforever which has similar goals. I definitely like some of the ideas in runsit like automatic config watching. I still need to handle log rotation with something like https://github.com/natefinch/lumberjack https://github.com/natefinch/lumberjack Thanks for putting this out there.

aktau 12y ago

I'm a big fan of Go and have a few Go binaries running on hundreds of clients (and a few servers) right now. They are being managed by Runit though. Since the names are so similar, did you get inspiration from Runit? And if so, what would be the main differences/advantages besides being portable to more platforms (Windows I presume)? @chubot mentions the thread-per-process overhead. Runit does process-per-process so in that aspect Runsit should be a bit lighter. Then again, Runit is extremely tiny, its statically compiled binaries taking up next to nothing. The wait(-1) trick sounds good, but there must be a reason why for example Runit doesn't use it, since afaik Runit only runs on POSIX systems. Keep up the good work! EDIT: a web interface, that's pretty spiffy! (though I've made something similar work for Runit by querying the status of a service and serving up a dashboard, with a Go webserver of course).

kylered 12y ago

In case anyone is interested, we open sourced a process manager and web interface a few months ago. https://github.com/VividCortex/pm https://github.com/VividCortex/pm https://github.com/VividCortex/pm-web https://github.com/VividCortex/pm-web