7 ms·

Related to the discussion: "A fork() in the road": https://www.microsoft.com/en-us/research/wp-content/uploads/2019/04/fork-hotos19.pdf https://www.microsoft.co

by rom1v 3mo ago

Related to the discussion: "A fork() in the road": https://www.microsoft.com/en-us/research/wp-content/uploads/2019/04/fork-hotos19.pdf https://www.microsoft.com/en-us/research/wp-content/uploads/... > ABSTRACT > The received wisdom suggests that Unix’s unusual combination of fork() and exec() for process creation was an inspired design. In this paper, we argue that fork was a clever hack for machines and programs of the 1970s that has long outlived its usefulness and is now a liability. We catalog the ways in which fork is a terrible abstraction for the modern programmer to use, describe how it compromises OS implementations, and propose alternatives. > As the designers and implementers of operating systems, we should acknowledge that fork’s continued existence as a first-class OS primitive holds back systems research, and deprecate it. As educators, we should teach fork as a historical artifact, and not the first process creation mechanism students encounter.

pizlonator 3mo ago
Fork is marvelous for the zygote pattern Hard to come up with an optimization that is equally efficient and elegant
- toast0 3mo ago
  The zygote pattern[1] is a great optimization to deal with the cost of forking, but IMHO, being able to inexpensively spawn a carefully tailored process regardless of the size and scope of the current process would be better. I would guess it would be a small difference in measurable performance between zygote and a direct clean spawn, but it's one less trick an application needs to do, and it would be very helpful for libraries that spawn things. Spawning inside a library isn't always a great thing to do, but some things would really benefit from process level isolation. [1] In case one isn't aware, the zygote pattern involves forking a 'zygote' process during application startup, and having that process do any forks that need to happen during application runtime. This reduces the cost of forking in large applications, because the zygote will have few fds open and use little memory. This lets your large application spawn new processes without delaying the application or the startup of the new processes. Some applications will spawn many zygotes to allow parallelism for spawning at runtime.
  - pizlonator 3mo ago
    You're referring to something else, and maybe I'm using the term "zygote" incorrectly. In all uses of zygotes that I have seen, here's what's really happening: - `fork` is being used to reduce the cost of starting a process that has a high start-up cost. So, you start one process, run it through the expensive initialization, and then fork it from there to start new processes. - To make this even faster, you have a pool of pre-forked processes sit around. - Having pre-forked processes sitting around ready to be used is not expensive because of the CoW property and the fact that a process that forks and then immediately pauses will not have triggered any significant CoW yet. So, the zygote optimization you speak of is in practice only meaningful on top of systems that are using an optimization uniquely enabled by `fork` (avoiding process initialization costs by cloning a process), and that zygote optimization is further optimized by another property of `fork` (memory sharing of forked processes that haven't done anything else yet).
    - toast0 3mo ago
      Oh I see. I guess your zygotes have developed more than mine. I think Google may have coined or at least popularized the term zygote for this in Chrome and Android, Chrome documentation [1] says: > A zygote process is one that listens for spawn requests from a main process and forks itself in response. Generally they are used because forking a process after some expensive setup has been performed can save time and share extra memory pages. I think reading the first sentance and stopping covers my zygote, but adding the second sentance covers yours. So I think we're both right! I think both paths are useful. If your children need time to startup and become ready, spawn one that does start up work, and then it (pre)forks at the ready state to have processes ready to handle requests (your zygote). This does require a traditional fork() to avoid duplication of work. But if forking is expensive at runtime because you have a million FDs open and a whole lot of memory allocations, spawn spawners before you start doing work (my zygote). This could be unnecessary with a inexpensive way to spawn a new process from an process that has lots of resources in use. Of course, you can also use my zygotes to spawn your zygotes. Zygoteception. [1] https://chromium.googlesource.com/chromium/src/+/HEAD/docs/linux/zygote.md https://chromium.googlesource.com/chromium/src/+/HEAD/docs/l...
- vlovich123 3mo ago
  The paper explicitly covers it that various memory COW/snapshot mechanisms are probably faster and safer than the zygote pattern. As it stands getting the zygote pattern correct and safe is something you have to plan for upfront. You can’t retrofit it which is why the paper mentions it has poor composability. Also the advantages of the zygote pattern can be overstated since the memory sharing benefit is minimal since it has to happen so early and modern OSes already transparently CoW duplicate pages in the background.
  - loeg 3mo ago
    In what sense can you not retrofit the zygote pattern?
    - vlovich123 3mo ago
      I recommend at least skimming the paper as it covers this. But essentially you can’t just inject a call at a random point in code to start being a zygote. It’s something you have to plan up front as to the exact point you’re going to fork and that you’re going to do it at the start of program before any threads have started or any files are open and before any locks have been acquired. It’s basically all the challenges of invoking fork at arbitrary points in time. The reason to do a zygote in the first place could be solved with alternative special APIs that are safer and harder to misuse. But we have fork so there’s not as big of a demand despite the warts.
      - loeg 3mo ago
        Sure, but you can always retrofit a program to fork early on... this is a relatively trivial change. No?
- deleted 3mo ago
  [deleted]
- p_l 3mo ago
  And so easy to make into bottleneck. Yes, zygote pattern makes it easy to make fork() into bottleneck - it requires a lot more discipline and low level tricks (linker scripts, compiler-specific extensions, custom sections, low level dependencies on pagesize that get "fun" on ARM servers). If you don't, you might wake up with fork() causing latency issues.
- cyberax 3mo ago
  Unless you want to create a thread in your zygote. Then it breaks down. Raw fork() is terrible. Instead we need a proper primitive to stop and make a snapshot of a process.
  - pizlonator 3mo ago
    You can create threads in the zygote. It doesn't "break down", but sure, there's a bit more work. My trick for that is that the set of threads that I create pre fork have to be suspendable and resumable, preferably lazily (they resume when they are actually needed). So, the zygotes are sitting with those threads suspended. When they become active, they can do work immediately. They might lazily resume those threads as needed. There are other idioms for this too. > Raw fork() is terrible. Instead we need a proper primitive to stop and make a snapshot of a process. Folks have been saying that it's terrible for as long as I can remember. But it's still there, because it's better than the alternatives
    - cyberax 3mo ago
      > My trick for that is that the set of threads that I create pre fork have to be suspendable and resumable Well, yes. You need to wait for all the threads to park themselves at safepoints. This can work if you control the whole runtime, and you don't use something that creates threads behind your back. This is actually why I've always been interested in a better fork(), it has a lot of parallels with stop-the-world needed for GCs. > Folks have been saying that it's terrible for as long as I can remember. But it's still there, because it's better than the alternatives I don't think we have alternatives? Except maybe ptrace()?
anarazel 3mo ago
It is somewhat interesting that the most widely used "big" OS that doesn't use fork, i.e. Windows, has dog slow process creation... I agree that there should be non-fork primitives, I'm just not that sure that performance is the best argument.
- pjmlp 3mo ago
  Because that OS best practices is to use threads. Traditionally Windows applications that create processes all the time come from UNIX heritage. Contrary to UNIX, Windows NT was designed with threads first mentality, from the get go. While on UNIX they were added after fact, and to this day there are gotchas mixing posix threads with signals, fork and exec.
  - zozbot234 3mo ago
    Windows was designed with threads-first mentality because on pre-386 machines you don't have viable process memory protection, so your tasks share memory by necessity. This is not a great argument.
    - JdeBP 3mo ago
      Windows NT was never designed with pre-386 machines in mind. That was the territory of the old DOS+Windows. Windows NT from the get-go was for machines with page-based virtual memory. * https://computernewb.com/~lily/files/Documents/NTDesignWorkbook/vm.pdf https://computernewb.com/~lily/files/Documents/NTDesignWorkb...
      - pstuart 3mo ago
        WinNT 3.5 was a solid offering.
    - epcoa 3mo ago
      This is not true. NT never had fork, was always based on the assumption of an MMU and Dave Cutler was a well known fork hater in the 80s long before this paper came out and made it cool to be so. By the time Windows 95 was out, the baseline was 386 with an MMU. CreateThread was initially designed for NT in 1993 though (which didn’t support pre-386 CPUs).
omoikane 3mo ago
Discussion at the time: https://news.ycombinator.com/item?id=19621799 https://news.ycombinator.com/item?id=19621799 - A fork() in the road (2019-04-10, 178 comments)
- jwilk 3mo ago
  Discussed also in 2021: https://news.ycombinator.com/item?id=29709802 https://news.ycombinator.com/item?id=29709802 (16 comments)
aseipp 3mo ago
This paper is great and I also really like one of its references [29] as it goes into some more subtle parts of scalable interfaces, including fork. It's a gem IMO: The Scalable Commutativity Rule: Designing Scalable Software for Multicore Processors https://people.csail.mit.edu/nickolai/papers/clements-sc.pdf https://people.csail.mit.edu/nickolai/papers/clements-sc.pdf
Animats 3mo ago
> The received wisdom suggests that Unix’s unusual combination of fork() and exec() for process creation was an inspired design. No, it was done that way so that you could launch a program that was too big to fit in memory with the parent program. The original implementation worked by swapping out the forking program to disk on a fork() call. Then, at the moment the program was swapped out but control had not returned, the process table entry was duplicated and adjusted so that there were now two processes, one in memory and one swapped out. The one in memory then got control, and could do an exec() call. This allowed large programs to run on small PDP-11 machines. It was needed back in the era of really expensive memory. That's why. QNX had an interesting approach. Program loading isn't in the OS at all. There's "fork", but program loading is in a library. It links to a .so file which reads the executable header, allocates memory, loads the program, gets it ready to run, and starts it. The program loader runs in user space and is unprivileged. This is probably the right way to do it.
- lukan 3mo ago
  It is almost as if you agree with the authors .. "In this paper, we argue that fork was a clever hack for machines and programs of the 1970s that has long outlived its usefulness and is now a liability" (But thanks for the good explanation)
- dcrazy 3mo ago
  Don’t pretty much all OSes implement process startup in userspace? On macOS, the kernel creates a process with an image of dyld and points it at dyld_start, which actually takes care of parsing the Mach-O header. I assumed ld.so does the same job on Linux.
  - purkka 3mo ago
    Nope, the kernel can load static ELF binaries. ld.so is only needed for dynamically linked binaries, and in fact many Go applications (for example, as they're statically linked) ship as containers with nothing but the single binary.
    - 3mo ago
up2isomorphism 3mo ago
Not sure if fork is outdated or not, but people calling it a “hack” obviously have pretty bad engineering taste.
cryptonector 3mo ago
Ah, my one time on the HN front-page: Fork() is evil; vfork() is goodness; afork() would be better; clone() is stupid (https://news.ycombinator.com/item?id=30502392 https://news.ycombinator.com/item?id=30502392).

by rom1v 3mo ago

pizlonator 3mo ago

Fork is marvelous for the zygote pattern Hard to come up with an optimization that is equally efficient and elegant

toast0 3mo ago
The zygote pattern[1] is a great optimization to deal with the cost of forking, but IMHO, being able to inexpensively spawn a carefully tailored process regardless of the size and scope of the current process would be better. I would guess it would be a small difference in measurable performance between zygote and a direct clean spawn, but it's one less trick an application needs to do, and it would be very helpful for libraries that spawn things. Spawning inside a library isn't always a great thing to do, but some things would really benefit from process level isolation. [1] In case one isn't aware, the zygote pattern involves forking a 'zygote' process during application startup, and having that process do any forks that need to happen during application runtime. This reduces the cost of forking in large applications, because the zygote will have few fds open and use little memory. This lets your large application spawn new processes without delaying the application or the startup of the new processes. Some applications will spawn many zygotes to allow parallelism for spawning at runtime.
- pizlonator 3mo ago
  You're referring to something else, and maybe I'm using the term "zygote" incorrectly. In all uses of zygotes that I have seen, here's what's really happening: - `fork` is being used to reduce the cost of starting a process that has a high start-up cost. So, you start one process, run it through the expensive initialization, and then fork it from there to start new processes. - To make this even faster, you have a pool of pre-forked processes sit around. - Having pre-forked processes sitting around ready to be used is not expensive because of the CoW property and the fact that a process that forks and then immediately pauses will not have triggered any significant CoW yet. So, the zygote optimization you speak of is in practice only meaningful on top of systems that are using an optimization uniquely enabled by `fork` (avoiding process initialization costs by cloning a process), and that zygote optimization is further optimized by another property of `fork` (memory sharing of forked processes that haven't done anything else yet).
  - toast0 3mo ago
    Oh I see. I guess your zygotes have developed more than mine. I think Google may have coined or at least popularized the term zygote for this in Chrome and Android, Chrome documentation [1] says: > A zygote process is one that listens for spawn requests from a main process and forks itself in response. Generally they are used because forking a process after some expensive setup has been performed can save time and share extra memory pages. I think reading the first sentance and stopping covers my zygote, but adding the second sentance covers yours. So I think we're both right! I think both paths are useful. If your children need time to startup and become ready, spawn one that does start up work, and then it (pre)forks at the ready state to have processes ready to handle requests (your zygote). This does require a traditional fork() to avoid duplication of work. But if forking is expensive at runtime because you have a million FDs open and a whole lot of memory allocations, spawn spawners before you start doing work (my zygote). This could be unnecessary with a inexpensive way to spawn a new process from an process that has lots of resources in use. Of course, you can also use my zygotes to spawn your zygotes. Zygoteception. [1] https://chromium.googlesource.com/chromium/src/+/HEAD/docs/linux/zygote.md https://chromium.googlesource.com/chromium/src/+/HEAD/docs/l...
vlovich123 3mo ago
The paper explicitly covers it that various memory COW/snapshot mechanisms are probably faster and safer than the zygote pattern. As it stands getting the zygote pattern correct and safe is something you have to plan for upfront. You can’t retrofit it which is why the paper mentions it has poor composability. Also the advantages of the zygote pattern can be overstated since the memory sharing benefit is minimal since it has to happen so early and modern OSes already transparently CoW duplicate pages in the background.
- loeg 3mo ago
  In what sense can you not retrofit the zygote pattern?
  - vlovich123 3mo ago
    I recommend at least skimming the paper as it covers this. But essentially you can’t just inject a call at a random point in code to start being a zygote. It’s something you have to plan up front as to the exact point you’re going to fork and that you’re going to do it at the start of program before any threads have started or any files are open and before any locks have been acquired. It’s basically all the challenges of invoking fork at arbitrary points in time. The reason to do a zygote in the first place could be solved with alternative special APIs that are safer and harder to misuse. But we have fork so there’s not as big of a demand despite the warts.
    - loeg 3mo ago
      Sure, but you can always retrofit a program to fork early on... this is a relatively trivial change. No?
deleted 3mo ago
[deleted]
p_l 3mo ago
And so easy to make into bottleneck. Yes, zygote pattern makes it easy to make fork() into bottleneck - it requires a lot more discipline and low level tricks (linker scripts, compiler-specific extensions, custom sections, low level dependencies on pagesize that get "fun" on ARM servers). If you don't, you might wake up with fork() causing latency issues.
cyberax 3mo ago
Unless you want to create a thread in your zygote. Then it breaks down. Raw fork() is terrible. Instead we need a proper primitive to stop and make a snapshot of a process.
- pizlonator 3mo ago
  You can create threads in the zygote. It doesn't "break down", but sure, there's a bit more work. My trick for that is that the set of threads that I create pre fork have to be suspendable and resumable, preferably lazily (they resume when they are actually needed). So, the zygotes are sitting with those threads suspended. When they become active, they can do work immediately. They might lazily resume those threads as needed. There are other idioms for this too. > Raw fork() is terrible. Instead we need a proper primitive to stop and make a snapshot of a process. Folks have been saying that it's terrible for as long as I can remember. But it's still there, because it's better than the alternatives
  - cyberax 3mo ago
    > My trick for that is that the set of threads that I create pre fork have to be suspendable and resumable Well, yes. You need to wait for all the threads to park themselves at safepoints. This can work if you control the whole runtime, and you don't use something that creates threads behind your back. This is actually why I've always been interested in a better fork(), it has a lot of parallels with stop-the-world needed for GCs. > Folks have been saying that it's terrible for as long as I can remember. But it's still there, because it's better than the alternatives I don't think we have alternatives? Except maybe ptrace()?

anarazel 3mo ago

It is somewhat interesting that the most widely used "big" OS that doesn't use fork, i.e. Windows, has dog slow process creation... I agree that there should be non-fork primitives, I'm just not that sure that performance is the best argument.

pjmlp 3mo ago
Because that OS best practices is to use threads. Traditionally Windows applications that create processes all the time come from UNIX heritage. Contrary to UNIX, Windows NT was designed with threads first mentality, from the get go. While on UNIX they were added after fact, and to this day there are gotchas mixing posix threads with signals, fork and exec.
- zozbot234 3mo ago
  Windows was designed with threads-first mentality because on pre-386 machines you don't have viable process memory protection, so your tasks share memory by necessity. This is not a great argument.
  - JdeBP 3mo ago
    Windows NT was never designed with pre-386 machines in mind. That was the territory of the old DOS+Windows. Windows NT from the get-go was for machines with page-based virtual memory. * https://computernewb.com/~lily/files/Documents/NTDesignWorkbook/vm.pdf https://computernewb.com/~lily/files/Documents/NTDesignWorkb...
    - pstuart 3mo ago
      WinNT 3.5 was a solid offering.
  - epcoa 3mo ago
    This is not true. NT never had fork, was always based on the assumption of an MMU and Dave Cutler was a well known fork hater in the 80s long before this paper came out and made it cool to be so. By the time Windows 95 was out, the baseline was 386 with an MMU. CreateThread was initially designed for NT in 1993 though (which didn’t support pre-386 CPUs).

omoikane 3mo ago

Discussion at the time: https://news.ycombinator.com/item?id=19621799 https://news.ycombinator.com/item?id=19621799 - A fork() in the road (2019-04-10, 178 comments)

jwilk 3mo ago
Discussed also in 2021: https://news.ycombinator.com/item?id=29709802 https://news.ycombinator.com/item?id=29709802 (16 comments)

aseipp 3mo ago

This paper is great and I also really like one of its references [29] as it goes into some more subtle parts of scalable interfaces, including fork. It's a gem IMO: The Scalable Commutativity Rule: Designing Scalable Software for Multicore Processors https://people.csail.mit.edu/nickolai/papers/clements-sc.pdf https://people.csail.mit.edu/nickolai/papers/clements-sc.pdf

Animats 3mo ago

> The received wisdom suggests that Unix’s unusual combination of fork() and exec() for process creation was an inspired design. No, it was done that way so that you could launch a program that was too big to fit in memory with the parent program. The original implementation worked by swapping out the forking program to disk on a fork() call. Then, at the moment the program was swapped out but control had not returned, the process table entry was duplicated and adjusted so that there were now two processes, one in memory and one swapped out. The one in memory then got control, and could do an exec() call. This allowed large programs to run on small PDP-11 machines. It was needed back in the era of really expensive memory. That's why. QNX had an interesting approach. Program loading isn't in the OS at all. There's "fork", but program loading is in a library. It links to a .so file which reads the executable header, allocates memory, loads the program, gets it ready to run, and starts it. The program loader runs in user space and is unprivileged. This is probably the right way to do it.

lukan 3mo ago
It is almost as if you agree with the authors .. "In this paper, we argue that fork was a clever hack for machines and programs of the 1970s that has long outlived its usefulness and is now a liability" (But thanks for the good explanation)
dcrazy 3mo ago
Don’t pretty much all OSes implement process startup in userspace? On macOS, the kernel creates a process with an image of dyld and points it at dyld_start, which actually takes care of parsing the Mach-O header. I assumed ld.so does the same job on Linux.
- purkka 3mo ago
  Nope, the kernel can load static ELF binaries. ld.so is only needed for dynamically linked binaries, and in fact many Go applications (for example, as they're statically linked) ship as containers with nothing but the single binary.
  - 3mo ago

up2isomorphism 3mo ago

Not sure if fork is outdated or not, but people calling it a “hack” obviously have pretty bad engineering taste.

cryptonector 3mo ago

Ah, my one time on the HN front-page: Fork() is evil; vfork() is goodness; afork() would be better; clone() is stupid (https://news.ycombinator.com/item?id=30502392 https://news.ycombinator.com/item?id=30502392).