6 ms·
It is somewhat interesting that the most widely used "big" OS that doesn't use fork, i.e. Windows, has dog slow process creation... I agree that there should b
by anarazel 3mo ago
It is somewhat interesting that the most widely used "big" OS that doesn't use fork, i.e. Windows, has dog slow process creation...
I agree that there should be non-fork primitives, I'm just not that sure that performance is the best argument.
- pjmlp 3mo agoBecause that OS best practices is to use threads. Traditionally Windows applications that create processes all the time come from UNIX heritage. Contrary to UNIX, Windows NT was designed with threads first mentality, from the get go. While on UNIX they were added after fact, and to this day there are gotchas mixing posix threads with signals, fork and exec.
- zozbot234 3mo agoWindows was designed with threads-first mentality because on pre-386 machines you don't have viable process memory protection, so your tasks share memory by necessity. This is not a great argument.
- JdeBP 3mo agoWindows NT was never designed with pre-386 machines in mind. That was the territory of the old DOS+Windows. Windows NT from the get-go was for machines with page-based virtual memory. * https://computernewb.com/~lily/files/Documents/NTDesignWorkbook/vm.pdf https://computernewb.com/~lily/files/Documents/NTDesignWorkb...
- pstuart 3mo agoWinNT 3.5 was a solid offering.
- epcoa 3mo agoThis is not true. NT never had fork, was always based on the assumption of an MMU and Dave Cutler was a well known fork hater in the 80s long before this paper came out and made it cool to be so. By the time Windows 95 was out, the baseline was 386 with an MMU. CreateThread was initially designed for NT in 1993 though (which didn’t support pre-386 CPUs).
- keitmo 3mo agoNT performed unnatural acts to implement fork semantics for the POSIX subsystem.
- JdeBP 3mo agoAs mentioned elsewhere on this page, Windows NT had fork from the start. Vide NtCreateProcess and what happens if an image file is not explicitly supplied. * https://computernewb.com/~lily/files/Documents/NTDesignWorkbook/proc.pdf https://computernewb.com/~lily/files/Documents/NTDesignWorkb...
- dcrazy 3mo agoNtCreateProcess doesn’t accept an image file parameter.
- JdeBP 3mo agoYou haven't read the doco. I did point to some. The image file is supplied (or not) via the section object. Think it through. Windows NT supported fork from the start in its POSIX subsystem, that subsystem was layered on top of the Native API, and this is the Native API mechanism that the POSIX subsystem employed. Although it took until Gary Nebbett for someone to publicly show how, even though people knew informally back in 1993.
- epcoa 3mo agoNtCreateProcess was not a public Windows API. NT was flexible, that’s not what was being discussed, which should have been clear from the context.
- deleted 3mo ago[deleted]
- pjmlp 3mo agoWindows NT! Misread on purpose to make a point?
- dcrazy 3mo agoNT was designed to be platform-agnostic, and its original target was the DEC Alpha. Its process model owes nothing to pre-386 CPUs. The WinAPI CreateProcess function is a layer atop NtCreateProcess, so that is where the pre-386 heritage lives. But even the WinAPI process model changed significantly with 32-bit Windows.
- peterfirefly 3mo agoNo. https://en.wikipedia.org/wiki/Windows_NT#Development https://en.wikipedia.org/wiki/Windows_NT#Development Windows NT was developed on various different CPUs before the Alpha was a thing. When it was released in 1993, it was released for three CPUs: IA-32, MIPS, and Alpha.
- dcrazy 3mo agoSorry, I had conflated Windows NT development with development of 64-bit Windows as told by Raymond Chen: https://learn.microsoft.com/en-us/previous-versions/technet-magazine/cc718978(v=msdn.10) https://learn.microsoft.com/en-us/previous-versions/technet-... Raymond also says elsewhere that most WinNT engineers did development on i386, but doesn’t explicitly say what time period he is describing: https://devblogs.microsoft.com/oldnewthing/20250513-00/?p=111176 https://devblogs.microsoft.com/oldnewthing/20250513-00/?p=11...
- PaulDavisThe1st 3mo agoA more accurate way to describe this is that Windows' (NT onward) core execution context model is a bunch of threads that by default share memory, whereas Unixen have a core task context model of a bunch of threads that by default do not share memory. Both systems are implemented using threads as the execution context, but in Unix, the history means that that you fork+exec most of the time, resulting in a two tasks that do not share memory any more. By contrast, on Windows (NT onward) the common case when creating a new execution context is to create a thread that shares memory with others in its process. Both systems allow the easy use of the other's core abstraction. On Unix, you can either code like its 1986 and use fork without exec, or use clone(3) or any of its higher level abstractions like pthreads. You're right that POSIX semantics get tangled when using threads.
- pjmlp 3mo agoWell, Windows before NT isn't the same design as Windows 16 bit, it only shares the name for all practical purposes, and has more influence from OS/2 than Windows 16 bit. Which is why I took the effort to explicitly refer to Windows NT on my comment, already expecting some traditional answers from UNIX folks. Also due to historical reasons POSIX threads are the outcome of every UNIX going their own way implementing threads, finally coming to an agreement years later, with all the plus and minus of relying in POSIX for portable code.
- snozolli 3mo agowhereas Unixen have a core task context model of a bunch of threads that by default do not share memory. How are those not simply child processes? I don't understand your use of the word 'threads' here. Does the Unix world not distinguish between threads and processes? In Win32, threads exist within processes, and you can create new threads or child processes.
- pjmlp 3mo agoActually on Windows a process is a thread with additional information. The unit of execution is the thread. On the UNIX world it depends on which UNIX you are talking about. Linux has a similar model to Windows NT nowadays, hence clone() as key primitive. Other UNIXes have different approaches.
- knome 3mo agothe only difference between a thread and a process on linux is how many structures they share. the function is identical.
- pjmlp 3mo agoAgreed, however not all UNIXes are like Linux.
- sunshowers 3mo agoThe problem is that threads are not fault boundaries but processes are. So they're not interchangeable when you care about resilience and misbehaving code.
- pjmlp 3mo agoTrue, but on Windows the approach is then to use COM servers, which have a faster IPC model, and can even serve multiple clients, depending on how the appartement space is configured.
- dcrazy 3mo agoIf you want the isolation features of a separate process, you can’t substitute it with a single multithreaded COM server process. .NET tried this with app domains, which are now deprecated.
- pjmlp 3mo agoApp Domains were in process, which isn't was I am talking about with outproc COM. Also App Domains are partially back in .NET Core, isolation features aren't there, but code unloading is, via AssemblyLoadContext.
- dcrazy 3mo agoMy point is that “just write a COM server” is not an answer to the problem of “I want each work item to be segregated from each other.”
- mort96 3mo ago"Faster IPC model" than what? Faster than writing to and reading from a pipe? Faster than POSIX shared memory?
- pjmlp 3mo agoThan UNIX fork/exec model, or calling into Create Process all the time. Windows has a more rich set of IPC stuff than POSIX, especially since it has a microkernel like design. If you are going to say it is everything on the same memory space anyway, it isn't. Optional on Windows 10, and enforced on Windows 11, Hyper-V is always running, and several components including kernel and driver modules are sandboxed into their little worlds. Several additional sandboxing changes were announced at BUILD.
- nine_k 3mo agoPOSIX threads having problems with signals is, imho, mostly the problem with signals in general. They are pretty poorly designed: https://lwn.net/Articles/414618/ https://lwn.net/Articles/414618/
- deleted 3mo ago[deleted]
- nvme0n1p1 3mo agoThat's not the reason for the performance difference. Windows does have a fork primitive (ZwCreateProcess) and it's still slower than Linux's equivalent.
- dcrazy 3mo agoAgain, NtCreateProcess does not implement fork(). The fundamental characteristic of fork is that the child is an exact replica of the parent, down to the instruction pointer. Windows does not have a way to create a process object with such a configuration. Also, using the Zw prefix doesn’t make you look more knowledgeable, it makes you look like you’re trying way too hard to borrow credibility.
- nvme0n1p1 3mo agoOkay but people don't claim that copying the instruction pointer (a single machine register) is the reason for any speed difference. They claim it's due to the memory sharing. And that's easily disproven since you can share pages, just like on Linux, simply by passing null for the section handle, yet there's still a performance difference. Why does it matter which prefix I used? They both point to the same routine so my point applies either way.
- netbsdusers 3mo agoIt's a completely uncontroversial fact that NT does implement fork(). Turn to page 183 of Helen Custer's "Inside Windows NT" and you will read about it.
- aseipp 3mo agoI suspect it's a long tail sort of thing; it mostly doesn't matter except when it really matters. It's interesting that the stated motivation for the patch is in the context of agentic tools spawning subcommands. There's some related prior art in this area where the payoffs could be much greater, like fuzzing: https://gts3.org/assets/papers/2017/xu:os-fuzz.pdf https://gts3.org/assets/papers/2017/xu:os-fuzz.pdf is an example. It would be very interesting to see this patch applied to e.g. AFL++
- mort96 3mo agoThe problem with fork isn't really that it's slow. The problem is that if you want it to be not-slow, it locks you into a bunch of OS design decisions: you more or less need a memory subsystem where all writable pages are refcounted and copy-on-write when the refcount is bigger than 1, and you need overcommit. Now these decisions aren't objectively bad, but they have significant trade-offs and it's probably not a good idea that they're forced simply because we use fork()+exec() for process creation.
- thayne 3mo agoWith large enough processes, like say a server JVM process that uses 10s of GBs of RAM, even just copying the page tables for CoW can be slow. And unless you have aggressive overcommit settings you can get an OOM on fork, even if you're just going to exec something small. vfork helps a little, but it has a lot of restrictions on what you can do before the exec, and on unix that's basically the only place you can do things like close files, change signal masks, drop privileges or set up seccomp, etc.
- cryptonector 3mo agovfork() helps a LOT. The restrictions on what you can do on the child-side of vfork() are pretty much the same ones as for fork() + you must not do anything to damage the stack frame of the vfork() caller (i.e., you can't return).
- thayne 3mo ago> the behavior is undefined if the process created by vfork() either modifies any data other than a variable of type pid_t used to store the return value from vfork(), or returns from the function in which vfork() was called, or calls any other function before successfully calling _exit(2) or one of the exec(3) family of functions That's a lot more restrictive. You can't use local variables, or call any functions other than _exit or execve. On linux specifically, I _think_ those restrictions are more relaxed and you can call async-signal-safe functions, however I'm not entirely clear on how relaxed that is, and as far as I understand that isn't portable.