7 ms·

Without having looked into it too deeply I feel that they are somewhat “cheating” by using a superserver to launch a new process for each connection, thus letti

Without having looked into it too deeply I feel that they are somewhat “cheating” by using a superserver to launch a new process for each connection, thus letting the OS handle the dynamic allocation needed for each connection. Still pretty impressive project. Would be fun to take a deeper look at it at some point.

hkt 5y ago
It makes sense though - no memory management simplifies the codebase. Letting the host deal with it instead means you get the niceties of process level isolation and less complexity. More eyes are on the OS level code than would be on this project. It seems very clever to me.
zamadatix 5y ago
The OS is already handling dynamic memory allocation on each connection. This provides a nice split where you don't need to do it in 2 places.
throw0101a 5y ago
> […] thus letting the OS handle the dynamic allocation needed for each connection. This is what PHK did when designing Varnish (IIRC): instead of dealing with lots of files on its own (like Squid), just create some files and do a malloc() on them and let the OS do the work: * https://varnish-cache.org/docs/trunk/phk/notes.html https://varnish-cache.org/docs/trunk/phk/notes.html One discussion (many others): * https://news.ycombinator.com/item?id=4874304 https://news.ycombinator.com/item?id=4874304
- mrich 5y ago
  It's not malloc() but mmap() if I understand correctly. It's a beautifully powerful function :) https://man7.org/linux/man-pages/man2/mmap.2.html https://man7.org/linux/man-pages/man2/mmap.2.html
  - richardwhiuk 5y ago
    It kind of undercuts the argument, because the dynamic allocator basically just uses mmap as well. (Some allocators, like glibc may also use brk, but it's possible to write a fully functional allocator with only mmap).
    - mrich 5y ago
      I think the original point still stands. When the application tries to handle memory pressure itself by writing data structures to disk, it will hit the case where the kernel has already paged that memory out, has to reload it, only to write it to disk and free the memory afterwards.

hkt 5y ago

It makes sense though - no memory management simplifies the codebase. Letting the host deal with it instead means you get the niceties of process level isolation and less complexity. More eyes are on the OS level code than would be on this project. It seems very clever to me.

zamadatix 5y ago

The OS is already handling dynamic memory allocation on each connection. This provides a nice split where you don't need to do it in 2 places.

throw0101a 5y ago

> […] thus letting the OS handle the dynamic allocation needed for each connection. This is what PHK did when designing Varnish (IIRC): instead of dealing with lots of files on its own (like Squid), just create some files and do a malloc() on them and let the OS do the work: * https://varnish-cache.org/docs/trunk/phk/notes.html https://varnish-cache.org/docs/trunk/phk/notes.html One discussion (many others): * https://news.ycombinator.com/item?id=4874304 https://news.ycombinator.com/item?id=4874304

mrich 5y ago
It's not malloc() but mmap() if I understand correctly. It's a beautifully powerful function :) https://man7.org/linux/man-pages/man2/mmap.2.html https://man7.org/linux/man-pages/man2/mmap.2.html
- richardwhiuk 5y ago
  It kind of undercuts the argument, because the dynamic allocator basically just uses mmap as well. (Some allocators, like glibc may also use brk, but it's possible to write a fully functional allocator with only mmap).
  - mrich 5y ago
    I think the original point still stands. When the application tries to handle memory pressure itself by writing data structures to disk, it will hit the case where the kernel has already paged that memory out, has to reload it, only to write it to disk and free the memory afterwards.