Rant - CPUs have a cache hierarchy for a reason. We can use smart caches and speculative execution to fetch only the pieces of memory that actually are necessa

Rant - CPUs have a cache hierarchy for a reason. We can use smart caches and speculative execution to fetch only the pieces of memory that actually are necessary for computation. Large parts of the computing infra support this - like RAM, where batching requests is not really helpful, as you can hit full speed with 64 byte accesses. This means that the tradeoff between bandwidth and latency falls somewhere in the middle, and its worth being smart about it. MMAP and virtual memory is supposed to be the last level of this, managed by software. But it was designed in the time of spinning disks, where it took 100s of ms to serve a page fault, so parking a faulting process wasn't critical as it was almost certain you couldnt serve a request before the timeslice was up. RAM was comparatively huge to cache, so it made sense to bring in large chunks at once - in fact this was so true, that if your app started paging, essentially it became unusable. SSDs changed this math - you can almost certainly serve a request in tens of microseconds even with going through PCIe - hooking up flash directly to the memory hierarchy might make this even cheaper. So the reason why io_uring exists is because the syscall overhead is now comparable. But this involves a new API and devs having to manage this manually. I think this is wrong. I'm not sure if this is fixable with the current batch of CPUs, or needs HW support, but imo there needs to be some sort of page fault handling that's much more granular than an entire page (lets say 1-2KB, on the order of SSD page sizes)