5 ms·
> One of the nice things about using mmap'd data structures is that you don't have to slurp the entire thing into memory to work on it. Right, but one of the n
by wfunction 9y ago
> One of the nice things about using mmap'd data structures is that you don't have to slurp the entire thing into memory to work on it.
Right, but one of the not-so-nice-things is that you can't do the I/O asynchronously, so you can end up with poor performance depending on the access pattern.
(I guess you can, with another thread running in the background touching pages, but it's more of a pain and you're not really guaranteed the data will stay in memory until you use them.) [Edit: Actually I guess if you touch by writing to those pages rather than just reading from them then they'll have to stay in memory... though do note that I'm assuming no swap here.]
- luispedrocoelho 9y agoAt some cost to portability, IIRC, on Linux you can ask the operating system to keep pages in memory. Other systems will probably have similar functionality.
- otterley 9y agoThey can remain paged in, but unlike on FreeBSD, you can't control when dirty pages are flushed to the backing filesystem on Linux. Specifically, the MAP_NOSYNC option doesn't exist on Linux (see https://www.freebsd.org/cgi/man.cgi?sektion=2&query=mmap https://www.freebsd.org/cgi/man.cgi?sektion=2&query=mmap for a description).
- wfunction 9y agoWindows has NtLockVirtualMemory, but (a) it requires special permissions (meaning random apps can't do it without admin privileges), and (b) something about it feels like it's the wrong way to do it, but I can't pin down what it is exactly.