8 ms·

For a while since RAM sizes have gotten so big I would set up linux servers without swap at all. I figured what is a 2GB swap file going to do if the system has

For a while since RAM sizes have gotten so big I would set up linux servers without swap at all. I figured what is a 2GB swap file going to do if the system has already exhausted 128GB of real RAM? I never saw any issues doing this, but I learned that linux ideally wants some swap space, which it will use for some kind of housekeeping stuff.

malwrar 5d ago
Any tips on measuring the impact of not having swap? I never use it on my machines for similar reasons.
- ktm5j 5d ago
  Worst thing that's ever happened to me is having processes randomly killed by the OOM killer. Could be unpleasant if you're writing an email or document that suddenly disappears because your app got killed.
  - mitxela 4d ago
    The OOM killer usually kills the process that's using the most memory, which is the reason the system is OOM. Sometimes this can be adjusted. When I'm OOM it's almost always because some runaway loop used 50GB and it makes no sense to kill any other process. If it's killing your email editor then your email editor is bloated or you just need more RAM - or swap.
- anyfoo 5d ago
  It highly depends on your specific workload, but freeing up RAM by paging out rarely or downright unused pages to swap can definitely give a performance advantage. In consequence, I always give swap. Even on the off-chance it ends up unused, disk space is orders of magnitude cheaper than RAM, so I don't mind sacrificing it for potential gains.
linsomniac 5d ago
I have tried to forego swap because I had hoped that it would avoid swap thrashing, where performance goes into the toilet; I'd rather a process was killed but I could still access the system, than be unable to login to it. But I've found that even with no/little swap, the system can get into high memory pressure situations (even easier because it can't swap out unused pages of libraries and the like), and still go into a sort of thrashing state even though it isn't swapping. My main workstation is a Dell XPS15 with 32GB of RAM and I had 8GB of swap on it. About every month the system load would skyrocket even though it had plenty of mem+cache+swap available. If I killed off my browser and slack, load would return to normal, and I could use the system for a few more days or a week before it happened again. I struggled with this for a year or more, before (for reason's I don't remember), I added another 8GB of swap. Within the next day 80% of the extra swap was used right up. I decided to swapoff that new file and create a 64GB swapfile and just see what happened. It got to around 20-30GB IIRC before it leveled off. And, more importantly, over the next 100+ days I never had that thrashing happen again. That was with Ubuntu 22.04. I just recently (May-ish) switched over to a NixOS setup with 64GB of swapfile, and it's been very stable.
- SoftTalker 5d ago
  The old rule of thumb used to be 2x actual RAM for your swap file/partition size. That's probably not right with today's RAM sizes but I'm not sure what the current rule is. I just go with the Ubuntu installer default but I don't know if that's just a fixed number or computed in some way from amount of RAM available.
  - Sohcahtoa82 5d ago
    > The old rule of thumb used to be 2x actual RAM for your swap file/partition size. Which was always a dumbass rule that made no logical sense. Like, I remember a time when I had 4 GB of RAM and had my swap set to 8 GB. Then I upgraded to 16 GB of RAM. Theoretically, I shouldn't even need a swap file anymore, since my old setup would max at 12 GB and yet I never had OOM problems, but that "rule of thumb" would dictate that I should increase my swap to 32 GB.
rlpb 5d ago
It does give you some more RAM for free, as Linux can park allocated but rarely accessed memory and use the freed space for other purposes.
lokar 5d ago
It does not need it at all. I’ve run a million+ hosts with no swap. You need to be really disciplined about memory allocations (us cgroups) and ok with what happens if you run out (something in that cgroup dies). But there is no “housekeeping “ that requires it.
- anyfoo 5d ago
  This comes up every time swap is mentioned. Even if you have plenty of ram, having swap can help with performance. Why? Because the OS can page rarely used or seemingly downright unused dirty pages to swap, freeing up precious RAM for caches. I'd hazard a guess that especially if you run a ton of containers this can help, as there's probably a lot of memory getting dirtied on initialization of containers and their processes, which ends up never read afterwards. I always set up swap, because sacrificing some disk space in potential exchange of free RAM, which is orders of magnitudes more expensive, is a nice prospect.
  - lokar 5d ago
    We looked at this a lot (it was a lot of money at our scale). A cgroup itself is just a few small data structures. And a reasonable process inside will only ever have a small amount of dirty memory it won’t ever use. Even then you can’t be sure of it packing neatly into whole pages. So, the typical saving is tiny, and the cost in unpredictable performance is quite bad. If you really do have an issue with “lost” dirty memory, your time is best spent addressing that directly.
    - anyfoo 5d ago
      Fair enough, you looked and did the math. On my laptop (totally different use case), there is currently plenty of memory in the compressor.
      - lokar 5d ago
        Yeah, personal (desktop/laptop) vs production are totally different. The production workload should be well understood and stable.
anyfoo 5d ago
You never saw any issues, but your systems might have been faster with swap. Even if you're nowhere any threshing threshold, RAM used for caches is better than RAM used for unused pages, which could have been paged out to swapspace. Note that RAM is much, much more expensive than disk space. If I can sacrifice some of the latter to potentially get some of the former, it's usually a nice trade for me.
lotharcable 5d ago
It just meant that you spent too much on servers. Either that or they could of been doing more. Not a big deal. Being efficient isn't really all it is cracked up to be. Sometimes having a big server that sits mostly idle is a good trade off for what you want to accomplish.

malwrar 5d ago

Any tips on measuring the impact of not having swap? I never use it on my machines for similar reasons.

ktm5j 5d ago
Worst thing that's ever happened to me is having processes randomly killed by the OOM killer. Could be unpleasant if you're writing an email or document that suddenly disappears because your app got killed.
- mitxela 4d ago
  The OOM killer usually kills the process that's using the most memory, which is the reason the system is OOM. Sometimes this can be adjusted. When I'm OOM it's almost always because some runaway loop used 50GB and it makes no sense to kill any other process. If it's killing your email editor then your email editor is bloated or you just need more RAM - or swap.
anyfoo 5d ago
It highly depends on your specific workload, but freeing up RAM by paging out rarely or downright unused pages to swap can definitely give a performance advantage. In consequence, I always give swap. Even on the off-chance it ends up unused, disk space is orders of magnitude cheaper than RAM, so I don't mind sacrificing it for potential gains.

linsomniac 5d ago

I have tried to forego swap because I had hoped that it would avoid swap thrashing, where performance goes into the toilet; I'd rather a process was killed but I could still access the system, than be unable to login to it. But I've found that even with no/little swap, the system can get into high memory pressure situations (even easier because it can't swap out unused pages of libraries and the like), and still go into a sort of thrashing state even though it isn't swapping. My main workstation is a Dell XPS15 with 32GB of RAM and I had 8GB of swap on it. About every month the system load would skyrocket even though it had plenty of mem+cache+swap available. If I killed off my browser and slack, load would return to normal, and I could use the system for a few more days or a week before it happened again. I struggled with this for a year or more, before (for reason's I don't remember), I added another 8GB of swap. Within the next day 80% of the extra swap was used right up. I decided to swapoff that new file and create a 64GB swapfile and just see what happened. It got to around 20-30GB IIRC before it leveled off. And, more importantly, over the next 100+ days I never had that thrashing happen again. That was with Ubuntu 22.04. I just recently (May-ish) switched over to a NixOS setup with 64GB of swapfile, and it's been very stable.

SoftTalker 5d ago
The old rule of thumb used to be 2x actual RAM for your swap file/partition size. That's probably not right with today's RAM sizes but I'm not sure what the current rule is. I just go with the Ubuntu installer default but I don't know if that's just a fixed number or computed in some way from amount of RAM available.
- Sohcahtoa82 5d ago
  > The old rule of thumb used to be 2x actual RAM for your swap file/partition size. Which was always a dumbass rule that made no logical sense. Like, I remember a time when I had 4 GB of RAM and had my swap set to 8 GB. Then I upgraded to 16 GB of RAM. Theoretically, I shouldn't even need a swap file anymore, since my old setup would max at 12 GB and yet I never had OOM problems, but that "rule of thumb" would dictate that I should increase my swap to 32 GB.

rlpb 5d ago

It does give you some more RAM for free, as Linux can park allocated but rarely accessed memory and use the freed space for other purposes.

lokar 5d ago

It does not need it at all. I’ve run a million+ hosts with no swap. You need to be really disciplined about memory allocations (us cgroups) and ok with what happens if you run out (something in that cgroup dies). But there is no “housekeeping “ that requires it.

anyfoo 5d ago
This comes up every time swap is mentioned. Even if you have plenty of ram, having swap can help with performance. Why? Because the OS can page rarely used or seemingly downright unused dirty pages to swap, freeing up precious RAM for caches. I'd hazard a guess that especially if you run a ton of containers this can help, as there's probably a lot of memory getting dirtied on initialization of containers and their processes, which ends up never read afterwards. I always set up swap, because sacrificing some disk space in potential exchange of free RAM, which is orders of magnitudes more expensive, is a nice prospect.
- lokar 5d ago
  We looked at this a lot (it was a lot of money at our scale). A cgroup itself is just a few small data structures. And a reasonable process inside will only ever have a small amount of dirty memory it won’t ever use. Even then you can’t be sure of it packing neatly into whole pages. So, the typical saving is tiny, and the cost in unpredictable performance is quite bad. If you really do have an issue with “lost” dirty memory, your time is best spent addressing that directly.
  - anyfoo 5d ago
    Fair enough, you looked and did the math. On my laptop (totally different use case), there is currently plenty of memory in the compressor.
    - lokar 5d ago
      Yeah, personal (desktop/laptop) vs production are totally different. The production workload should be well understood and stable.

anyfoo 5d ago

You never saw any issues, but your systems might have been faster with swap. Even if you're nowhere any threshing threshold, RAM used for caches is better than RAM used for unused pages, which could have been paged out to swapspace. Note that RAM is much, much more expensive than disk space. If I can sacrifice some of the latter to potentially get some of the former, it's usually a nice trade for me.

lotharcable 5d ago

It just meant that you spent too much on servers. Either that or they could of been doing more. Not a big deal. Being efficient isn't really all it is cracked up to be. Sometimes having a big server that sits mostly idle is a good trade off for what you want to accomplish.