8 ms·

>"run this daemon that tries to beat the kernel OOM to the punch" Considering that the kernel OOM killer tends to be way too late in doing its thing I don't se

by esgwpl 5y ago

>"run this daemon that tries to beat the kernel OOM to the punch" Considering that the kernel OOM killer tends to be way too late in doing its thing I don't see how this is inelegant, maybe there's a reason you can't just have the kernel kill processes earlier in the face of memory pressure.

ladyanita22 5y ago
The kernel OOM is just plain broken. I can't understand how can it be possible that, on Windows, whenever I run out of RAM the OS kills whatever process is consuming too much and the computer keeps running flawlessly. However, on Linux, my computer just... freezes. It freezes and stops responding. Not even the mouse moves. Having to use a userspace OOM is the most inelegant thing I've seen. So I need to have 2 OOMs so that the good one can beat the bad one? It's so redundant, it's plain stupid. How come NOBODY is doing anything. If I knew c++ I would for sure send a patch.
- School-Cotton 5y ago
  The Linux kernel is written in C, not C++.
  - ladyanita22 5y ago
    o...kay?
- xioxox 5y ago
  You can already disable the linux memory overcommit feature if you want linux to never allow more memory to be allocated than exists. However, you may run into problems with programs which rely on the ability to allocate more memory than they need, or if you computer has low amounts of memory.
- rnhmjoj 5y ago
  The reason the OOM killer never kicks in is because you actually never run into an OOM, or almost never. What usually happens is that in near-OOM conditions, the kernel starts reclaiming memory pages backed by file (sometimes called "trashing"). This operation manages to keep some extra memory available, but it makes the system almost unresponsive because it's constantly copying memory back and forth from the disk. It may take anywhere between minutes to hours before the system finally OOMs and the OOM killer is invoked. This problem has been there forever but has been made worse by the improved speeds of modern storage technologies: with slower disk I/O, the OOM condition was reached sooner. There are several solutions: - Buy more RAM: if your system routinely goes nearly OOM something is not right. - Add a (small) swap. It doesn't have to be a partition: nowadays most filesystems support swap file. Just create an empty file and mark it as swap. - Limit the amount of thrashing or protect some pages from being reclaimed. This has been proposed by Google first and several other people since then, but AFAIK it has never been implemented in the mainline kernel. Regarding the latter solution, there is a patchset called le9-patch[1] that is included in some alternative Linux kernels and it should be relatively safe to use. [1]: https://github.com/hakavlad/le9-patch https://github.com/hakavlad/le9-patch
  - yunohn 5y ago
    I’m not sure if you misread the previous comment, but their point is quite commonly experienced by people who use Linux vs MacOS/Windows. All hardware being the same (RAM, SSD, CPU) as OOM is reached, Linux will freeze, whereas Windows continues to run smoothly. All OSes try to reclaim memory pages, just Linux seems to hang the user space while doing so. As someone who has dual-booted Windows and Linux for a decade, I can 100% attest to this glaring problem.
    - ladyanita22 5y ago
      Absolutely. This could be fixed. Android doesn't behave like that, and it's not too far from the mainline kernel nowadays.
  - deleted 5y ago
    [deleted]
  - kdmytro 5y ago
    I am sure that this distinction between a near-OOM condition and an actual OOM condition matters to someone familiar with the current kernel implementation. You seem confident describing what happens when the memory gets closer to full, so I believe you. However, the user experiences the PC freeze during certain conditions, however you choose to name them, and it is during that freeze period the user needs a program to be killed to free some memory and prevent the freeze. I would take one crashed program over power cycling the entire PC any day of the week.
    - rnhmjoj 5y ago
      > I am sure that this distinction between a near-OOM condition and an actual OOM condition matters to someone familiar with the current kernel implementation. You seem confident describing what happens when the memory gets closer to full, so I believe you. I'm not a kernel developer or anything like that, I've just spent some time investigating why this issue happens and has been happening for more than 10 years now. > the user experiences the PC freeze during certain conditions, however you choose to name them I'm not trying to defent the Linux kernel, I just described how it works. In particular it's not true that the OOM killer "takes too long" or doesn't work: it's just not invoked at all. If you invoke it manually (enable the magic SysRq with`sysctl kernel.sysrq=1` and press `alt-sysrq-f`) it does its job and solves the OOM instantly. So, if you don't want to deal lockups and don't like an OOM userspace daemon (I don't), these are the possible solutions. > I would take one crashed program over power cycling the entire PC any day of the week. On a laptop or desktop PC, you don't need to power cycle in a near-OOM: use the magic SysRq key.
- AshamedCaptain 5y ago
  Huh? I have never seen desktop Windows killing process due to out of memory -- does it even do that? It does thrash much more gracefully than Linux, though. In fact the "your computer is low on memory" prompt actually can show up even when severely thrashing, something utmost impossible in Linux (even starting something like zenity may take hours..).
- fowl2 5y ago
  The reason is that Windows doesn’t have fork(), and therefore doesn’t have to promise huge multiples of the available memory only to be left holding the bag when that fiction failed. Look up “overcommit” if you’re interested.
- cmurf 5y ago
  The kernel OOM killer is only concerned about kernel survival. It isn't designed to care about user perception of system responsiveness. That's what resource control via cgroups is about. Fedora desktop folks (both GNOME and KDE) are working on ensuring minimum resources are available for the desktop experience, via cgroups, which then applies CPU, memory, and IO isolation when needed to achieve that. Also, systemd-oomd is enabled by default. The resource control picture isn't completely in place yet, but things are much improved.
  - ladyanita22 5y ago
    Okay, this is a good explanation of what's going on.
  - jeffbee 5y ago
    cgroups often make the situation worse, not better, by insisting that a small memcg drop caches because that control group is full while the system overall has plenty of resources. This can lead to a system severely swapping for no apparent reason. Putting desktop apps into individual cgroups is one of the more counter-productive ideas that has cropped up lately.

by esgwpl 5y ago

ladyanita22 5y ago

The kernel OOM is just plain broken. I can't understand how can it be possible that, on Windows, whenever I run out of RAM the OS kills whatever process is consuming too much and the computer keeps running flawlessly. However, on Linux, my computer just... freezes. It freezes and stops responding. Not even the mouse moves. Having to use a userspace OOM is the most inelegant thing I've seen. So I need to have 2 OOMs so that the good one can beat the bad one? It's so redundant, it's plain stupid. How come NOBODY is doing anything. If I knew c++ I would for sure send a patch.

School-Cotton 5y ago
The Linux kernel is written in C, not C++.
- ladyanita22 5y ago
  o...kay?
xioxox 5y ago
You can already disable the linux memory overcommit feature if you want linux to never allow more memory to be allocated than exists. However, you may run into problems with programs which rely on the ability to allocate more memory than they need, or if you computer has low amounts of memory.
rnhmjoj 5y ago
The reason the OOM killer never kicks in is because you actually never run into an OOM, or almost never. What usually happens is that in near-OOM conditions, the kernel starts reclaiming memory pages backed by file (sometimes called "trashing"). This operation manages to keep some extra memory available, but it makes the system almost unresponsive because it's constantly copying memory back and forth from the disk. It may take anywhere between minutes to hours before the system finally OOMs and the OOM killer is invoked. This problem has been there forever but has been made worse by the improved speeds of modern storage technologies: with slower disk I/O, the OOM condition was reached sooner. There are several solutions: - Buy more RAM: if your system routinely goes nearly OOM something is not right. - Add a (small) swap. It doesn't have to be a partition: nowadays most filesystems support swap file. Just create an empty file and mark it as swap. - Limit the amount of thrashing or protect some pages from being reclaimed. This has been proposed by Google first and several other people since then, but AFAIK it has never been implemented in the mainline kernel. Regarding the latter solution, there is a patchset called le9-patch[1] that is included in some alternative Linux kernels and it should be relatively safe to use. [1]: https://github.com/hakavlad/le9-patch https://github.com/hakavlad/le9-patch
- yunohn 5y ago
  I’m not sure if you misread the previous comment, but their point is quite commonly experienced by people who use Linux vs MacOS/Windows. All hardware being the same (RAM, SSD, CPU) as OOM is reached, Linux will freeze, whereas Windows continues to run smoothly. All OSes try to reclaim memory pages, just Linux seems to hang the user space while doing so. As someone who has dual-booted Windows and Linux for a decade, I can 100% attest to this glaring problem.
  - ladyanita22 5y ago
    Absolutely. This could be fixed. Android doesn't behave like that, and it's not too far from the mainline kernel nowadays.
- deleted 5y ago
  [deleted]
- kdmytro 5y ago
  I am sure that this distinction between a near-OOM condition and an actual OOM condition matters to someone familiar with the current kernel implementation. You seem confident describing what happens when the memory gets closer to full, so I believe you. However, the user experiences the PC freeze during certain conditions, however you choose to name them, and it is during that freeze period the user needs a program to be killed to free some memory and prevent the freeze. I would take one crashed program over power cycling the entire PC any day of the week.
  - rnhmjoj 5y ago
    > I am sure that this distinction between a near-OOM condition and an actual OOM condition matters to someone familiar with the current kernel implementation. You seem confident describing what happens when the memory gets closer to full, so I believe you. I'm not a kernel developer or anything like that, I've just spent some time investigating why this issue happens and has been happening for more than 10 years now. > the user experiences the PC freeze during certain conditions, however you choose to name them I'm not trying to defent the Linux kernel, I just described how it works. In particular it's not true that the OOM killer "takes too long" or doesn't work: it's just not invoked at all. If you invoke it manually (enable the magic SysRq with`sysctl kernel.sysrq=1` and press `alt-sysrq-f`) it does its job and solves the OOM instantly. So, if you don't want to deal lockups and don't like an OOM userspace daemon (I don't), these are the possible solutions. > I would take one crashed program over power cycling the entire PC any day of the week. On a laptop or desktop PC, you don't need to power cycle in a near-OOM: use the magic SysRq key.
AshamedCaptain 5y ago
Huh? I have never seen desktop Windows killing process due to out of memory -- does it even do that? It does thrash much more gracefully than Linux, though. In fact the "your computer is low on memory" prompt actually can show up even when severely thrashing, something utmost impossible in Linux (even starting something like zenity may take hours..).
fowl2 5y ago
The reason is that Windows doesn’t have fork(), and therefore doesn’t have to promise huge multiples of the available memory only to be left holding the bag when that fiction failed. Look up “overcommit” if you’re interested.
cmurf 5y ago
The kernel OOM killer is only concerned about kernel survival. It isn't designed to care about user perception of system responsiveness. That's what resource control via cgroups is about. Fedora desktop folks (both GNOME and KDE) are working on ensuring minimum resources are available for the desktop experience, via cgroups, which then applies CPU, memory, and IO isolation when needed to achieve that. Also, systemd-oomd is enabled by default. The resource control picture isn't completely in place yet, but things are much improved.
- ladyanita22 5y ago
  Okay, this is a good explanation of what's going on.
- jeffbee 5y ago
  cgroups often make the situation worse, not better, by insisting that a small memcg drop caches because that control group is full while the system overall has plenty of resources. This can lead to a system severely swapping for no apparent reason. Putting desktop apps into individual cgroups is one of the more counter-productive ideas that has cropped up lately.