man8alexd

6 ms·

1.
by man8alexd 15d ago
Location: Belgrade, Serbia (UTC+2) Remote: yes (working exclusively remotely for the past 20 years) Willing to relocate: yes Technologies: AWS, Kubernetes, Nomad, Docker, podman, Hashicorp Consul, Vault, Packer, Terraform, Ansible, Chef, Pu
2.
by man8alexd 1mo ago
Location: Belgrade, Serbia (UTC+2) Remote: yes (working exclusively remotely for the past 20 years) Willing to relocate: yes Technologies: AWS, Kubernetes, Nomad, Docker, podman, HashiCorp Consul, Vault, Packer, Terraform, Ansible, Chef, Pu
3.
by man8alexd 2mo ago
Why do you think it is a better default? What does it solve?
4.
by man8alexd 2mo ago
You have a lot of unused anonymous memory that is better swapped out and used as a file cache. Specifically for Linux swap, see https://chrisdown.name/2018/01/02/in-defence-of-swap.html
5.
by man8alexd 2mo ago
You can't fix the fact that you can't predict the future. The software will always allocate more memory than it needs because it can't predict its future resource usage, so limiting memory allocation with strict overcommit is
6.
by man8alexd 2mo ago
About shared memory included in the memory size calculations https://lkml.iu.edu/1902.2/05674.html
7.
by man8alexd 2mo ago
The kernel keeps track of active/inactive pages by scanning page tables during the reclaim process. It only scans until it finds enough inactive pages to reclaim. Scanning all pages all the time is very expensive from a performance poi
8.
by man8alexd 2mo ago
You can't know when the OOM killer is awake and hunting. The kernel doesn't know that.
9.
by man8alexd 3mo ago
OOM is better. If a program doesn't handle ENOMEM properly, then its state is unpredictable and can lead to data corruption.
10.
by man8alexd 3mo ago
Linux MemAvailable from /proc/meminfo is just an estimation calculated as an arbitrary percentage (50%) of free and potentially reclaimable memory. You can't determine how much of the memory can actually be reclaimed under me
11.
by man8alexd 3mo ago
There is no point in managing memory allocations as they have little relation to actual memory usage. There are also other methods than the OOM killer to handle OOM, like process throttling using cgroups "memory.high" limits.
12.
by man8alexd 3mo ago
In this specific case, the correct behaviour would be to drop a part of the buffer pool until the memory pressure is gone. The context-dependent question is how much and how fast to drop. The current implementation drops to a single configu
13.
by man8alexd 3mo ago
Because no one can predict the future, and they don't know how many resources they will need.
14.
by man8alexd 3mo ago
Maybe you should skip running a useless child process and just use PSI to monitor memory pressure.
15.
by man8alexd 3mo ago
You can always tell the system not to kill your Photoshop (or whatever else) by setting the OOM Score Adjust. This mechanism has existed for almost two decades and systemd has supported it for over a decade.
16.
by man8alexd 3mo ago
You will be surprised by how inaccurate memory measurements are.
17.
by man8alexd 3mo ago
See this retrocomputing.SE question https://retrocomputing.stackexchange.com/questions/32492/ori...
18.
by man8alexd 3mo ago
FreeBSD didn’t have memory overcommit and instead used strict swap reservation - each allocated anonymous memory page was supposed to have a corresponding swap page. This required 2x RAM swap space, otherwise you would get “out of swap” whe
19.
by man8alexd 3mo ago
No, it doesn't handle failing allocations gracefully - https://unix.stackexchange.com/questions/797841/firefox-died...
20.
by man8alexd 3mo ago
Is Firefox low quality?
21.
by man8alexd 3mo ago
People like to reinvent things that they are not aware of. Original BSDs used to use strict swap reservation - every anonymous memory page had to have an associated swap page. You had to have the swap 2x of RAM to allow large processes to f
22.
by man8alexd 3mo ago
MariaDB recently implemented memory PSI monitoring but failed with that in a curious way and disabled it afterwards by default. The failure is that under memory pressure, they flushed the entire InnoDB buffer pool.
23.
by man8alexd 3mo ago
The last paragraph: > On the modern desktop, where programmers don't care about failing malloc(), disabling overcommit is shooting yourself in the foot. As you can observe, the memory allocations start failing long before the memory
24.
by man8alexd 3mo ago
They do have sidecars like prometheus, node_exporter running alongside Postgres and they include them in their MemoryLimit calculations.
25.
by man8alexd 3mo ago
There is some kind of illusion or myth that strict overcommit solves memory management issues.
26.
by man8alexd 3mo ago
cgroups have nothing to do with overcommit and memory allocation. They limit actual memory usage for a specific program or group of programs. If this program tries to use more memory than the cgroup memory limit, the program gets OOM killed
27.
by man8alexd 3mo ago
Yes, many have tried to use strict overcommit on the desktop. It is a good footgun. https://unix.stackexchange.com/a/797888/1027
28.
by man8alexd 3mo ago
Mode 0 (Heuristic) is described incorrectly. All this complex heuristic was removed almost a decade ago. Currently, the kernel refuses a single allocation that exceeds the physical memory. That is all. The article ignores the proper modern
29.
by man8alexd 3mo ago
Windows doesn't need to fork, and you can't fork a large process without overcommit.
30.
by man8alexd 3mo ago
disabling overcommit is trading OOM for random program crashes due to the inability to handle ENOMEM. It also wastes a lot of system memory. https://unix.stackexchange.com/questions/797835/disabling-ov...

More ›