10 ms·

Help, Linux ate my RAM

102 points by ez77 15y ago

bshep 15y ago
Is there any way to have top display this information?
- viraptor 15y ago
  Use `htop` instead. It displays the information in a bit more verbose way. You'll get each section of "used" memory colour-coded, so that the last yellow area can be ignored as cache.
  - datagramm 15y ago
    yep +1 for htop.
- obtu 15y ago
  free does the job of taking cache into account (mentioned in the original post). If you've read the neugierig post, and want a better per-process monitor: gnome-system-monitor has a top-like monitor as well as graphs, and measures memory properly (including a discount for shared maps); smem works in the console; it doesn't have a term interface like top, but it can be combined with watch.
- bcl 15y ago
  atop is what I use these days, very flexible.
click170 15y ago
It's cute that this pops up every few years, and I think it points to a steady (if slow) attraction of new users to Linux.
- scott_s 15y ago
  People say the same things on Mac: http://news.ycombinator.com/item?id=3584609 http://news.ycombinator.com/item?id=3584609 It represents a fundamental misunderstanding of how modern OSes work. That misunderstanding is not the problem; modern OSes are complex pieces of software, and most people shouldn't have to understand them. OSes should just work. The problem comes in when people who don't understand how they work get the itch to "improve" their system.
  - jws 15y ago
    Programmers need to remember that users are like this: (from an Apple discussion thread) … [periodically]'installd' begins using 100% of all 4 CPU cores, my fan goes full speed and the whole computer gets very hot. Seems to happen before Software Update checks for updates. I usually go to the terminal and kill the 'installd' process, which reduces the fan speed and heat to normal within a minute. I wonder if this guy is also one that says you need to reinstall from scratch every few months to keep things working?
    - nandemo 15y ago
      I'm a programmer but not a Mac user. I fail to see what is wrong with the bug report above. Care to explain?
      - jws 15y ago
        "installd" is a completely undocumented (outside of Apple) daemon that is somehow involved in installing your software (it runs while installing App store programs) and checking to see what updates you might need. It has no user visible existence. No one would ever know about it unless they ran "ps" or something similar. They will find no hint at what it does, or if killing it is safe.
        nandemo 15y ago
        Thanks for the explanation. I still don't think it's a good example in the context of this thread. My impression is similar to onli's: http://news.ycombinator.com/item?id=3699158 http://news.ycombinator.com/item?id=3699158
      - paxswill 15y ago
        installd is part of framework that handles installed packages. It's fired up as part of the normal software update process (although it really shouldn't be taking a core for itself). Killing it would probably abort the software update process.
    - 15y ago
- dfc 15y ago
  It makes me feel old. As soon as I saw the headline I knew what the story was going to be. Another fresh wave of young faces learning about memory...I just wish they'd stay off the lawn.
  - spudlyo 15y ago
    It makes me feel a little embarrassed for HN actually. This article might be a revelation for the noobs on the Linux subreddit, but I'd expect the HN crowd to find it pretty pedestrian. Not only that, but in some cases it is flat-out wrong: No, disk caching only borrows the ram that applications don't currently want. It will not use swap. If applications want more memory, they just take it back from the disk cache. They will not start swapping. Try again. You can tune this to some extent with /proc/sys/vm/swappiness but Linux is loathe to abandon buffer cache, and will often choose to swap old pages instead. I have learned this the hard way. For example, on a database machine (where > 80% of the memory is allocated to the DB's buffer pool) try to take a consistent filesystem snapshot of the db's data directory and then rsync it to another machine. The rsync process will read a ton of data, and Linux will dutifully (and needlessly) try to jam this into the already full buffer cache. Instead of ejecting the current contents of the buffer cache, Linux will madly start swapping out database pages trying to preserve buffer cache. Some versions of rsync support direct i/o on read to avoid this, but they're not mainstream and readily available on Linux. You can also use iflag=direct with dd to get around this problem.
mwexler 15y ago
firefoxatemyram.com is still available. Perhaps we can put a site up there as well.
- jff 15y ago
  Yep, firefox is currently 555 MB resident and using 1.5 GIGABYTES of virtual memory space. Goddamn, firefox, you are a pig. Saddest thing? I've got gmail, github, and maybe 10 static pages open. Linux just looks like it ate your RAM. Firefox straight up does eat it.
  - tiles 15y ago
    Am I missing something? Does Firefox not use unused RAM for cache in a similar manner as Linux uses unused RAM?
    - ArbitraryLimits 15y ago
      555 MB is still a lot. That's the actual memory usage.
      - scott_s 15y ago
        Modern web browser are complicated pieces of software that are also the nexus for most people's interactions with their computer. tiles' point is correct: Firefox will tend to cache things much like the OS itself. We should expect it to use a lot of memory. If your system performance has not suffered, there is no problem.
        ArbitraryLimits 15y ago
        I'm sorry, but this reply doesn't make any sense to me. If applications have to cache something, they should use the file system, which would not affect their resident memory at all. Maybe I'm just old school, but where I come from operating systems handle the memory hierarchy, not applications.
plaes 15y ago
The cache can be cleared via `/proc/sys/vm/drop_caches`. http://linux-mm.org/Drop_Caches http://linux-mm.org/Drop_Caches
- mjb 15y ago
  Yes it can, but you probably don't want to do that. You definitely don't want to do it in an automated way when the machine is experiencing memory pressure. There are very good reasons that Linux (and most other modern operating systems) makes aggressive use of page caches and buffers. For the vast majority of applications dropping these caches is going to reduce performance considerably (disk is really really slow) and most applications for which this isn't true are probably using O_DIRECT anyway. The arguments in favor of page caching are: (a) disks have very high latency (b) disks have relatively low bandwidth (c) for hot data RAM is cheaper disk IO both in dollars and in watts [1] and (d) it's basically free because the memory would have been unused anyway. The arguments against page caching are: (a) occasionally the kernel will make poor choices and do something sub-optimal and (b) high numbers in 'free' make me feel better. Too many inexperienced operators (or those experienced on other OSs) confuse disadvantage (a) for disadvantage (b) and decide to drop caches using a cron job. [1] Old but good: ftp://ftp.research.microsoft.com/pub/tr/tr-97-33.pdf
  - plaes 15y ago
    Yes, it was actually sort of a response to that webpage that said it was not possible to free this cached memory. The cache dropping is actually useful when you are doing benchmarking...
    - mjb 15y ago
      > The cache dropping is actually useful when you are doing benchmarking... Agreed. My response was more to the "Let's Put 'echo 3 > /proc/sys/vm/drop_caches' In Cron and Get Free RAM!!!!" thinking, which sadly seems to be widespread.
illumin8 15y ago
I've tried to educate Oracle DBAs on why top is wrong and their memory really isn't being used. It's painful and they often refuse to believe that I know what I'm talking about, and that they should use the free -m command to see what memory is actually available for use. Is there any particular reason why Oracle DBAs are less likely to believe this? Perhaps it's because most of them grew up in legacy UNIX environments rather than Linux.
- greedo 15y ago
  I've had the same discussion with Websphere administrators who can't grasp the concept of caching and the role of swapping. I even had one admin think that modifying the swappiness setting to keep memory free would be a good idea...
- ivan78 15y ago
  As a longtime Linux user/admin and beginning Oracle DBA I now know that all memory should be occupied by Oracle, not by OS. :-) Serious mode on: I'm sure it's a big problem to be narrow expert. They can be brilliant specialist in their field, but one step aside and they are absolutely helpless.
  - illumin8 15y ago
    Thanks for the great comment. I really wish more people like yourself at least understood Linux memory management. I think it will truly help you to be an exceptional DBA. I can also understand how an expert DBA might not know or care too much about the OS underneath his software, although I would argue that if you understand the OS fundamentals, you will be that much better at whatever specialty you have.
    - ivan78 15y ago
      Thank you. I'm on my way to become exceptional DBA. Oracle is only 2 years younger than me, but unfortunately it has been developing much faster than I am. It literally takes years just to become familiar with all these layers of Oracle technologies. So, it's really hard to blame DBAs for not knowing how underlying OS works. :-)
suboptical 15y ago
Looks like their site went down, have a mirror: http://webcache.googleusercontent.com/search?q=cache:http://www.linuxatemyram.com/ http://webcache.googleusercontent.com/search?q=cache:http://... http://webcache.googleusercontent.com/search?q=cache:http://www.linuxatemyram.com/play.html http://webcache.googleusercontent.com/search?q=cache:http://...
- JshWright 15y ago
  Looks like HN ate all their RAM...
mark-r 15y ago
Does the Linux disk cache push out pages that are used by running applications? I believe Windows does it, though I can't state that for a fact.
VengefullCarot 15y ago
Help! Hacker News link ate my Bandwidth
glfomfn 15y ago
And Hacker News ate your website :-/
gghh 15y ago
I found a good introduction on unix memory caching in chapter 3 (The Buffer Cache) of 'Design of the UNIX Operating System', http://www.amazon.com/Design-Operating-System-Prentice-Hall-Software/dp/0132017997 http://www.amazon.com/Design-Operating-System-Prentice-Hall-... . At least it was good for me (mathematician by training, programmer by profession)
deleted 15y ago
[deleted]
ez77 15y ago
I found this page because I was pretty confused about these issues, and still am... Question for the crowd: In this site the example given says that in reality there are 869MB of used RAM. I'm comparing this with my VPS values, and would like to know if this is the sum of some column in top. Is it? It looks like it's pretty close to the sum of the SHR column. Does this make sense? Thanks in advance.
- mgedmin 15y ago
  You can't really do sums of top columns, because some memory is shared and you'll end up double-counting it. And you can't just subtract the shared memory numbers, because different sets of pages are shared between different sets of processes, and top doesn't give enough information to figure out what's actually happening where. Running the pmaps tool on all pids and summing the Pss number is perhaps the closest you can get to the actual memory use.

102 points by ez77 15y ago

bshep 15y ago

Is there any way to have top display this information?

viraptor 15y ago
Use `htop` instead. It displays the information in a bit more verbose way. You'll get each section of "used" memory colour-coded, so that the last yellow area can be ignored as cache.
- datagramm 15y ago
  yep +1 for htop.
obtu 15y ago
free does the job of taking cache into account (mentioned in the original post). If you've read the neugierig post, and want a better per-process monitor: gnome-system-monitor has a top-like monitor as well as graphs, and measures memory properly (including a discount for shared maps); smem works in the console; it doesn't have a term interface like top, but it can be combined with watch.
bcl 15y ago
atop is what I use these days, very flexible.

click170 15y ago

It's cute that this pops up every few years, and I think it points to a steady (if slow) attraction of new users to Linux.

scott_s 15y ago
People say the same things on Mac: http://news.ycombinator.com/item?id=3584609 http://news.ycombinator.com/item?id=3584609 It represents a fundamental misunderstanding of how modern OSes work. That misunderstanding is not the problem; modern OSes are complex pieces of software, and most people shouldn't have to understand them. OSes should just work. The problem comes in when people who don't understand how they work get the itch to "improve" their system.
- jws 15y ago
  Programmers need to remember that users are like this: (from an Apple discussion thread) … [periodically]'installd' begins using 100% of all 4 CPU cores, my fan goes full speed and the whole computer gets very hot. Seems to happen before Software Update checks for updates. I usually go to the terminal and kill the 'installd' process, which reduces the fan speed and heat to normal within a minute. I wonder if this guy is also one that says you need to reinstall from scratch every few months to keep things working?
  - nandemo 15y ago
    I'm a programmer but not a Mac user. I fail to see what is wrong with the bug report above. Care to explain?
    - jws 15y ago
      "installd" is a completely undocumented (outside of Apple) daemon that is somehow involved in installing your software (it runs while installing App store programs) and checking to see what updates you might need. It has no user visible existence. No one would ever know about it unless they ran "ps" or something similar. They will find no hint at what it does, or if killing it is safe.
      - nandemo 15y ago
        Thanks for the explanation. I still don't think it's a good example in the context of this thread. My impression is similar to onli's: http://news.ycombinator.com/item?id=3699158 http://news.ycombinator.com/item?id=3699158
    - paxswill 15y ago
      installd is part of framework that handles installed packages. It's fired up as part of the normal software update process (although it really shouldn't be taking a core for itself). Killing it would probably abort the software update process.
  - 15y ago
dfc 15y ago
It makes me feel old. As soon as I saw the headline I knew what the story was going to be. Another fresh wave of young faces learning about memory...I just wish they'd stay off the lawn.
- spudlyo 15y ago
  It makes me feel a little embarrassed for HN actually. This article might be a revelation for the noobs on the Linux subreddit, but I'd expect the HN crowd to find it pretty pedestrian. Not only that, but in some cases it is flat-out wrong: No, disk caching only borrows the ram that applications don't currently want. It will not use swap. If applications want more memory, they just take it back from the disk cache. They will not start swapping. Try again. You can tune this to some extent with /proc/sys/vm/swappiness but Linux is loathe to abandon buffer cache, and will often choose to swap old pages instead. I have learned this the hard way. For example, on a database machine (where > 80% of the memory is allocated to the DB's buffer pool) try to take a consistent filesystem snapshot of the db's data directory and then rsync it to another machine. The rsync process will read a ton of data, and Linux will dutifully (and needlessly) try to jam this into the already full buffer cache. Instead of ejecting the current contents of the buffer cache, Linux will madly start swapping out database pages trying to preserve buffer cache. Some versions of rsync support direct i/o on read to avoid this, but they're not mainstream and readily available on Linux. You can also use iflag=direct with dd to get around this problem.

mwexler 15y ago

firefoxatemyram.com is still available. Perhaps we can put a site up there as well.

jff 15y ago
Yep, firefox is currently 555 MB resident and using 1.5 GIGABYTES of virtual memory space. Goddamn, firefox, you are a pig. Saddest thing? I've got gmail, github, and maybe 10 static pages open. Linux just looks like it ate your RAM. Firefox straight up does eat it.
- tiles 15y ago
  Am I missing something? Does Firefox not use unused RAM for cache in a similar manner as Linux uses unused RAM?
  - ArbitraryLimits 15y ago
    555 MB is still a lot. That's the actual memory usage.
    - scott_s 15y ago
      Modern web browser are complicated pieces of software that are also the nexus for most people's interactions with their computer. tiles' point is correct: Firefox will tend to cache things much like the OS itself. We should expect it to use a lot of memory. If your system performance has not suffered, there is no problem.
      - ArbitraryLimits 15y ago
        I'm sorry, but this reply doesn't make any sense to me. If applications have to cache something, they should use the file system, which would not affect their resident memory at all. Maybe I'm just old school, but where I come from operating systems handle the memory hierarchy, not applications.

plaes 15y ago

The cache can be cleared via `/proc/sys/vm/drop_caches`. http://linux-mm.org/Drop_Caches http://linux-mm.org/Drop_Caches

mjb 15y ago
Yes it can, but you probably don't want to do that. You definitely don't want to do it in an automated way when the machine is experiencing memory pressure. There are very good reasons that Linux (and most other modern operating systems) makes aggressive use of page caches and buffers. For the vast majority of applications dropping these caches is going to reduce performance considerably (disk is really really slow) and most applications for which this isn't true are probably using O_DIRECT anyway. The arguments in favor of page caching are: (a) disks have very high latency (b) disks have relatively low bandwidth (c) for hot data RAM is cheaper disk IO both in dollars and in watts [1] and (d) it's basically free because the memory would have been unused anyway. The arguments against page caching are: (a) occasionally the kernel will make poor choices and do something sub-optimal and (b) high numbers in 'free' make me feel better. Too many inexperienced operators (or those experienced on other OSs) confuse disadvantage (a) for disadvantage (b) and decide to drop caches using a cron job. [1] Old but good: ftp://ftp.research.microsoft.com/pub/tr/tr-97-33.pdf
- plaes 15y ago
  Yes, it was actually sort of a response to that webpage that said it was not possible to free this cached memory. The cache dropping is actually useful when you are doing benchmarking...
  - mjb 15y ago
    > The cache dropping is actually useful when you are doing benchmarking... Agreed. My response was more to the "Let's Put 'echo 3 > /proc/sys/vm/drop_caches' In Cron and Get Free RAM!!!!" thinking, which sadly seems to be widespread.

illumin8 15y ago

I've tried to educate Oracle DBAs on why top is wrong and their memory really isn't being used. It's painful and they often refuse to believe that I know what I'm talking about, and that they should use the free -m command to see what memory is actually available for use. Is there any particular reason why Oracle DBAs are less likely to believe this? Perhaps it's because most of them grew up in legacy UNIX environments rather than Linux.

greedo 15y ago
I've had the same discussion with Websphere administrators who can't grasp the concept of caching and the role of swapping. I even had one admin think that modifying the swappiness setting to keep memory free would be a good idea...
ivan78 15y ago
As a longtime Linux user/admin and beginning Oracle DBA I now know that all memory should be occupied by Oracle, not by OS. :-) Serious mode on: I'm sure it's a big problem to be narrow expert. They can be brilliant specialist in their field, but one step aside and they are absolutely helpless.
- illumin8 15y ago
  Thanks for the great comment. I really wish more people like yourself at least understood Linux memory management. I think it will truly help you to be an exceptional DBA. I can also understand how an expert DBA might not know or care too much about the OS underneath his software, although I would argue that if you understand the OS fundamentals, you will be that much better at whatever specialty you have.
  - ivan78 15y ago
    Thank you. I'm on my way to become exceptional DBA. Oracle is only 2 years younger than me, but unfortunately it has been developing much faster than I am. It literally takes years just to become familiar with all these layers of Oracle technologies. So, it's really hard to blame DBAs for not knowing how underlying OS works. :-)

suboptical 15y ago

Looks like their site went down, have a mirror: http://webcache.googleusercontent.com/search?q=cache:http://www.linuxatemyram.com/ http://webcache.googleusercontent.com/search?q=cache:http://... http://webcache.googleusercontent.com/search?q=cache:http://www.linuxatemyram.com/play.html http://webcache.googleusercontent.com/search?q=cache:http://...

JshWright 15y ago
Looks like HN ate all their RAM...

mark-r 15y ago

Does the Linux disk cache push out pages that are used by running applications? I believe Windows does it, though I can't state that for a fact.

VengefullCarot 15y ago

Help! Hacker News link ate my Bandwidth

glfomfn 15y ago

And Hacker News ate your website :-/

gghh 15y ago

I found a good introduction on unix memory caching in chapter 3 (The Buffer Cache) of 'Design of the UNIX Operating System', http://www.amazon.com/Design-Operating-System-Prentice-Hall-Software/dp/0132017997 http://www.amazon.com/Design-Operating-System-Prentice-Hall-... . At least it was good for me (mathematician by training, programmer by profession)

deleted 15y ago

[deleted]

ez77 15y ago

I found this page because I was pretty confused about these issues, and still am... Question for the crowd: In this site the example given says that in reality there are 869MB of used RAM. I'm comparing this with my VPS values, and would like to know if this is the sum of some column in top. Is it? It looks like it's pretty close to the sum of the SHR column. Does this make sense? Thanks in advance.

mgedmin 15y ago
You can't really do sums of top columns, because some memory is shared and you'll end up double-counting it. And you can't just subtract the shared memory numbers, because different sets of pages are shared between different sets of processes, and top doesn't give enough information to figure out what's actually happening where. Running the pmaps tool on all pids and summing the Pss number is perhaps the closest you can get to the actual memory use.