The danger of collecting data is that you need to know what you’re looking at before you conclude anything.
I say this because every once in a while someone new to system performance statistics starts perusing all the performance graphs we have for our servers. They see a graph like this:
and their reaction, seeing all the green “Used Memory” is “OH MY GOD WE ARE OUT OF RAM GET MORE RAM WHY AREN’T YOU DOING ANYTHING YOU SLOVENLY SYSADMINS OMFG WTF BBQ.”
My reaction:
A) No, you are not out of RAM. Our other monitoring systems tell us when that happens.
B) Operating systems know that RAM is way faster than disks, so when an operating system has RAM that is unused by applications it uses it to cache disk accesses (especially reads) and improve performance. As Martha Stewart would put it, this is a good thing.
C) If an application comes along that needs memory the operating system will shrink the cache and give the application that RAM. When the application quits it’ll go back to being filesystem cache. The operating system will often keep a certain percentage of memory as cache at all times, though, to help performance. This is tunable, though on Linux I haven’t ever needed to change the default behaviour.
D) RAM being used for cache is “in use” so it gets counted as being used. Not the most straightforward, but hey, that’s the way it works. When UNIX says something is free memory it really means it isn’t being used for anything at all. In a lot of cases ‘top’ will lie to you, too, and report the amount of memory used without correcting for cache.
E) You see that orangish line half way up in the graph? That’s “cache memory” and is basically telling us that 60% of system RAM is being used to cache filesystems until an application needs it.
F) On Linux you can see this with the ‘free’ command:
The entry where the “-/+ buffers/cache” row and the “free” column meet is about how much free RAM you have.
So the moral of the story: That crazy 100% RAM usage you’re seeing? It’s just the filesystem cache, and it’s a good thing.
Oh, I remember that conversation well…I was unable to convince a couple users that using RAM was a *good* thing, and insisted they needed more memory. I’m really glad I don’t have to have that talk at my current job.