Gain 30% Linux Disk Performance with noatime, nodiratime, and relatime

by Bob Plankers on December 8, 2013 · 8 comments

in Best Practices,Cloud,System Administration,Virtualization

This is post #8 in my December 2013 series about Linux Virtual Machine Performance Tuning. For more, please see the tag “Linux VM Performance Tuning.”

According to Red Hat’s Enterprise Linux 6 Power Management guide, the POSIX family of standards for operating systems “requires that operating systems maintain file system metadata that records when each file was last accessed.” This is called “atime” (a is for access) and is one of three timestamps Linux filesystems keep for their files (the other two are mtime and ctime, modify and change times). You can see these with the ‘stat’ command:

$ stat /bin/ls
 File: `/bin/ls'
 Size: 109208 Blocks: 216 IO Block: 4096 regular file
Device: fd00h/64768d Inode: 519 Links: 1
Access: (0755/-rwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2013-12-07 21:29:08.068475219 -0600
Modify: 2013-05-23 08:10:31.000000000 -0500
Change: 2013-06-11 19:12:14.577018287 -0500

On the surface this sounds like a good thing to track, but like many decisions it has unintended consequences: every read operation on a filesystem isn’t just a read operation, it’s a write operation, too, since the atime needs to be updated.

Reads are easy for systems to do, and they’re easy for systems to cache. Writes are hard to do, as they need to be done in a timely fashion, in a guaranteed order, journaled, etc. Doing a write for every read is a sure way to sap performance, especially since many executables need to dynamically load system libraries. Each time a library is accessed, even from cache, there are writes being made to update atime. Back in 2007, Ingo Molnar noted that with atime enabled “[Linux gives] Windows a 20%-30% performance edge, for almost nothing.”

How do I get my 30% disk performance back?

Simply add “noatime” to your filesystem mount options in /etc/fstab to prevent atime from being updated on file and directory accesses:

/dev/mapper/Volume00-root       /       ext4    defaults,noatime        1       1

You can also use Puppet’s mount type to control this:

mount { '/':
 options => 'defaults,noatime',
 pass => '1',
 dump => '1',
}

Be careful, though. Some applications, like tmpwatch, some backup programs, and some things having to do with email (mutt, pine, sendmail, etc.) care about atime. In those cases you have two options. First, commenter Mike Lowe pointed out that noatime implies nodiratime. If you have an application that cares about file atimes you might gain some performance with using just nodiratime instead of noatime to cover the directories but not the files.

Second, “relatime” is potentially a better option that only updates atime if the file or directory has been modified since the last atime update. You might choose to use ‘noatime’ on most of your filesystems but leave /var/spool and /tmp as ‘relatime’:

mount { '/var/spool':
 options => 'defaults,relatime',
 pass => '2',
 dump => '1',
}

You could also switch tmpwatch to use mtime or ctime, or eliminate it completely and use Puppet’s ‘tidy’ directive. A little testing will go a long way here.

Linux kernel 2.6.30 and newer, as well as the kernels with Red Hat Enterprise Linux 6, mount all filesystems with relatime by default, providing yet another example of why you should use the most recent OS you can. I’d still suggest that, no matter which distribution or kernel you use, you explicitly add the option for the behavior you want.

{ 7 comments }

Mike Lowe December 8, 2013 at 12:53 PM

I believe you only need noatime, it will disable file and directory atime and with those both off relative atime is inconsequential.

Bob Plankers December 8, 2013 at 1:10 PM

Hey look, you’re right (Andrew Morton commented at the bottom of http://lwn.net/Articles/245002/ that noatime is a superset of nodiratime).

relatime is useful for applications that care about atime.

Mike Lowe December 8, 2013 at 1:19 PM

By definition noatime and relatime are mutually exclusive.

Bob Plankers December 8, 2013 at 2:56 PM

Ah, yeah, I see what you’re saying. I think I’ve got it right now. Thanks man.

Bob Plankers December 8, 2013 at 1:18 PM

I updated the post to reflect the new information. Thanks Mike.

Oliver December 8, 2013 at 3:27 PM

“ctime” is not “creation time” as you mention, but change time (correctly indicated in the stat output you pasted). It is updated when file content or permissions/ownership etc are changed, so in practical senses it is used to determine when only the file metadata has changed. There is nothing which indicates when the file was created.

Bob Plankers December 9, 2013 at 9:56 AM

Absolutely correct. I caught that shortly after publication and made the correction, but a lot of RSS readers have no way of noticing an update.

Comments on this entry are closed.

{ 1 trackback }

Previous post:

Next post: