Linux VM Performance Tuning

Better Linux Disk Caching & Performance with vm.dirty_ratio & vm.dirty_background_ratio

by

This is post #16 in my December 2013 series about Linux Virtual Machine Performance Tuning. For more, please see the tag “Linux VM Performance Tuning.” In previous posts on vm.swappiness and using RAM disks we talked about how the memory on a Linux guest is used for the OS itself (the kernel, buffers, etc.), applications, and also for file cache. File caching is an important performance improvement, and read caching is a clear win in most cases, balanced against applications using the RAM directly. Write caching is trickier. The Linux kernel stages disk writes into cache, and over time asynchronously flushes them to disk. This has a nice effect of speeding disk I/O but it is risky. When data isn’t …

Read More

Account for the Bandwidth-Delay Product with Larger Network Buffers

by

This is post #14 in my December 2013 series about Linux Virtual Machine Performance Tuning. For more, please see the tag “Linux VM Performance Tuning.” At times we can refer to certain network concepts with a “pipe” analogy, where bandwidth is the diameter of the pipe and latency is the length of the pipe. A garden hose has a certain limited amount of “bandwidth” because it has a small interior diameter, and has a lot of latency because it is so long. Water you put in one end of the hose takes a while to come out the other end. A house in the USA likely has a main sewer pipe with 4 inches of “bandwidth” so it can move more, …

Read More

Use a RAM Disk to Improve Disk Access Times

by

This is post #15 in my December 2013 series about Linux Virtual Machine Performance Tuning. For more, please see the tag “Linux VM Performance Tuning.” One of the biggest things folks in IT worry about is data loss. We go to enormous lengths to protect our data, using backups, snapshots, remote replication, rsync, scp, temporary copies in our own home directories… you name it. The thing is, as we look at our systems we sometimes discover that our applications do a lot of writing of temporary files. These temporary files often don’t need any particular protection because they’re transient, yet we write them to our expensive, already overtaxed disk arrays, commit the writes over long distances to our DR sites, …

Read More

Use Paravirtualized Devices On Your Virtual Machines

by

This is post #13 in my December 2013 series about Linux Virtual Machine Performance Tuning. For more, please see the tag “Linux VM Performance Tuning.” para- a prefix appearing in loanwords from Greek, most often attached to verbs and verbal derivatives, with the meanings “at or to one side of, beside, side by side” ( parabola; paragraph; parallel; paralysis ),”beyond, past, by” ( paradox; paragogue ); by extension from these senses, this prefix came todesignate objects or activities auxiliary to r derivative of that denoted by the base word ( parody;paronomasia ), and hence abnormal or defective ( paranoia ), a sense now common in modernscientific coinages ( parageusia; paralexia ). Paravirtual drivers are ones where the virtualization platform does …

Read More

Do Not Collect System Performance Data From Guest OSes

by

This is post #12 in my December 2013 series about Linux Virtual Machine Performance Tuning. For more, please see the tag “Linux VM Performance Tuning.” Fans of the 12th Doctor Who have often heard the phrase “the Doctor lies.” The explanation for his lies is that, because he skips around in time, he knows things that others cannot know yet. Hypervisors are like that, too. Guest OSes don’t know that they aren’t the only OS on the hardware, and the hypervisor lies to them about things like CPUs, RAM, and things like system timers because, like the Doctor, the hypervisor is skipping a VM forward in time. And that’s the rub – only the hypervisor knows what the truth is. …

Read More

Adjust vm.swappiness to Avoid Unneeded Disk I/O

by

This is post #11 in my December 2013 series about Linux Virtual Machine Performance Tuning. For more, please see the tag “Linux VM Performance Tuning.” The Linux kernel has quite a number of tunable options in it. One of those is vm.swappiness, a parameter that helps guide the kernel in making decisions about memory. “vm” in this case means “virtual memory,” which doesn’t mean memory allocated by a hypervisor but refers to the addressing scheme the Linux kernel uses to handle memory. Even on a physical host you have “virtual memory” within the OS. Memory on a Linux box is used for a number of different things. One way it is used is internally for buffers for things like network …

Read More

Zero Out Free Space

by

This is post #10 in my December 2013 series about Linux Virtual Machine Performance Tuning. For more, please see the tag “Linux VM Performance Tuning.” When we talked about the rationale behind storing logs centrally one big reason was thin-provisioned virtual disks. Those disks grow over time because filesystems on a virtual machine currently have no way to tell the underlying storage that they’re done using certain blocks on disk. There is a way to make these VMs thin again, and I wrote about it as step 9 in my guide to preparing Linux Template VMs. In short, we run a script on the VM that writes zeroes to most of the free space on the VM: #!/bin/sh # Determine …

Read More

Disk Partition Alignment Is Still Important

by

This is post #9 in my December 2013 series about Linux Virtual Machine Performance Tuning. For more, please see the tag “Linux VM Performance Tuning.” I have written about this almost yearly (beginning all the way back in 2006), but even now I routinely run across something, like a virtual appliance, that has poor partition alignment. What’s the big deal? In short, misaligned I/O is killing your disk performance. Blame Logical Block Addressing, or LBA. Back in the day, a BIOS interacted with drives by knowing the exact geometry of the drive, namely how many cylinders, heads, and sectors were on a disk (CHS). Unfortunately that limited the size of the drives that could be used, and ignored some basic …

Read More

Gain 30% Linux Disk Performance with noatime, nodiratime, and relatime

by

This is post #8 in my December 2013 series about Linux Virtual Machine Performance Tuning. For more, please see the tag “Linux VM Performance Tuning.” According to Red Hat’s Enterprise Linux 6 Power Management guide, the POSIX family of standards for operating systems “requires that operating systems maintain file system metadata that records when each file was last accessed.” This is called “atime” (a is for access) and is one of three timestamps Linux filesystems keep for their files (the other two are mtime and ctime, modify and change times). You can see these with the ‘stat’ command: $ stat /bin/ls File: `/bin/ls’ Size: 109208 Blocks: 216 IO Block: 4096 regular file Device: fd00h/64768d Inode: 519 Links: 1 Access: (0755/-rwxr-xr-x) …

Read More

Store Logs Centrally

by

This is post #7 in my December 2013 series about Linux Virtual Machine Performance Tuning. For more, please see the tag “Linux VM Performance Tuning.” It is universally true that over time a thin-provisioned virtual machine will slowly expand on disk. This happens because of the way files are deleted from disks. When files are written to disk the storage subsystem writes those blocks out, and the thinly-provisioned disk file grows a bit. But when a filesystem deletes a file all it does is update its own internal storage map to “forget” about the file. Nothing ever removes the data from the disk itself (which is why undelete utilities work, and why there are stringent procedures for securely erasing a …

Read More

+ More Posts +