tuning

Use elevator=noop For Linux Virtual Machines

by

This is post #6 in my December 2013 series about Linux Virtual Machine Performance Tuning. For more, please see the tag “Linux VM Performance Tuning.” Modern operating systems are fairly modular, and often have different modules to deal with memory, network I/O, and CPU scheduling. Disk I/O is no exception under Linux, as there are usually four different schedulers a sysadmin can choose from. Red Hat had a nice write-up on these a few years back and it remains relevant today: The Completely Fair Queuing (CFQ) scheduler is the default algorithm in Red Hat Enterprise Linux 4. As the name implies, CFQ maintains a scalable per-process I/O queue and attempts to distribute the available I/O bandwidth equally among all I/O …

Read More

Don’t Start GUIs and X Windows On Servers

by

This is post #5 in my December 2013 series about Linux Virtual Machine Performance Tuning. For more, please see the tag “Linux VM Performance Tuning.” There are stories out there about system administrators with intermittent performance problems on their Windows servers. Like many problems, it would never happen when the administrator was around, but when they walked away from their servers suddenly the issues would crop up. It was the OpenGL screensaver, sucking up all the CPU. GUIs make life easier sometimes, especially if you aren’t as familiar with an environment as you’d like to be. However, they come at a cost: additional RAM, CPU, and disk overhead. In my travels I’ve seen virtual environments more than double their consolidation …

Read More

The Best OS Installation Is Minimal

by

This is post #4 in my December 2013 series about Linux Virtual Machine Performance Tuning. For more, please see the tag “Linux VM Performance Tuning.” In this day & age of virtualization and clouds the best choice for an OS installation, be it a virtual machine template or a physical machine, is a minimal install. From there you can use a configuration management tool like Chef or Puppet to add exactly the packages you need for the host to perform its functions. Doing a minimal install has several advantages: A minimal installation doesn’t install a lot of content that will just sit there and consume disk space. As a result your template VMs will be really small, which leads to fast provisioning …

Read More

Minimal Kickstart File for Red Hat Enterprise Linux 6, CentOS 6, Oracle Linux 6 Virtual Machines

by

This is a helper post in my December 2013 series about Linux Virtual Machine Performance Tuning. For more, please see the tag “Linux VM Performance Tuning.” Here’s a minimal kickstart file for a virtual machine installation of RHEL 6, CentOS 6, or Oracle Linux 6. I use this in conjunction with tools like Puppet and Chef to do the rest of my system configuration. I provision 30 GB thin-provisioned disks to each VM, leaving the rest of it unallocated until needed. While this leads to a disk overcommitment situation that’s manageable, and a tradeoff to ensure more standardization and ease of administration & automation. I remove a number of packages that we don’t necessarily need on a virtual machine, either …

Read More

Performance Tuning and Configuration Management Go Hand In Hand

by

This is post #3 in my December 2013 series about Linux Virtual Machine Performance Tuning. For more, please see the tag “Linux VM Performance Tuning.” I’ve really become a fan of configuration management tools like Chef and Puppet. Those types of tools help keep my systems in sync, help enforce standards I set for my systems, and help me rapidly deploy new systems that look exactly like my existing systems. When I’m doing performance testing & tuning these aspects are helpful. It’s easy to deploy a new virtual machine that looks exactly like the production VMs you have. Tuning is all about making a lot of little changes and seeing what they do, so by building test systems from the …

Read More

One Workload Per Virtual Machine

by

This is post #2 in my December 2013 series about Linux Virtual Machine Performance Tuning. For more, please see the tag “Linux VM Performance Tuning.” Back when the computing world was built on bare metal we often tried to squeeze more than one workload on a physical machine. We did this to save money. Servers are expensive so we wanted to make the most of each one. What we didn’t account for was how complicated things could get. Applications fought with each other over system libraries and DLLs. Security was complicated. And, most relevant to this series, performance tuning became this multivariate dance involving resource limits and other arcane system witchcraft. In the end we saved a little money on …

Read More

For The Best Performance Use a Recent Operating System and a Recent Hypervisor

by

This is post #1 in my December 2013 series about Linux Virtual Machine Performance Tuning. For more, please see the tag “Linux VM Performance Tuning.” Just like in construction, performance tuning is most effective when you have a good foundation. If the underpinnings of your efforts are weak you won’t be able to build a skyscraper, just a small office building. While that’s still better than nothing, there is often a lot to be gained by using the newest versions of your OS & hypervisor of choice. Some quick examples: Windows Server 2012 R2 added Virtual Receive-side Scaling, which allows the load from network traffic to be processed by multiple virtual CPUs. VMware vSphere 5.5 added support for latency-sensitive VMs, …

Read More

How Large Your Linux Swap Partition Should Be

by

This is post #4 in my December-long series on Linux VM performance tuning, Tuningmas. One of those timeless questions in system administration has always been “how much swap space do I configure on my server?” The old rule used to be twice the amount of memory, but does a server with 256 GB of RAM really need a half terabyte of swap? And what about VMs? Swapping on VMs is a serious performance drag. Would it be a good idea to just disable swap completely? One thing to consider is that there’s a tunable kernel parameter, /proc/sys/vm/swappiness, that controls the tendency of Linux to scavenge inactive memory pages and swap them out. It is a number from 0 to 100, …

Read More

Leave Some RAM For Filesystem Cache

by

This is post #3 in my December-long series on Linux VM performance tuning, Tuningmas. Many system administrators don’t realize it, but in most OSes RAM that’s unused by applications goes towards filesystem cache, which speeds disk operations. Some VM “right-sizing” tools don’t take this into account, and recommend pretty tight memory allocations which end up causing more disk I/O in the long term. Trading some RAM for better I/O performance is often a very good move, both for an individual VM and for the virtual environment as a whole. To understand what’s happening on a Linux VM and make a decision about how much RAM we should leave for filesystem cache we need to understand the ‘free’ command: total used …

Read More

Disable GUIs and X Windows on Linux VMs

by

This is post #2 in my December-long series on Linux VM performance tuning, Tuningmas. I remember an old story about people having performance issues on their Windows servers. It would happen intermittently, and never when the system administrator was around. Turns out it was the pretty OpenGL screensaver, which would kick in and slow everything down tremendously. When the admin was around he’d be using the console of the server, so no screen saver, and no problems. Graphical user interfaces make one-off administration tasks easier in some cases, but for virtual environments they come at a cost: additional RAM and CPU overhead. For a virtual environment you should shut them off, or install as little of them as you need, …

Read More

+ More Posts +