Minimal Kickstart File for Red Hat Enterprise Linux 6, CentOS 6, Oracle Linux 6 Virtual Machines

This is a helper post in my December 2013 series about Linux Virtual Machine Performance Tuning. For more, please see the tag “Linux VM Performance Tuning.” Here’s a minimal kickstart file for a virtual machine installation of RHEL 6, CentOS 6, or Oracle Linux 6. I use this in conjunction with tools like Puppet and Chef to do the rest of my system configuration. I provision 30 GB thin-provisioned disks to each VM, leaving the rest of it unallocated until needed. While this leads to a disk overcommitment situation that’s manageable, and a tradeoff to ensure more standardization and ease of administration & automation. I remove a number of packages that we don’t necessarily need on a virtual machine, either …

Read More

Performance Tuning and Configuration Management Go Hand In Hand

This is post #3 in my December 2013 series about Linux Virtual Machine Performance Tuning. For more, please see the tag “Linux VM Performance Tuning.” I’ve really become a fan of configuration management tools like Chef and Puppet. Those types of tools help keep my systems in sync, help enforce standards I set for my systems, and help me rapidly deploy new systems that look exactly like my existing systems. When I’m doing performance testing & tuning these aspects are helpful. It’s easy to deploy a new virtual machine that looks exactly like the production VMs you have. Tuning is all about making a lot of little changes and seeing what they do, so by building test systems from the …

Read More

One Workload Per Virtual Machine

This is post #2 in my December 2013 series about Linux Virtual Machine Performance Tuning. For more, please see the tag “Linux VM Performance Tuning.” Back when the computing world was built on bare metal we often tried to squeeze more than one workload on a physical machine. We did this to save money. Servers are expensive so we wanted to make the most of each one. What we didn’t account for was how complicated things could get. Applications fought with each other over system libraries and DLLs. Security was complicated. And, most relevant to this series, performance tuning became this multivariate dance involving resource limits and other arcane system witchcraft. In the end we saved a little money on …

Read More

For The Best Performance Use a Recent Operating System and a Recent Hypervisor

This is post #1 in my December 2013 series about Linux Virtual Machine Performance Tuning. For more, please see the tag “Linux VM Performance Tuning.” Just like in construction, performance tuning is most effective when you have a good foundation. If the underpinnings of your efforts are weak you won’t be able to build a skyscraper, just a small office building. While that’s still better than nothing, there is often a lot to be gained by using the newest versions of your OS & hypervisor of choice. Some quick examples: Windows Server 2012 R2 added Virtual Receive-side Scaling, which allows the load from network traffic to be processed by multiple virtual CPUs. VMware vSphere 5.5 added support for latency-sensitive VMs, …

Read More

Linux VM Performance Tuning, Take 3

A couple years ago I started rewriting my venerable Linux VM Tuning Guide, as a series of posts throughout the month of December inspired by Sysadvent (which is running again, and always has good stuff). I called it “Tuningmas” like Christmas. Yeah, corny, I know, but hey. I got about three posts out, then some life stuff happened, and the rest of my draft posts never made it out. 2011 was a year of tremendous highs and tremendous lows. 2012 was a blur, and 2013 has been, uh, sporty, too, but things are settling down. So I’m trying again. I’m not much of a revisionist blogger but the new series will deprecate all my old work, so I’ll likely go …

Read More

Updates to My vCSA 5.1 to 5.5 Notes

I just updated my notes on upgrading the VMware vCenter Server Appliance from 5.1 to 5.5. I added a couple of things that have become issues: Remove all non-standard users from SSO before the upgrade. If you added users to the 5.1 Single Sign-On system directly those users will be copied to the 5.5 vCSA as members of SYSTEM-DOMAIN. Unfortunately they will then become trapped, undeleteable & unchangeable, as VMware didn’t think to make the SYSTEM-DOMAIN an editable domain. You can see them, and you can still log in, but you cannot remove them or change their passwords. Your only recourse is to remove the permissions for that user from vCenter, which still means they can log in, but won’t have …

Read More

Notes on Upgrading the VMware vCenter Server Appliance 5.1 to 5.5

I’ve done a few upgrades of the VMware vCenter Server Appliance (vCSA) 5.1 now, to the GA release of 5.5 (build 1312297). Here are my observations: You need a second IP temporarily for the upgrade. The way it works is that you deploy a new vCSA, then the two of them talk to each other to do the upgrade. When they’re done copying stuff around the process will shut the old one off and reboot the new one so it’s fully functional. While the need for a second IP is fairly obvious, I managed to overlook it. Don’t specify a hostname for the new vCSA in the OVF/OVA deployment wizard if you don’t want to change the name of the …

Read More

Coho Data

Over the last year or so I’ve been fortunate to work with a bunch of great folks over at Coho Data, who are coming out of stealth mode with the debut of their storage product, the DataStream[0]. I’ve got a write-up over at The Virtualization Practice on the device, but I’ve also got a prerelease unit running in my lab, and I’ve been liking it a lot. Neither Coho nor EMC nor Nutanix will like this comment, but if an Isilon got frisky with a Nutanix cluster the DataStream might well be the love child. Scale-out architecture and great software smarts on value commodity hardware. For both personal and technical reasons I’ve always liked the Isilon products, and they’ve done …

Read More

VMware vSphere 5.5 & Dell 12G Servers: Reliable Memory Technology

A few days ago Dell released BIOS updates for their 12th generation servers. Among all the notes about preparations for the Intel E5-2600 v2 refresh there’s one line that’s of interest to those of us thinking about running vSphere 5.5 on our version 1 12G hardware: New Memory Operating Mode setup option ‘Dell Fault Resilient Mode’ This is a patented new technology from Dell, wherein the hypervisor and system hardware can work together to place the hypervisor in a more redundant section of memory. Dell servers have shipped with a variety of tricks to protect against memory faults, things like Memory Page Retire, which will dynamically remove a page from usable memory space if it encounters an error. However, to …

Read More

Troubleshooting Puppet PostgreSQL Locale Mismatches

I’ve been doing some work lately with VMware Hyperic Server and Puppet, and I’ve been having issues where the Puppet Labs PostgreSQL module refuses to create a PostgreSQL database for me. I try to call it with: class { ‘postgresql’: charset => ‘UTF8’, }-> class { ‘postgresql::server’: config_hash => { ‘listen_addresses’ => ‘127.0.0.1’, ‘manage_redhat_firewall’ => true, ‘postgres_password’ => ‘goatsaresupercool’, }, require => Mount[‘/var/lib/pgsql’], } postgresql::db { ‘HQ’: user => ‘hyperic’, password => ‘sheeparecooltoo’, require => Class[‘postgresql::server’], } …and it throws this error into Puppet’s output: Error: /usr/bin/initdb –encoding ‘UTF8’ –pgdata ‘/var/lib/pgsql/data’ returned 1 instead of one of [0] “No problem,” I said. Since Puppet is kind enough to give me the command it’s trying I switched to the postgres user …

Read More