Bob Plankers

Bob Plankers is a man who wears many hats. In his professional career he has spent three decades leading teams that design, build, operate, and talk about reliable, secure, and compliant IT infrastructures, both in the public sector and in the high tech industry. He currently focuses on applied security, compliance, and operational resilience for virtualization and cloud platforms.

Redundant Gigabit Management NICs, Please

by

I’ve been doing a lot of system design work lately, building virtualization infrastructure for places where there is no pre-existing infrastructure available (also known as the revered “green field” deployment). One of the biggest issues I’ve had is that 10 Gbps switches can fall back to 1 Gbps when the proper transceiver is installed. However, they cannot go to 10 or 100 Mbps. “So what?” you ask. “Nobody in their right mind uses 10 or 100 Mbps anymore.” Management interfaces do, because the manufacturers haven’t bothered to update them to triple speed NICs (10/100/1000 Mbps). The Dell PowerVault 124T tape library can only do 10/100 Mbps. Brocade fibre channel switches, including their newest models, only have 10/100 Mbps capabilities on …

Read More

Better Linux Disk Caching & Performance with vm.dirty_ratio & vm.dirty_background_ratio

by

This is post #16 in my December 2013 series about Linux Virtual Machine Performance Tuning. For more, please see the tag “Linux VM Performance Tuning.” In previous posts on vm.swappiness and using RAM disks we talked about how the memory on a Linux guest is used for the OS itself (the kernel, buffers, etc.), applications, and also for file cache. File caching is an important performance improvement, and read caching is a clear win in most cases, balanced against applications using the RAM directly. Write caching is trickier. The Linux kernel stages disk writes into cache, and over time asynchronously flushes them to disk. This has a nice effect of speeding disk I/O but it is risky. When data isn’t …

Read More

Account for the Bandwidth-Delay Product with Larger Network Buffers

by

This is post #14 in my December 2013 series about Linux Virtual Machine Performance Tuning. For more, please see the tag “Linux VM Performance Tuning.” At times we can refer to certain network concepts with a “pipe” analogy, where bandwidth is the diameter of the pipe and latency is the length of the pipe. A garden hose has a certain limited amount of “bandwidth” because it has a small interior diameter, and has a lot of latency because it is so long. Water you put in one end of the hose takes a while to come out the other end. A house in the USA likely has a main sewer pipe with 4 inches of “bandwidth” so it can move more, …

Read More

Use a RAM Disk to Improve Disk Access Times

by

This is post #15 in my December 2013 series about Linux Virtual Machine Performance Tuning. For more, please see the tag “Linux VM Performance Tuning.” One of the biggest things folks in IT worry about is data loss. We go to enormous lengths to protect our data, using backups, snapshots, remote replication, rsync, scp, temporary copies in our own home directories… you name it. The thing is, as we look at our systems we sometimes discover that our applications do a lot of writing of temporary files. These temporary files often don’t need any particular protection because they’re transient, yet we write them to our expensive, already overtaxed disk arrays, commit the writes over long distances to our DR sites, …

Read More

Use Paravirtualized Devices On Your Virtual Machines

by

This is post #13 in my December 2013 series about Linux Virtual Machine Performance Tuning. For more, please see the tag “Linux VM Performance Tuning.” para- a prefix appearing in loanwords from Greek, most often attached to verbs and verbal derivatives, with the meanings “at or to one side of, beside, side by side” ( parabola; paragraph; parallel; paralysis ),”beyond, past, by” ( paradox; paragogue ); by extension from these senses, this prefix came todesignate objects or activities auxiliary to r derivative of that denoted by the base word ( parody;paronomasia ), and hence abnormal or defective ( paranoia ), a sense now common in modernscientific coinages ( parageusia; paralexia ). Paravirtual drivers are ones where the virtualization platform does …

Read More

Do Not Collect System Performance Data From Guest OSes

by

This is post #12 in my December 2013 series about Linux Virtual Machine Performance Tuning. For more, please see the tag “Linux VM Performance Tuning.” Fans of the 12th Doctor Who have often heard the phrase “the Doctor lies.” The explanation for his lies is that, because he skips around in time, he knows things that others cannot know yet. Hypervisors are like that, too. Guest OSes don’t know that they aren’t the only OS on the hardware, and the hypervisor lies to them about things like CPUs, RAM, and things like system timers because, like the Doctor, the hypervisor is skipping a VM forward in time. And that’s the rub – only the hypervisor knows what the truth is. …

Read More

Adjust vm.swappiness to Avoid Unneeded Disk I/O

by

This is post #11 in my December 2013 series about Linux Virtual Machine Performance Tuning. For more, please see the tag “Linux VM Performance Tuning.” The Linux kernel has quite a number of tunable options in it. One of those is vm.swappiness, a parameter that helps guide the kernel in making decisions about memory. “vm” in this case means “virtual memory,” which doesn’t mean memory allocated by a hypervisor but refers to the addressing scheme the Linux kernel uses to handle memory. Even on a physical host you have “virtual memory” within the OS. Memory on a Linux box is used for a number of different things. One way it is used is internally for buffers for things like network …

Read More

Zero Out Free Space

by

This is post #10 in my December 2013 series about Linux Virtual Machine Performance Tuning. For more, please see the tag “Linux VM Performance Tuning.” When we talked about the rationale behind storing logs centrally one big reason was thin-provisioned virtual disks. Those disks grow over time because filesystems on a virtual machine currently have no way to tell the underlying storage that they’re done using certain blocks on disk. There is a way to make these VMs thin again, and I wrote about it as step 9 in my guide to preparing Linux Template VMs. In short, we run a script on the VM that writes zeroes to most of the free space on the VM: #!/bin/sh # Determine …

Read More

Disk Partition Alignment Is Still Important

by

This is post #9 in my December 2013 series about Linux Virtual Machine Performance Tuning. For more, please see the tag “Linux VM Performance Tuning.” I have written about this almost yearly (beginning all the way back in 2006), but even now I routinely run across something, like a virtual appliance, that has poor partition alignment. What’s the big deal? In short, misaligned I/O is killing your disk performance. Blame Logical Block Addressing, or LBA. Back in the day, a BIOS interacted with drives by knowing the exact geometry of the drive, namely how many cylinders, heads, and sectors were on a disk (CHS). Unfortunately that limited the size of the drives that could be used, and ignored some basic …

Read More

Gain 30% Linux Disk Performance with noatime, nodiratime, and relatime

by

This is post #8 in my December 2013 series about Linux Virtual Machine Performance Tuning. For more, please see the tag “Linux VM Performance Tuning.” According to Red Hat’s Enterprise Linux 6 Power Management guide, the POSIX family of standards for operating systems “requires that operating systems maintain file system metadata that records when each file was last accessed.” This is called “atime” (a is for access) and is one of three timestamps Linux filesystems keep for their files (the other two are mtime and ctime, modify and change times). You can see these with the ‘stat’ command: $ stat /bin/ls File: `/bin/ls’ Size: 109208 Blocks: 216 IO Block: 4096 regular file Device: fd00h/64768d Inode: 519 Links: 1 Access: (0755/-rwxr-xr-x) …

Read More

+ More Posts +