No VMware NSX Hardware Gateway Support for Cisco

I find it interesting, as I’m taking my first real steps into the world of VMware NSX, that there is no Cisco equipment supported as a VMware NSX hardware gateway (VTEP). According to the HCL on March 13th, 2018 there is a complete lack of “Cisco” in the “Partner” category: I wonder how that works out for Cisco UCS customers. As I continue to remind vendors, virtualization environments cannot virtualize everything. There are still dependencies on things like DNS, DHCP, NTP, and AD that need a few physical servers. There will also always be a few hosts that can’t be virtualized because of vendor requirements, politics, and/or fear. Any solution for a virtual environment needs to help take care of those …

Read More

Should We Panic About the KPTI/KAISER Intel CPU Design Flaw?

As a followup to yesterday’s post, I’ve been asked: should we panic about the KPTI/KAISER/F*CKWIT Intel CPU design flaw? My answer was: it depends on a lot of unknowns. There are NDAs around a lot of the fixes so it’s hard to know the scope and effect. We also don’t know how much this will affect particular workloads. The folks over at Sophos have a nice writeup today about the actual problem (link below) but in short, the fix will reduce the effectiveness of the CPU’s speculative execution and on-die caches, forcing it to go out to main memory more. Main memory (what we call RAM) is 20x slower than the CPU’s L2 cache (look below for a good link showing …

Read More

Intel CPU Design Flaw, Performance Degradation, Security Updates

I was just taking a break and reading some tech news and I saw a wonderfully detailed post from El Reg (link below) about an Intel CPU design flaw and impending crisis-level security updates to fix it. As if that wasn’t bad enough, the fix for the problem is estimated to decrease performance by 5% to 30%, with older systems being the hardest hit. Welcome to 2018, folks. In short, an Intel CPU tries to keep itself busy by speculating about what it’s going to need to work on next. On Intel CPUs (but not AMD) this speculative execution doesn’t properly respect the security boundaries between the OS kernel and userspace applications, so you can trick an Intel processor into letting …

Read More

Standards, to and with Resolve

As the holiday season has progressed I’ve spent a bunch of time in the car, traveling three hours at a crack to see friends and family in various parts of Midwestern USA. Much of that travel has been alone, my family having decided to ensconce themselves with my in-laws for the full duration of the week. That has left me ample time to sing aloud in the car, take unplanned detours to collect growlers of beer from esteemed breweries, and to think. I don’t do New Year’s resolutions. I’m not against them, per se, but I just think they’re too conveniently abandoned. I like the noun form of “resolve” better — a firm determination to do something. I aspire to …

Read More

Better Linux Disk Caching & Performance with vm.dirty_ratio & vm.dirty_background_ratio

This is post #16 in my December 2013 series about Linux Virtual Machine Performance Tuning. For more, please see the tag “Linux VM Performance Tuning.” In previous posts on vm.swappiness and using RAM disks we talked about how the memory on a Linux guest is used for the OS itself (the kernel, buffers, etc.), applications, and also for file cache. File caching is an important performance improvement, and read caching is a clear win in most cases, balanced against applications using the RAM directly. Write caching is trickier. The Linux kernel stages disk writes into cache, and over time asynchronously flushes them to disk. This has a nice effect of speeding disk I/O but it is risky. When data isn’t …

Read More

Account for the Bandwidth-Delay Product with Larger Network Buffers

This is post #14 in my December 2013 series about Linux Virtual Machine Performance Tuning. For more, please see the tag “Linux VM Performance Tuning.” At times we can refer to certain network concepts with a “pipe” analogy, where bandwidth is the diameter of the pipe and latency is the length of the pipe. A garden hose has a certain limited amount of “bandwidth” because it has a small interior diameter, and has a lot of latency because it is so long. Water you put in one end of the hose takes a while to come out the other end. A house in the USA likely has a main sewer pipe with 4 inches of “bandwidth” so it can move more, …

Read More

Use a RAM Disk to Improve Disk Access Times

This is post #15 in my December 2013 series about Linux Virtual Machine Performance Tuning. For more, please see the tag “Linux VM Performance Tuning.” One of the biggest things folks in IT worry about is data loss. We go to enormous lengths to protect our data, using backups, snapshots, remote replication, rsync, scp, temporary copies in our own home directories… you name it. The thing is, as we look at our systems we sometimes discover that our applications do a lot of writing of temporary files. These temporary files often don’t need any particular protection because they’re transient, yet we write them to our expensive, already overtaxed disk arrays, commit the writes over long distances to our DR sites, …

Read More

Use Paravirtualized Devices On Your Virtual Machines

This is post #13 in my December 2013 series about Linux Virtual Machine Performance Tuning. For more, please see the tag “Linux VM Performance Tuning.” para- a prefix appearing in loanwords from Greek, most often attached to verbs and verbal derivatives, with the meanings “at or to one side of, beside, side by side” ( parabola; paragraph; parallel; paralysis ),”beyond, past, by” ( paradox; paragogue ); by extension from these senses, this prefix came todesignate objects or activities auxiliary to r derivative of that denoted by the base word ( parody;paronomasia ), and hence abnormal or defective ( paranoia ), a sense now common in modernscientific coinages ( parageusia; paralexia ). Paravirtual drivers are ones where the virtualization platform does …

Read More

Do Not Collect System Performance Data From Guest OSes

This is post #12 in my December 2013 series about Linux Virtual Machine Performance Tuning. For more, please see the tag “Linux VM Performance Tuning.” Fans of the 12th Doctor Who have often heard the phrase “the Doctor lies.” The explanation for his lies is that, because he skips around in time, he knows things that others cannot know yet. Hypervisors are like that, too. Guest OSes don’t know that they aren’t the only OS on the hardware, and the hypervisor lies to them about things like CPUs, RAM, and things like system timers because, like the Doctor, the hypervisor is skipping a VM forward in time. And that’s the rub – only the hypervisor knows what the truth is. …

Read More

Adjust vm.swappiness to Avoid Unneeded Disk I/O

This is post #11 in my December 2013 series about Linux Virtual Machine Performance Tuning. For more, please see the tag “Linux VM Performance Tuning.” The Linux kernel has quite a number of tunable options in it. One of those is vm.swappiness, a parameter that helps guide the kernel in making decisions about memory. “vm” in this case means “virtual memory,” which doesn’t mean memory allocated by a hypervisor but refers to the addressing scheme the Linux kernel uses to handle memory. Even on a physical host you have “virtual memory” within the OS. Memory on a Linux box is used for a number of different things. One way it is used is internally for buffers for things like network …

Read More