VMware L2 cache

The other day I had a del.icio.us bookmark for a paper from Dell on VMware ESX Server performance on Dell 2850 and 6850 hardware. If you read it take note of the tests with LAMP stack servers. A 6850 with four 3.33 GHz CPUs with 8 MB of cache could only support 2 more VMs in their benchmark than a server with four 3.66 GHz CPUs and 1 MB of cache.

That’s fairly interesting, because CPUs with a lot of cache on them are usually quite a bit more expensive. This seems to imply that L2 cache is not a big factor in VMware ESX Server VM performance, at least for that particular LAMP workload.

In this case the difference between the processors is about $4000, which is enough to get about 16-24 GB of RAM. Or make some different choices of CPUs.

I take cache into consideration when designing a standalone system, but with VMware clusters I tend to opt for more cache all the time. Maybe a design for a VMware cluster should include CPUs with different amounts of cache on them (but still in the same family) so that you can still use VMotion for maintenance but also optimize the hardware for the workloads.

Then there’s another paper on performance gains with dual-core CPUs on 2850s. Two dual-core CPUs only seem to give a server about 28% more capacity with LAMP stacks. Is it worth the price difference? I’d like to see dual-core CPUs compared to faster single core CPUs. You have to remember that the two cores are sharing the single 800 MHz bus, which can be a problem sometimes.

Hmmm….

(sorry, kinda thinking out loud here)