Oooh, a new version of VMmark is out. From Bruce Herndon on the VROOM! blog:
I am pleased to announce the release of VMmark 2.5, the latest edition of VMware’s multi-host consolidation benchmark. The most notable change in VMmark 2.5 is the addition of optional power measurements for servers and servers plus storage. This capability will assist IT architects who wish to consider trade-offs in performance and power consumption when designing datacenters or evaluating new and emerging technologies, such as flash-based storage.
A long time ago I was pretty skeptical of yet-another-benchmark, but it’s been useful to help compare physical hosts with virtual workloads. Unlike most benchmarks, the results from previous versions are still relevant to the new version. I like that. Normally new benchmarks require complete recertification.
Still, it would be nice if more vendors used this, and on more models. Searching the results for Dell yields four (4) results, all ESX 4.1, on an M620, R710, R910, and R720 (ESX 4.1? Seriously?). In contrast, SPEC.org has 3744 results for Dell, including all the 12th generation servers with most combinations of available CPUs.
As an aside, it’s fun to look through the tuning notes in the benchmark disclosures to see how they set things up, like BIOS parameters, OS tunables, etc. For example, the Cisco B200 M3 VMmark results indicate that they disabled the hardware and adjacent cache line prefetchers. Perhaps they were worried about bus bandwidth being consumed by those operations.