Update to VMware vCenter Server Appliance & NTP Issues

Earlier today I posted “VMware vCenter Server Appliance 5.5.0 Has An Insecure NTP Server.” One of the reasons I like VMware is that they’re responsive to customer issues. This situation is no different. I just spoke with a few guys involved in VMware security, and this is what I’ve learned.

1. There has been mitigation information available internally to VMware Support/GSS since shortly after the vulnerability was published.

If you call VMware Support your best bet is to reference the CVE number, CVE-2013-5211. I have not called VMware Support to confirm this, or to verify that they’re able to properly resolve the issue if you don’t reference the CVE number. In the future I’ll make sure to reference the CVE number if a problem I’m dealing with has one.

2. If you do not have NTP time sync enabled you are not susceptible. If you are using Integrated Windows Authentication on your vCSA you are not susceptible.

With AD the appliance syncs to the time on the domain controllers. These now become alternate remediation paths, though you probably shouldn’t shut NTP time sync off without syncing another way.

3. There is public KB information available on remediating this problem, in the “Timekeeping best practices for Linux guests.“

The fix is really inconspicuous, at the bottom, in red. The VMware folks and I disagree thoroughly that a passing generic mention of how to fix this is an adequate way to inform affected customers of a security issue on a particular shipping product. Their point is that they didn’t have a fix for it yet, and did not want to alarm customers or give attackers more information. My point is that the attackers have already found these open NTP servers by scanning the whole internet for the last month, so there’s no more damage to be done in that regard. Letting people know they’re probably affected and that it can be simply mitigated is a responsible thing to do, especially since there are likely folks out there that don’t know this is even happening to them.

Furthermore, by not publishing a real, public KB article they are doing a disservice to Windows-oriented system administrators who likely are unfamiliar with Linux and the vi editor. Sysadmins don’t like calling support when we can just fix something ourselves and get on with our lives, and it’d be nice to afford sysadmins more familiar with Windows than Linux the same opportunity.

4. There is a pending fix for these issues in all affected VMware products and the fixes will ship soon.

It sounds like we won’t have to wait too long for a real fix for the immediate problem. We also discussed the need for the other things I asked for, like proper firewalling and control of said firewall via the management interface (VAMI on port 5480), and that’s being taken back to Engineering.

5. Concerns about security issues can always be sent to [email protected], which is always staffed and responsive.

I have not tried emailing them, yet, but after my conversations this afternoon I fully believe it.

I’ve updated my original post to reflect some of this new information. To be clear, I’m not at all angry about this. I’m mostly just disappointed, as I expected more public disclosure for a very public vulnerability, especially since mitigation techniques were available. I love being able to search the KB, find a fix for a problem, and move on. That wasn’t really the case here, and I hope this post goes some way towards explaining to all those inside VMware what I’m thinking as a long-time customer, vExpert, and usually friendly extroverted blogger. 🙂

A big thanks to my old friends in VMware for helping out with this, and my new friends in VMware Security!