Retrieve an SSL Certificate from a Server With OpenSSL

I was setting up VMware vRealize Automation’s Active Directory connections the other day and I needed the public SSL certificate for the AD DCs to authenticate correctly. You can use OpenSSL to get that information. I used a Linux shell but this should be do-able from a Mac or with OpenSSL installed on Windows, too. If you wanted to read the SSL certificates off this blog you could issue the following command, all on one line: openssl s_client -showcerts -servername lonesysadmin.net -connect lonesysadmin.net:443 < /dev/null In this case you’ll get a whole bunch of stuff back: CONNECTED(00000003)depth=2 O = Digital Signature Trust Co., CN = DST Root CA X3verify return:1depth=1 C = US, O = Let’s Encrypt, CN = Let’s …

Read More

Fixing Veeam Backup & Replication Proxy Install Errors

Every once in a while I struggle a little to add a new Veeam Backup & Replication hot-add proxy. If you’re like me and seeing proxy install errors maybe some of these will fix you up. This is what worked for me on Windows Server 2016 when I was getting error 0x00000057, “Failed to create persistent connection to ADMIN$” and some other unhelpful messages. If you’re using a hardened Windows installation all bets are off, since the goal of hardening is to intentionally disrupt remote access. I’d get it running with as close to a stock Windows installation as possible and then work from there if you need to secure things further. There are also ways to manually install the Veeam …

Read More

vSphere 6.7 Will Not Run In My Lab: A Parable

“Hey Bob, I tried installing vSphere 6.7 on my lab servers and it doesn’t work right. You tried using it yet? Been beating my head against a wall here.” “Yeah, I really like it. A lot. Like, resisting the urge to be irresponsible and upgrade everything. What are your lab servers?” I knew what he was going to say before he said it. “Dell PowerEdge R610s.” I was actually surprised it was that new, and rack-mountable. “Yeah, you’re out of luck. CPUs before the E3/E5/E7 family didn’t have VT-x extensions in them to make virtualization easy so VMware had to do this thing called binary translation. vSphere 6.5 was the last release that they supported that on because, frankly, it’s slow …

Read More

Intel X710 NICs Are Crap

(I’m grumpy this week and I’m giving myself permission to return to my blogging roots and complain about stuff. Deal with it.) In the not so distant past we were growing a VMware cluster and ordered 17 new blade servers with X710 NICs. Bad idea. X710 NICs suck, as it turns out. Those NICs do all sorts of offloads, and the onboard processor intercepts things like CDP and LLDP packets so that the OS cannot see or participate. That’s a real problem for ESXi hosts where you want to listen for and broadcast meaningful neighbor advertisements. Under Linux you can echo a bunch of crap into the right spot in /dev and shut that off but no such luck on …

Read More

Should We Panic About the KPTI/KAISER Intel CPU Design Flaw?

As a followup to yesterday’s post, I’ve been asked: should we panic about the KPTI/KAISER/F*CKWIT Intel CPU design flaw? My answer was: it depends on a lot of unknowns. There are NDAs around a lot of the fixes so it’s hard to know the scope and effect. We also don’t know how much this will affect particular workloads. The folks over at Sophos have a nice writeup today about the actual problem (link below) but in short, the fix will reduce the effectiveness of the CPU’s speculative execution and on-die caches, forcing it to go out to main memory more. Main memory (what we call RAM) is 20x slower than the CPU’s L2 cache (look below for a good link showing …

Read More

Intel CPU Design Flaw, Performance Degradation, Security Updates

I was just taking a break and reading some tech news and I saw a wonderfully detailed post from El Reg (link below) about an Intel CPU design flaw and impending crisis-level security updates to fix it. As if that wasn’t bad enough, the fix for the problem is estimated to decrease performance by 5% to 30%, with older systems being the hardest hit. Welcome to 2018, folks. In short, an Intel CPU tries to keep itself busy by speculating about what it’s going to need to work on next. On Intel CPUs (but not AMD) this speculative execution doesn’t properly respect the security boundaries between the OS kernel and userspace applications, so you can trick an Intel processor into letting …

Read More

Fixing Veeam “Can’t Delete Replica When It Is Being Processed” Errors

I’ve used Veeam Backup & Replication for a long time now, and when we restructure storage, redeploy VMs, or change our replication jobs we sometimes get situations where we get the error: Error: Can’t delete replica when it is being processed Here’s how I fix it. As always, your mileage may vary, and free advice is often worth what you paid, especially from a stranger on the Internet. Veeam support is probably a safe but much higher latency source of non-free advice. Stop the affected jobs and disable them. Ensure that the replicas are gone, from both the VMware environment (vCenter) and in Backup & Replication (Replicas -> Ready, then right-click and Delete From Disk). Don’t delete it from the …

Read More

Advice On Downgrading Adobe Flash

VMware has a KB article out (linked below) about the Adobe Flash crashes that happen if you’re running the latest version of Flash (27.0.0.170). A lot of us were caught off guard recently when our PCs updated themselves and we couldn’t get into our VMware vSphere environments. The VMware KB article suggests downgrading your Flash client. Left by itself this is completely irresponsible advice. 1. The Adobe Flash update addresses a critical security vulnerability that is being exploited in the wild. The security advisory (linked below) states: Adobe has released a security update for Adobe Flash Player for Windows, Macintosh, Linux and Chrome OS. This update addresses a critical type confusion vulnerability that could lead to code execution. Adobe is …

Read More

Software is Always Broken

I’m sitting here watching my iPhone update to iOS 11.0.1. Apple says that there are just a couple of fixes: some security updates and a fix for the Exchange email problems. The update is sure taking a while, though. That’s consistent with my knowledge of how software development works. Color me skeptical that the first point release of a new iOS only has a couple of changes. My bet is that there are hundreds of fixes for all sorts of problems reported during the beta, but weren’t large enough to stop the release. Development of software like Apple’s iOS or VMware’s vCenter never stops. At a certain point someone takes a snapshot of the way it looks and decides that …

Read More

Install the vCenter Server Appliance (VCSA) Without Ephemeral Port Groups

Trying to install VMware vCenter in appliance/VCSA form straight to a new ESXi host? Having a problem where it isn’t listing any networks, and it’s telling you that “Non-ephemeral distributed virtual port groups are not supported” in the little informational bubble next to it? Thinking this is Chicken & Egg 101, because you can’t have an ephemeral port group without a Distributed vSwitch, and you can’t have a dvSwitch without a vCenter, so how do you install vCenter when you need something that only vCenter can create? Yeah, me too. Here’s the secret, though: don’t remove the default “VM Network” port group, or if you did, put it back, and restart the installer (or just back up to select the host …

Read More