Easy Dell PowerEdge Firmware Updates, 2019 Edition

I’ve become quite the minimalist in my environments, mostly because I’ve been doing a lot of compliance & security work. Speaking generally, most hardware management tools don’t & won’t pass any form of compliance audit and in that context are way more trouble than they’re worth (negative ROI, see my post “Free, Like a Puppy“). I tend to skip anything that isn’t delivered directly as part of the OS from the OS vendor. It’s a complication I don’t need or want. I use this same thinking when updating Dell PowerEdge firmware.

With Dell hardware you can get the iDRAC to do firmware updates directly, either through the GUI, SSH, or the racadm client tool. Racadm can be obtained from support.dell.com. From there, here’s what I do, often in a PowerShell ForEach loop or pasting from an Excel spreadsheet.

First, I clear the old job log, which makes it easy to see my new jobs and their state, as well as guarantees any old jobs that haven’t run get deleted. Replace <iDRAC> with the IP or host name of the iDRAC itself, and <PASSWORD> with the password for the user.

racadm.exe -r <iDRAC> -u root -p '<PASSWORD>' jobqueue delete --all

Give that a minute to complete and the iDRAC should show no jobs:

From there, issue the command to pull the firmware from downloads.dell.com. This assumes that the iDRACs are properly configured with either a direct route out or a proxy. Having a direct path out — properly protected with firewalls or ACLs, of course — is easiest. It also enables you to set your NTP servers correctly so your SSL works right and your logs have the correct timestamps. #justsayin

However, if these aren’t options for you you might explore the Dell EMC Repository Manager (Google it, the link changes all the time), which allows you to download the updates and place them on your own web server or file share. I’d recommend a web server at this point, because only the most recent iDRACs can do modern SMB protocols, and you really need to have SMB1 off on your stuff.

racadm.exe -r <iDRAC> -u root -p '<PASSWORD>' update -t http -e downloads.dell.com -a FALSE

This tells it to download via HTTP, from downloads.dell.com, and the “-a FALSE” tells it not to restart the machine but just queue the updates for the next reboot. In contrast, if you have a machine that you’re okay with restarting right away you can flip that to TRUE. If there are updates you should see some jobs show up in the queue:

From here, just reboot the hosts. I like doing this as part of a VMware vSphere patch as well so that Update Manager does all the hard work of rebooting my clusters for me. Once that’s done the jobs should change to:

A couple other things to note:

1. I’m serious about getting NTP set up on your iDRACs, it’s important. People block it “because security” but security is exactly why it needs to be on. While you’re in there, set up syslog and turn SSL to TLS 1.2.
2. I usually run the Dell PowerEdge firmware update job once from the iDRAC GUI to see if there are any relevant updates. Then I know what I’m up against. You don’t have to schedule updates in the GUI, but it’ll show them to you. The update function moves around between versions a bit but is usually called “update/rollback” under the iDRAC settings.
3. If you do use a separate repository there are other flags you can reference to help you. Best way to get help is to ask the iDRAC for help, “racadm help update” (use the command format above).
4. If you have iDRAC SSH on you can log in that way and run these commands, too. I don’t recommend enabling SSH unless you really need it, though, it’s just attack surface that bad actors can use and compliance people get jumpy about.
5. You should watch the first host you patch to make sure everything goes as planned. For example, the early versions of 14G PowerEdge servers had some absolutely essential NIC updates that needed the host to be power-cycled, and that could have been disastrous if we weren’t doing staged rollouts. We ended up cancelling those and then orchestrating them with PowerCLI and racadm outside of Update Manager.
6. Don’t preload Dell PowerEdge firmware a long time before you plan to reboot. If the host crashes or you have a power outage it’ll update itself on reboot, and that’s just another complication you’ll have to face in the middle of an outage.
7. That said, don’t ever let a crisis go to waste.
8. If there are iDRAC updates (the iDRAC itself or the OS driver packs) those will apply by themselves automatically regardless of the “-a” flag. This generally causes a brief iDRAC outage and some weirdness during the update, which isn’t a problem if you know it’ll happen. If you check for updates using the GUI you’ll see this and can plan ahead.

As always, good luck.