Configuring and Securing IPMI on Dell PowerEdge x8xx Hardware

Update: I have posted a newer version of this article.

So I just spent about 10 hours of my life getting IPMI working on some Dell PowerEdge 1850s and 2800s so I can cycle their power over the network, and turn them on if the power goes out. That was a lot more challenging than I thought it’d be, mostly because there are about zero good places for someone who generally knows what they’re doing to get an idea of where to start. So here’s my slightly-convoluted guide to configuring IPMI on eighth-generation Dell PowerEdge servers, with emphasis on Red Hat Enterprise Linux AS 4.

This will probably work for Dell PowerEdge 1650s, 2650s, and 1750s, too, but they don’t have a full-fledged BMC. You’ll be able to use IPMItool locally but not over the network. Obviously if you’re using Red Hat Enterprise Linux AS 3 you should grab the packages labeled RHEL3.

You also might want to read Dell’s whitepaper on “Managing Dell PowerEdge Servers using IPMItool.”

Getting it Working

Step 1: Find a server that supports IPMI that is within reach. You’ll soon discover that some of the commands cause bad things to happen, so while you’re learning you’ll want the machine reachable if you put it in an unresponsive state. You’ll also need a machine to send the IPMI commands from.

Step 2: The Baseboard Management Controller (BMC) is piggybacking on the first built-in NIC (eth0, probably) so you’ll have to have that attached to the network.

Step 3: Get the tools you’ll need from Dell. In short, the Dell Dynamic Kernel Module Support utilities are going to be the way to go. The place that has the stuff you need is http://linux.dell.com/files/openipmi/. Grab the latest appropriate openipmi-*-1dkms.tar.gz, and then grab an IPMItool from http://linux.dell.com/files/openipmi/ipmitool/.
You could also get IPMItool from http://ipmitool.sourceforge.net/ and build it yourself.

Step 4: Install the Dell OpenIPMI RPMs by untarring the file and running install.sh. Install IPMItool either by building it or installing the RPM. You will need to install IPMItool on both the target machine and the machine from which you’ll send commands.

Step 5: By default Dell doesn’t enable the IPMI service, so enable it with

/sbin/chkconfig --level 35 ipmi on

Step 6: We have to tell the DKMS autoinstaller, which runs at boot time, to keep these modules updated with our kernels. That means that when you install a new kernel and reboot anything managed by DKMS will get rebuilt. Edit the file /usr/src/openipmi-*/dkms.conf and add a line underneath “PACKAGE_VERSION” that says:

AUTOINSTALL="yes"

I’m not quite sure why they don’t do this automatically. Seems like it’d be handy to me.

Step 7: Unless you can reboot right now we’ll need to start the services manually:

/sbin/service dkms_autoinstaller start
/sbin/service ipmi start

These commands will build the modules for you if your running kernel doesn’t have them already, and then start IPMI. At this point you should have a /dev/ipmi0, which is what ipmitool needs in order to interact with the BMC. It should also persist through a reboot.

Configuring the BMC for Remote Usage

Step 8: The BMC needs an IP address that is in the same subnet/VLAN as whatever you have eth0 (the primary onboard NIC) on. I am assuming that you don’t have 802.1q VLAN tagging on. If you do there the BMC understands that, but I don’t know how to enable that except through the Ctrl-E boot menu at startup. If someone lets me know what to do I’ll add it to this post.

You can set the IP for the BMC via the Ctrl-E boot menu. That might be a good way to start, and you can move on to the command-line stuff later.

Once you have the IP address, gateway, etc. you can issue the commands below. Obviously you’ll need to replace the dummy values with real ones:

/usr/bin/ipmitool -I open lan set 1 ipaddr 192.168.40.88
/usr/bin/ipmitool -I open lan set 1 defgw ipaddr 192.168.40.1
/usr/bin/ipmitool -I open lan set 1 netmask 255.255.255.0
/usr/bin/ipmitool -I open lan set 1 access on

Step 9: Now we need to secure the BMC so that random people can’t hose your machines. Do this even if your network is secure. It’s the whole defense-in-depth thing, where you have multiple layers of protection. Plus, you don’t want some n00b admin discovering IPMI and inadvertently rebooting your machines, do you?

I use mkpasswd, which comes with Expect. I love that thing. The command:

/usr/bin/mkpasswd -s 0

will give you a nice eight character random string, devoid of special characters. which you can feed into lots of stuff using backticks.

You have two passwords and one SNMP community string to set. The BMC has a null user, and a user named “root.” One of the passwords, the root user’s, will be the password you use to interact with the BMC via the network.

This sets the SNMP community. Replace YOURSNMPCOMMUNITY with something else, like a string from mkpasswd. Unless, of course, you plan to use the SNMP functionality. I’m not that advanced, yet, and ipmitool does everything I need, so you’re on your own with that:

/usr/bin/ipmitool -I open lan set 1 snmp YOURSNMPCOMMUNITY

This sets the null user’s password. Replace CRAPRANDOMSTRING with something else:

/usr/bin/ipmitool -I open lan set 1 password CRAPRANDOMSTRING

This sets the root user’s password. Whatever you set this to, remember it:

/usr/bin/ipmitool -I open user set password 2 REMEMBERTHIS

Yay!

Double-check your settings with:

/usr/bin/ipmitool -I open lan print 1

Doing Stuff to Your Machine from Afar

Step 10: Now you get to try messing with the power remotely. Go to the machine that just has IPMItool on it and issue the command:

/usr/bin/ipmitool -I lan -U root -H 192.168.40.88 -a chassis power status

Obviously you’ll need to use the IP you set the BMC to in Step 8. After entering the password you used for the root user in Step 9, you should see:

Chassis Power is on

Unless the power is actually off, but at this point I’m trusting that you’re not that clueless. 🙂

If you get anything else, or nothing, double-check to make sure the BMC is set right, you entered the right password, and the IP it has is reachable from the machine you’re on. You can double-check your work via the Ctrl-E boot menu, too.

From here, the sky is the limit. During your testing make sure you’re familiar with:

/usr/bin/ipmitool -I lan -U root -H 192.168.40.88 -a chassis power off

/usr/bin/ipmitool -I lan -U root -H 192.168.40.88 -a chassis power cycle

There are a lot more commands you can issue via ipmitool, so read the man page or the help information.

Let me know if something here is wrong. After 10 hours of messing with this I’m a little fried. 🙂