This is intended to help fairly knowledgeable people get IPMI working on their hosts so they can issue remote commands to their hardware. I focus on Red Hat Enterprise Linux on a Dell, but it is likely to work on other hosts, distributions, and OSes, too. This works for me on Dell PowerEdge 1850, PowerEdge 2850, Dell PowerEdge 1950, and Dell PowerEdge 2950 hardware.
Dell PowerEdge 1650, 2650, and 1750 servers have an older implementation of IPMI which will let you issue commands locally, but not to these models over the network.
Before you begin:
The Baseboard Management Controller (BMC) is the thing that implements IPMI. It piggybacks on the first built-in NIC so you have to have that attached to the network on the hosts you wish to manage. It uses its own IP address (so you need an extra one).
If you are new to this get a server that is nearby to act as your test machine. Most of the IPMI commands that shut the host down also kill the BMC. If that happens you’ll need to go power the machine on manually. Until you figure out exactly what is okay and what isn’t you’ll be pushing the power button a lot.
You will need a Linux host to send the IPMI commands from. Undoubtedly there are ways to send these commands from other operating systems, but as I am a Linux guy I’ll use that as my example. Feel free to post comments addressing other OSes.
Getting the OS prepared:
1) Install IPMItool and the startup scripts. On Red Hat Enterprise Linux install the OpenIPMI, OpenIPMI-tools, OpenIPMI-libs, and OpenIPMI-devel packages. That will get you everything you need. There are similar packages available for other distributions (SuSE, Ubuntu, CentOS, etc.). You’ll need IPMItool on any machine you want to configure, and any machine you want to send commands from.
2) Enable the IPMI service:
/sbin/chkconfig ipmi on
3) Start the IPMI service, which will load the kernel modules for you:
/sbin/service ipmi start
Configure the BMC for Remote Usage:
1) There are two ways to configure the BMC. You can configure it through the boot-time menu (Ctrl-E), where you can set the management password and IP address information. Or, you can configure it with ipmitool from the OS. Replace my sample IP address, gateway, and netmask with your own:
/usr/bin/ipmitool -I open lan set 1 ipaddr 192.168.40.88
/usr/bin/ipmitool -I open lan set 1 defgw ipaddr 192.168.40.1
/usr/bin/ipmitool -I open lan set 1 netmask 255.255.255.0
/usr/bin/ipmitool -I open lan set 1 access on
2) Secure the BMC, so unauthorized people can’t power cycle your machines. To do this you want to change the default SNMP community, the “null” user password, and the root user password. First, set the SNMP community, either to a random string or something you know:
/usr/bin/ipmitool -I open lan set 1 snmp YOURSNMPCOMMUNITY
Then set the null user password to something random. Replace CRAPRANDOMSTRING with something random and secure:
/usr/bin/ipmitool -I open lan set 1 password CRAPRANDOMSTRING
Last, set the root user password to something you know:
/usr/bin/ipmitool -I open user set password 2 REMEMBERTHIS
Double-check your settings with:
/usr/bin/ipmitool -I open lan print 1
Trying it:
1) You can set an environment variable, IPMI_PASSWORD, with the password you used above. That will save some typing:
export IPMI_PASSWORD="REMEMBERTHIS"
If you use this substitute the “-a” in the following commands with a “-E”.
2) From another machine issue the following command, obviously replacing the IP with the target BMC’s IP:
/usr/bin/ipmitool -I lan -U root -H 192.168.40.88 -a chassis power status
You should get something like:
Chassis Power is on
If you get anything else, or nothing, double-check to make sure the BMC is set right, you entered the right password, and the IP it has is reachable from the machine you’re on. You can double-check your work via the Ctrl-E boot menu, too.
Beyond that, get familiar with:
/usr/bin/ipmitool -I lan -U root -H 192.168.40.88 -a chassis power off
/usr/bin/ipmitool -I lan -U root -H 192.168.40.88 -a chassis power cycle
/usr/bin/ipmitool -I lan -U root -H 192.168.40.88 -a sel list
For me, a “chassis power off” command kills the box. “SEL” is the system event log.
You can issue all of these commands locally, too:
/usr/bin/ipmitool sel list
Hopefully this helps a little. If you find any errors in this please leave me a comment or send me an email. Thanks!
I’m using IPMI2 Serial over LAN with our PE1950s. By default the Dells provide this on ttyS1 at 57600.
I’ve then tied this in with conmux to provide a seamless console solution across many different platforms. It will be nice once everything provides IPMI, until then I’ve got a variety of console access methods, it’s nice to abstract away all the differences.
http://test.kernel.org/autotest/AutoTest?action=AttachFile&do=get&target=conmux.html
That is cool. I’ve done very little with the Serial-over-LAN stuff. I might have to give it a shot if it’s working for you. Does it do the POST/boot-time console stuff, too, or just work once you have a running OS?
The SOL can show you the boot-time info (just add console=ttyS0,19200n8 to the kernel parameters in grub) and of course will give you a nice login if you run a agetty in your initab.
I use the IPMI on 1850’s and 1950’s but for some reason console redirection is really flaky on the 1850’s. You have to use a SOL proxy program running on another machine. You connect to that through telnet, and the proxy redirects from/to a machine.
But I keep getting timeouts and disconnects when trying to do something useful. For this reason performing a remote PXE boot and re-install is completely unusable. I haven’t tried this on the 1950’s, these machines have IPMI 2.0 and you should be able to do console redirection through ipmitool only.
I have to add I’m running the 1850’s on a VLAN, and with the BMC configuration not using any VLAN designation. Maybe that’s the problem, but I’m guessing the SOL proxy is just plain Dell crap.
I’ve also got a Sun X2100 server with a Tyan IPMI 1.5 board, and that just works perfectly in every way (on the same VLAN), so Dell can still learn something from Sun/Tyan 😉
I just tried to set this up using Centos 5’s OpenIPMI tools on a PE850, and after issuing the initial command[*], the machine lost contact with the network completely. Someone suggested that the BMC and the broadcom network bits might be fighting with each other, but I wasn’t able to find any more details. Sadly, I didn’t heed your proximity warnings and someone is walking to the datacenter to revive the machine for me now…
[*] ipmitool -I open lan set 1 ipaddr 10.1.1.4
Hi,
I read this article. Its nice. However I was having some other error
when I reboot remote system I get following errors:
Error: No response to keepalive-Terminating sesn
Error: No response de-activating SOL payload
Any idea about this.
Thanks,
Shyam
Hi
is there any neat document on how to configure the sending traps using IPMI. We are running suse 9 ES with IPMI 2
Just wanted to say thanks for this article. I was able to adapt it pretty easily for my ‘ASUS ASMB3-SOL PLUS’ IPMI BMC. Saved me a lot of hassle. 🙂
great article ! thank you !
i am now able to monitor the DELL R200 with IPMI.
I am running XEN Server on the R200.
Since it’s CentOS it also works fine.
Good stuff. I just now got here with what I’m doing. I wish I’d found this before.