System Administration

How To Configure Linux Serial Consoles

by

Need a serial console on your Linux box, just in case something happens to the network? Yeah, me too. It’s fairly simple to get it running (at least on Red Hat Enterprise Linux 4), if you have the right steps. This is not an in-depth tutorial on serial consoles, it’s just designed to get you moving faster. I assume you know how to install packages, edit files, etc., or are at least resourceful enough to figure it out. I also assume you’re s-m-r-t enough to try this on a machine you can get to easily, if you mess it up the first time. On newer machines there are often BIOS serial redirection options, too, which might be useful. Conveniently, check …

Read More

Cisco NERV

by

Recently I had a chance to see the Cisco Network Emergency Response Vehicle. It is a truck put together by Cisco as a technology demonstration, but also is used by teams within Cisco to assist emergency response efforts. These guys and this truck have been to New Orleans after Katrina, to New York City for 9/11, and recently to San Diego to assist with the wildfires. My tour guide was Bob Browning, the senior manager of the Tactical Operations Support team. That guy really knows his stuff when it comes to handling disaster situations, with practical experience gained from every situation he’s worked in. From a technology standpoint the truck is pretty cool. Built on an International Truck DuraStar 4300 …

Read More

Not Seeing Partitions on USB Disks

by

I have a 750 GB WD MyBook drive that I move lots of data around on, specifically from work to home and back. Today I wanted to hook it up to my development Linux box. The last time I did this I walked up and plugged it into the front USB ports on the machine (a Dell PowerEdge 2850 running Red Hat Enterprise Linux 4). No sweat. Today was different, though. For whatever reason the partition on the disk didn’t get picked up automatically. I plugged the drive in and got /dev/sdt, but no /dev/sdt1, though fdisk showed it there. Um, WTF. After a few minutes of futzing around with it, unplugging it, plugging it back in, flailing with pvscan, …

Read More

Use a Laptop For Single Machine KVM?

by

Okay, here’s a silly question: I want to use my laptop as the monitor and keyboard for a server. Kinda like KVM over IP, but I only need to attach to a single server. You might ask, “Bob, what sort of ridiculous thing are you trying to do?” I’m glad you asked! Basically I have a bunch of machines deployed all over the place where there aren’t monitors. I hate dragging a monitor with me when I go to visit them. Sometimes there are even KVM-over-IP switches, but no monitor, and getting a network connection to use the KVM-over-IP is a bunch of hassle. Sometimes the machines can do serial console redirection, but not universally and that doesn’t seem to …

Read More

Predictions For 2008

by

What is it with people predicting things for next year? Am I the only one without a crystal ball? I was thinking that if I have to read another predictive blog post I’m going to hurt myself. But then, right as I was picking up the meat tenderizer for a whack on the skull, I thought, “hey, why don’t I post my own predictions for 2008?” 1. Virtualization will continue to exist. All throughout 2008 people will continue to virtualize things, doing it at least slightly faster than they did in 2007. 2. Hardware will get faster and have new features that do new things. Bus, CPU, and network speeds will get faster. Hardware vendors will release new models which …

Read More

One To One Mapping

by

Systems that collect data need a one to one mapping between data entered and its meaning. Truthfully, this is most common for data NOT entered. I’ve seen a lot of systems where a field which is not relevant is left blank. How do you know which ones were left blank intentionally and which ones were accidental? If you’re going to bother collecting data why not be as specific as possible? Use units. Number of CPUs? Did you mean sockets or cores? Amount of RAM? Disk size? Is that in MB, GB, or TB? “N/A” is a perfectly acceptable answer, too, if everybody knows what it means.

Systems Shouldn't Force People To Make Things Up

by

A friend of mine manages the directory services for a large academic institution. He was telling me today about a lot of the bogus information his team has to sort out. Their directory aggregates information from payroll systems, student data systems, and a few other sources. Occasionally, but more frequently than they’d like, someone substitutes words like “NONE” or “NOTPRESENT” or even a single ‘.’ for missing information. This happens often with foreign students, and until they show up for class there is no good way to get the missing data. When these people get assigned a username by my friend’s system they get none, notpresent2, rdnone, or something equally ridiculous (sometimes just a single number, like ‘4’). Eventually these …

Read More

Fun with ipcrm

by

Speaking of screwing up, I was reminiscing about some of the other screwups I’ve made. We were having trouble with a server that appeared to have a memory leak. The major application on it was an Oracle database, and I had been reading something about ipcs and ipcrm and how you should check for orphaned shared memory segments and delete them. Sounds dumb now but at the time it seemed compelling. Not wanting to do anything rash, I checked AIX’s man page for ipcrm, which indicated that ipcrm run with certain flags wouldn’t delete a memory allocation that was still in use. Cool. I ran the command, and it did nothing. I just figured it was a dead end. What …

Read More

You Know It Is Going To Be A Good Day When…

by

You know it’s going to be a good day when you spend 30 minutes debugging a problem which ultimately ends up being a one character typo, an extra ‘p’, in /etc/exports: /export/rippper *(ro) That’s something you just don’t see unless you’re really looking for it. My best screwup yet[0] was a few years back. I’d reported a problem with compiling Python on AIX 4.3.3. The Python guys got back to me and told me that they didn’t have AIX boxes, so if it was going to get fixed I had to do it. So I set about changing the autoconf/configure scripts to do the right thing. I had two terminal windows open, one where I was editing, one where I …

Read More

Anatomy Of a Saturday On Call

by

10:30 AM: Blizzard begins. 12:45 PM: Receive call from NOC. Two virtual machines down. Am at a hockey game but a coworker investigates. 3:30 PM: Coworker calls me back, informs me that VMware environments have crashed. Since they just don’t do that I posit that a storage issue is occurring. Coworker is a storage administrator, he investigates. 3:45 PM: It is determined that two RAID groups are offline in one of our EMC CX3-80s. I am driving around in my full-time 4WD Jeep, decide to go to work to spare others from having to do it. 5:00 PM: Storage vendor technician tells me that they are trying to get replacement parts which are on the other side of town. ETA …

Read More

+ More Posts +