Archive for August, 2009

links for 2009-08-29 »

Popularity: 1% [?]

ALUA Rocks »

Ack, I was seconds away from writing the same post as Nick Triantos on Upgrading from non-ALUA to ALUA, as I’ve spent the afternoon so far doing this and documenting the process.

The process works great, and it didn’t disrupt my clusters at all. The only things I did differently were:

1. I issued the “/usr/sbin/esxcli nmp satp setdefaultpsp –psp VMW_PSP_RR –satp VMW_SATP_ALUA” command on all the hosts prior to rebooting them.

2. I was able to combine this work with the application of the ESX 4.0 patch for the RAID controller cache problems, so Update Manager took care of all the rebooting for me. It does feel good to be lazy some days.

If you have a NetApp array and are running ESX 4.0 I wholeheartedly suggest enabling ALUA. As Vaughn Stewart puts it, it really is Plug & Play, and it removes all the manual tuning & balancing you needed to do to spread I/O across all your HBAs while making sure you’re talking directly to the right storage controllers.

Popularity: 2% [?]

links for 2009-08-21 »

Popularity: 1% [?]

Brent »

Dear Vikings,

I hope he waffles about retiring and pisses you guys off as much as he did with the Packers & Jets. Congratulations on signing a complete jackass.

…Me

I think this t-shirt sums it up nicely:

ex_always_remember

(my apologies to my non-U.S. readers for rambling on about stuff like this, but I wouldn’t be surprised if even you knew the ridiculous story)

Popularity: 1% [?]

VMware & Linux Hot-Add Disks »

Did you know you can add new virtual hard disks to a running Linux VMware VM, and they can be visible to the OS without a reboot?

First, add a new hard disk the way you’d normally do it (Edit Settings -> etc.). Add it to the existing virtual SCSI adapter (don’t create a new one).

Then, use a script like Kurt Garloff’s rescan-scsi-bus.sh to pick up the new device.

If you’re using the Linux Logical Volume Manager you can partition the new device, run pvcreate, add it to a volume group, and grow a filesystem with ext2online/resize2fs (RHEL 4/5), all without the end users noticing.

Just don’t forget to align the partitions on the new device before you use it. Misaligned I/O is a serious performance drag.

Popularity: 2% [?]

It’s Been Four Years? »

Holy crap, I totally missed the birthday of this blog this year. It’s four, or rather, four + 2 days (8/8/2005 was when it was born). Some stats:

1,231 posts = 0.84 posts a day.

1,628 comments = 1.114 comments a day.

203,360 spam comments = 139.19 spam a day.

Yeah, right on track, even with a lot of the posts being from Delicious. Those posts often have remarks from me in them, after all. :-) I’m aiming for 0.75 posts a day. Daily would be nice but I sometimes suffer from ranter’s block. :-)

Anyhow, thanks for reading! When I started this thing I couldn’t imagine who would read it, but I hope my snide commentary and massive ego is entertaining to at least a few of you. :-) Thanks again for keeping me honest, prodding me here and there, and defending my honor on occasion. Y’all rock!

Popularity: 2% [?]

Tell Your Purchasing People: IPv6, VMware »

In the world of system administration there aren’t too many things that are black & white. Everything is a shade of gray where admins solve their own problems however they need to, bending to the local desires & needs of their users or management. Everybody is right, nobody is wrong. I’ve come to realize that, and it’s no big deal.

I am convinced, however, that if your organization does not have “fully implemented IPv6 support” and “full support under VMware virtual infrastructure” as requirements for purchasing any new hardware, software, or services, you’re doing it wrong[1][2].

Let’s assume that anything you’re buying now will last 5+ years. In 5+ years we will be out of IPv4 address space[3]. And it goes without saying that you’ll want to run things in your virtual environments, right?  So don’t let your organization buy products that are just going to make life tougher two years from now — let your management know now that things need to change. Even if IPv6 isn’t currently on your organization’s to-do list, it will be soon, whether you like it or not. Get on top of it.

————————————

[1] Not desirables, not plusses or bonuses, but a full functional requirement for support that knocks a candidate out of the running in an evaluation if they don’t meet the criteria. You might also be tempted to say “well, if it’s on the vendor’s roadmap…” — don’t. IPv6 has been around for years, and if your vendor doesn’t have it implemented right now it isn’t a good sign. Besides, during purchasing exercises I always suggest that folks treat roadmaps as vaporware and assume that none of the features listed on them will ever ship.

[2] Yes, my organization is doing it wrong, too. On both counts. It’s getting fixed, slowly, but we should have started fixing it three years ago. Which is why I’m writing this post. :-)

[3] ARIN/LACNIC/APNIC have issued statements that we’ll be out of IPv4 space in 2010, and Tony Hain has a report on the current state of the space. Geoff Huston has a daily report and prediction on his web site, too. Hurricane Electric also has a big page of exhaustion stats. None of them are showing anything good.

Popularity: 2% [?]

…now with 100% more IPv6 »

The web server I run this blog’s virtual host from has been IPv6-enabled for about three years. On Monday night I asked myself why I’d never given the blog an AAAA record. So I did.

I just looked at the logs, out of curiosity, to see how many of my readers are IPv6-enabled. It’s painful. Discounting myself, 19 unique visitors out of 1683, 1.13%, came in via IPv6.

If you aren’t thinking about IPv6 you should start. Enabling IPv6 really isn’t a big problem, by itself, as most ISPs can handle requests like that now. If you’re anything like me the problems that will vex you are the little ones: death by a thousand paper cuts. It’ll be the fact that your hosts.allow files need new stuff in them. Or the guy who does DNS doesn’t know what a quad-A record is. Or your web log analyzer script that’s worked flawlessly for a decade now barfs. Each problem, by itself, takes a week or two to fix, and now it’s three years later.

I’ve been running a couple of hosts on their own little VLAN, separate from everything else so that I can figure out what we need to fix at an OS level. Because they’ve been separate I could take my time, fixing things slowly so that as we get ready to turn IPv6 on for more of my organization we’re ready. It’s worked out really well so far, and I’d recommend it to anybody who isn’t testing IPv6 already. Get on it, get an IPv6 allocation and build yourself a small DMZ testing area. Put your desktop in it, too. It won’t be long until you be forced to implement IPv6, and this way you’ll be prepared.

Popularity: 1% [?]