Dell PERC 5/i Not Caching

I noticed this morning that the RAID containers on my new Dell PowerEdge x9xx servers don’t have their caching options enabled. Now, I understand that write caching is potentially risky, and I understand why Dell ships the PERC 5/i controllers without write caching. However, no read caching? That doesn’t make much sense. While I was mucking around I did some benchmarking of the controllers with different cache settings. I used Red Hat Enterprise Linux 4 Update 4, running on a Dell PowerEdge 2950 with six 146 GB 10K SAS disks. The filesystem is a 20 GB ext3 volume created in LVM, mounted with data=writeback. I used bonnie++ to generate load, which for that tool means sequential writes and reads. Each …

Read More

inodes, or "Why is my disk full, but it isn't?"

System administrators need a solid background in the fundamentals of operating systems. Unfortunately, good defaults and better tools mask the need for that until something breaks. On top of that there is very little recognition of the theory behind how operating systems work and its use in practical, day-to-day system administration. This morning my team was faced with a puzzling problem. A file system which had 34% free space was no longer allowing new files to be created in it, but pre-existing files were still being appended to? How does that work? The junior admin who was the first responder was puzzled. Could it be a permission problem? Maybe a reboot will fix it… Inodes are one of those things …

Read More

Leagues of System Administrators

When I think of system administration I think of the folks I just spent three days with. LOPSA’s Phoenix Sysadmin Days was fun. LOPSA is the “League of Professional System Administrators.” The gathering in Phoenix was their first conference-style gathering since they formed. I was asked to do a presentation on virtualization for it. I hadn’t really done a presentation of that magnitude before, and I have to say my audience was quite forgiving. 🙂 Thanks guys! It was interesting to get a message from my friend Nate, who caught Chris St. Pierre’s blog entry on the topic. Thanks for the press, Chris. I’m hoping that in the next year I can continue to be involved in LOPSA, especially as …

Read More

Veer?

I don’t know what goes through the heads of marketers, but why does Juniper call its customer magazine “Veer?” The magazine just appeared in my mailbox. Oh, I see, it’s printed in light yellow inside the cover, like they’re ashamed of it: It’s more like “3. An erratic swerve into a bridge abutment, ending one’s life.” Personally, I’m not a big fan of Juniper. I went around and around with them over a Netscreen I inherited. They couldn’t put it under support because it hadn’t been registered, and because I didn’t have the card it shipped with (or specifically the apparently unretrievable and unreissueable registration number) I couldn’t register it. After about two weeks of idiocy I said “screw it” …

Read More

What I Instantly Changed in Firefox 2.0

There were three things I instantly changed in Firefox 2.0. In the navigation bar enter “about:config” and: set “browser.tabs.closeButtons” to 3 to emulate Firefox 1.5. set “network.http.pipelining.maxrequests” to something higher to enable more downloads from a single web site. right-click and add a new Boolean preference named “browser.sessionstore.resume_session” with the value “True” to enable session restore. This way closing the browser will not result in losing all your sessions. w00t. I’m glad this thing is out.

Firefox 2.0

It looks like Firefox 2.0 is getting copied to mirrors right now. Booya! A huge thank you to everybody who worked on it. I’ve been using it since it was in alpha and I’m really happy with everything that got added (especially now that all the additions are stable!) 🙂 Two mirrors I know of are at TDS and the University of Wisconsin – Madison.

What an Installer Should Do

Certainly many of you have seen my rants about installers that don’t do a very good job of installing. I must extend props to the EA/Dice guys for doing the right thing with the Battlefield 2142 installer when it encounters a version of Windows it doesn’t know about: It warns you and then lets you continue. Which means that, unlike other retarded installers, I can actually play Battlefield 2142 on my Windows Server 2003 desktop machine. Thank you, whoever built this installer.

ESX Server 3.0.1 & VirtualCenter 2.0.1

A) Amen. I’ve been waiting for these releases. ESX 3.0 and VC 2.0 have some rough edges, and from the looks of the release notes it’s a lot of rough edges. B) The documentation isn’t super clear about how to upgrade. Though, it isn’t like I read the docs real carefully. 🙂 For VirtualCenter it’s just like any other VC upgrade. Download, update everything, don’t delete your database. For ESX Server you should download the “ESX Server 3.0.1 Upgrade Package from earlier releases of ESX Server 3” option. – Get it to your ESX Server (scp, magic, whatever you like) – “tar xlzf 3.0.1-32039-full.tgz” – “cd 32039/” – “esxupdate update” It’ll automatically reboot the ESX Server, so look out. Hey …

Read More

Tivoli Storage Manager on ESX Server 3.0

Okay, I just installed the Tivoli Storage Manager client, version 5.3.4, on my ESX Servers. This is my concise guide to getting it working, because I couldn’t find any documentation describing any of this in any decent order, at least on VMware’s web site. I don’t intend to back my .vmdk files up, just the service console settings and whatnot (I’m doing the VM backups from inside each VM). Obviously that doesn’t stop you from modifying this, I just state it so you know what you’re getting into. 🙂 0. Log into your ESX Server service console via SSH. 1. Install the client. It’ll complain about not having ksh installed. Try using “–nodeps”: rpm -ihv –nodeps TIVsm-API.i386.rpm TIVsm-BA.i386.rpm I don’t …

Read More

Lessons of a VI3 Upgrade

I am just about finished with my VMware Virtual Infrastructure 3 upgrades, and I have made a few observations about the process based on what happened to our clusters. 1. Don’t do an in-place upgrade. Sure, it might work, but it seems unnecessarily complex. What I did was remove a server from a cluster and use it to start a whole new VI3 cluster, including its own VMFS 3 storage. Then I’d scp the disk images from the ESX 2.5.3 cluster to the ESX 3 machine. When I freed enough capacity in the old cluster I’d remove another machine, reinstall it, and tack it into the new VI3 cluster. This plan worked great. It had several benefits, including being able …

Read More