Error: I'm afraid this is the first I've heard of a "trackback" flavoured Blosxom. Try dropping the "/+trackback" bit from the end of the URL.

Sun, 23 Nov 2008

New Drive Setup

I had forgotten how well-designed the interior of the server case is. There are two drive cages for 3.5 inch drives that came out smoothly so I could unmount the old drives from them and install the new drives.

I think I'm beginning to understand what happened to the old drives, and why I had problems booting after the update.

Three of the old drives had similar partitioning, but one had an additional partiton for swap space. I am quite certain I would have put swap on the first drive (sda) when I installed, but after the upgrade from FC7 to FC8 and 9, the drive with swap space was sdb. The boot failed until I reinstalled grub into the new sda.

Then one of the drives started to fail. I suspect that may have been the old sda drive: anaconda may have moved it to sdb because it was responding more slowly to the configuration tool.

When I first powered up the server with the new drives, it could only see 4 of them: 1 in the upper drive cage and three in the lower drive cage. When I rechecked the power connections, it turned out that the plastic shroud around one of the power connectors I was using in the lower drive cage was broken, so the leads weren't making a solid connection. Switching to a different power connector brought all of the drives up.

It is possible the drive was failing because its power had gone flakey. We'll need to test the old drives before we install them into the office servers: it is possible they are all usable provided they have clean power.

The old configuration was 3 drives in the lower chamber drive cage, using 3 SATA connectors on a single power cable, with one hard drive and a DVD-RAM drive in the upper chamber on a different power cable.

I had a power-to-dual-SATA power splitter (I think it came with the motherboard, actually) that is long enough to give me power for two additional drives in the lower chamber. So the new configuration has 4 drives in the lower drive cage (filling it) using two different power cables, and 1 in the upper chamber in the 3.5 inch cage, with the DVD still in the 5.5 inch cage. The motherboard supports 6 SATAs, so with 5 DHs and the DVD, that is now maxed out.

In researching Linux soft RAID, I learned that you need to wait for the RAID to fully sync after the first boot before you do anything else.

Then

grub-install /dev/md0
should prep all of the physical drives to be fully bootable in case sda fails.

The new disk configuration is

/dev/md0 RAID1 /boot (/dev/sd*1) 100 meg each
swap (/dev/sd*2) 1 gig each, not raided
/dev/md1 RAID10 LVMGroup00 (/dev/sd*3)

/dev/sde1 and /dev/sde3 are the RAID hot spares

LVG00:
root  /      10%
home  /home  20%
var   /var   20%
data  /data  50%

Once everything is stable I give low priority to the swap on /dev/sde2 so it won't be accessed except in emergencies. I set the other 4 swap spaces to equal priority:

# /dev/sda2
UUID= pri=1            swap    sw              0       0
# /dev/sdb2
UUID= pri=1            swap    sw              0       0
# /dev/sdc2
UUID= pri=1            swap    sw              0       0
# /dev/sdd2
UUID= pri=1            swap    sw              0       0
# /dev/sde2
UUID= pri=2            swap    sw              0       0

The I set all the drives to spin down after 15 minutes of inactivity to save power and wear on the drives. The spare should be hardly ever be spun up.

Note: use

ls -l /dev/disk/by-uuid
to determine which partition goes with which UUID in the fstab.

# hdparm -S 180 /dev/[sh]d[a-z]
and add the command to the end of /etc/rc.d/rc.local so it will happen on every reboot.

And once I get the OS updated and configured I need to make sure any RAID or drive failure notices (from SMARTD) get forwarded to my regular email address.

I also need to create a couple of mount points for external drives: one for the firewire Tbyte drive and at least one for a thumb drive or other miscellaneous drive.

#