Teleidoscope

Fri, Dec 05, 2008

Hard Drive Hell

Posted at 3:56 pm MST to Technology

Two of the new hard drives in quadriga (my big server that I just upgraded with new drives) died over the long weekend. I've been working on reconfiguring things and restoring the data from the 1 Tbyte backup drive. And learning a lot about RAIDs and LVMs. sigh.

I'll have to arrange to get them returned and replaced. And then reconfigure everything again to a 5-drive RAID 10+spare instead of the 3-drive RAID 1+spare I'm using at the moment.

I also need to make sure my ISP's spam filter doesn't block the status emails from the server: after things failed, I discovered that the server had been trying to send me emails warning that the drives were dying.

Yesterday, one of our servers at the office failed. I went over to the office to see if I could get it back up, and found another drive failure.

Shawn is working on recovering data from the drives of the failing system while I am reformatting and running bad-blocks checks on the 500 Gig drives I took out of my server, so they can be used at the office.

One of those drives was definitely getting bad block errors before I got the new drives, but all 4 of them reformatted. I hope the e2fsck -c -f commands that I am running will either identify the bad drives or make it usable if the bad blocks are localized enough.

permanent link || trackback || 0 comments || Add a comment