I was on my way back from lunch at Thai Chilli’s to the SiCortex office in the Mill in Maynard, MA. On the way across the bridge over the Assabet I noticed ice in the water, flowing downstream.
Here’s another just coming out from under the bridge.
Author: stewart
Getting at old RAID sets
After I got Ubuntu 8.10 working with MD RAID Sata drives, I wanted to move my old data onto the new drives. As I explained previously, the system would not boot with the old drives plugged into the primary IDE controller and the CD and extra drive plugged into the secondary IDE. It would boot with the old raid set plugged into the secondary IDE and the primary left unused.
Now, running on the Sata drives, I wanted to access the old drives, which were set up as a number of MD Raid-1 sets. After the break I’ll explain step by step how to find and mount the old raid sets.
Continue reading “Getting at old RAID sets”
The Great Sata Raid
I ran out of space on my home Linux machine. I suppose it started out life as a 350 MHz Pentium II in about 1998, but I think everything has been replaced since then. The current system is a dual processor (two cores total) Athlon 1.8 GHz machine with 1 GB memory. It has a Tyan Tiger MPX motherboard in an Antec Sonata case and a Matrox G550 graphics card. I run Gentoo Linux on it.
The disk system is software RAID-1 (mirrored disks) on a pair of 250 GB drives. These are on the primary IDE port. A third 250 GB drive and a DVD burner are on the secondary IDE.
I started reading in an accumulated pile of Sony MicroMV tapes (mpeg2) from the family camcorder because I am nervous that the camcorder is getting flaky. On tape 20, the disks filled up. The single drive is mostly full of other video (like 150 odd episodes of Good Eats pulled off the Replay TV with DVArchive.
All this is merely setup about why I have been trying to upgrade the storage. My colleague Jeff Darcy solved a similar problem with a QNAP NAS box, but I am kind of cheap and stubborn. Instead, I got a Syba SD-SATA2-2E2I SATA card and a pair of 1 TB Western Digital drives from Newegg. Oh, and I decided to give Ubuntu a try.
Step 1 – Get the controller to work
The first controller was DOA. This is always a little frightening, did I break it? But not this time. I know how to handle the cards, use good anti-static protocols, and so forth. This card, when plugged into any PCI slot, prevented the machine from booting at all. No beeps, no BIOS. Unplug, and it works. The Syva card is a little weird in that it has PCI slot cutouts for both 5 volt and 3.3 volt slots, but according to the Internet, it really only works in 5 volt slots. Fair enough, but dead. Newegg sent an RMA, and, eventually, a new card that behaved much better.
The next huge issue is BIOS extensions. I guess this is a good idea, so that random new controllers can come with the software to run them. The 2E2I comes with a fake RAID BIOS that is just a Bad Idea. I planned to not use it, and instead to use Linux MD software RAID.
A digression about storage policy.
I’d basically like to save everything forever. I still have files from my grad school days at Stanford. We had a PDP-11/34 running V7 Unix. Since then I’ve been through four companies and a variety of mail system. I have nested archives, typically underneath trees of directories where each level is named “old stuff” or the equivalent. During my entire career, this stuff has been too large to easily put on off-line media, be it floppies, tapes, CDs, ZIP drives, or whatever. In any case, even if you believe the media are stable, and I don’t, the devices to read old media quickly become unavailable. I think the only real solution is to keep rolling your data forward onto new online devices.
Next. I am acutely aware of single points of failure. I am not really happy with only one of something, and I am not really happy with something that I can’t fix. With this home machine, everything is available commodity parts. I’ve had the memory break, the fans break, the power supply break, the graphics card break, and these are petty annoyances, but my data isn’t really at risk. I can fix the hardware, or if necessary, put the drives into another PC. Appliance NAS boxes make me nervous. It isn’t that the drives are flaky or the software buggy, it is that the rest of the hardware isn’t repairable. It isn’t commodity. If something breaks, you have to hope the company is still in business and deal with slow and expensive repairs. Jeff’s QNAP at least can copy itself to another QNAP, but then you really need two, and they aren’t inexpensive.
So I have a RAID system built with commodity parts, and I back up that to another system that is similar. I’d kind of like the backup to be offsite. Cloud storage for offsite makes sense, but 250 GB to 1 TB is sort of unwieldy. One of the other reasons I have for choosing the Syba card is that it has eSata ports so I can occasionally make copies on 1 T external drives and leave them, say, at Grandma’s house.
Back to the hardware setup.
Since I don’t want the hardware RAID, I didn’t set it up. I learned from the Web that Ubuntu 8.10 can install onto RAID, but you have to use the Alternate installer. I followed directions, ignored installer complaints about an inability to reread partition tables, and got through the install…and it wouldn’t boot.
In fact, in the BIOS boot sequence screen, the Sata drives wouldn’t show up at all. According to the Internet, you can get around this by declaring one of the drives to be a concatenated drive with one sub-drive, but that did not work. I tried re-installing, thinking that setting up the concatenated drive scrambled grub, but that didn’t work. By the way, the Ubuntu installer should let you repeat individual steps, like installing grub, without wanting to repartition your drives. Just a thought.
Back to the web. Apparently, the controller has two BIOS flash images. The RAID version is standard, but you can reflash the card with the other image, with basically lets you use the controller in JBOD mode. The actual Sata chip is a Silicon Image 3124, and the flash images and installers are on their website. And only work on DOS or Windows, don’t get me started on that.
I plugged the card into an old Sony Vaio desktop we have for games, and downloaded the installer for Windows, and it didn’t work. It could not find the card at all. The Windows XP device manager knew about it, but the flash installer didn’t. I guessed that the XP driver needed to be installed, and the “check for new hardware” actually worked without requiring a reboot. That never happens. After the driver was running, the flash installer worked ok.
Back to my system, now the disks show up in the boot menu, but won’t boot. Time for another install. Same warning messages, same lack of booting. This time I applied a Principle of Debugging: Do not ignore errors you don’t understand. I send the literal error text to Google, and discovered it was a patch to the Ubuntu installer to prevent it from failing when it encountered remnants of old RAID metadata. My guess is that this stuff was left laying around on the disks by the RAID version of the controller firmware. After more web searching, I found that the dmraid Linux utility, among other things, has the power to delete such things. I got to dmraid by using the install CD to give me a shell on my installed not not working root file system. I erased the RAID junk, but the system still would not boot.
Next I guessed that this was grub not being correctly installed, quite, so I installed it manually, and wrong, so that now I have /boot/boot/<stuff> for some reason, but the system can now boot.
Copying old data
Next, I wanted to copy data from my old raid sets onto the new 960 gigabyte /home partition of the new ones. All this time, I had the primary IDE controller cable unplugged, so that there would be no way for the Ubuntu installer to erase my data. I plugged it in again, and the Sata controller disappeared! The old drives were now visible, and bootable, but once booted, lspci couldn’t see the new controller. Unplug the primary IDE cable and it came back. Weird. The CD drive and my other 250 GB drive on the secondary IDE were working with the Sata drives, but not the primary IDE.
Finally, I just unplugged the secondary, and plugged my old RAID drives into the secondary IDE connector on the motherboard. That worked, and now I have the new terabyte RAID system and my old data at the same time. As we speak, I am rsync’ing the contents onto the new drives.
It should go without saying. This is all way too complicated. And this isn’t really my primary computer anymore. Mostly I use a 17″ Macbook Pro. That I back up with an Apple Time Capsule with a 1 Terabyte drive. Yes, it doesn’t really meet my goals of repairability, but it just works.
Garlic!
Anything worth doing is worth overdoing.
The garlic bagel is a food object of tremendous power. However, my wife has been pointing out just how unhealthy white bread bagels are, so I have, sadly, cut way back.
It doesn’t have to be this way. It turns out that my local shop, a branch of Finagle A Bagel, will make bagels to order. I called up at 6:15 this morning and a dozen multi grain with garlic bagels were ready at 7.
I am a happy boy today. I am safe from vampires, but my colleagues are advised to keep back.
Making laws
As a followup to the post about delivering the tax code in source form, I think that congress should be required to use a revision control system for bills. Such a system could help us figure out who added what provisions. You know how when a provision benefiting some group is added and noone seems to know how or when it got in the final bill?
I am most familiar with cvs and svn, but there are many others. All of these systems track revisions and permit the user to go back and check out any previous version. Good ones can show you the changes between any two versions, … and who made those changes. Svn has a feature called “svn blame” which produces a listing of a file with every line of text annotated by its author. So you want to know who added the earmark giving a billion dollars to Halliburton? Svn blame!
Tax ideas
Is anyone not offended by the complexity of tax forms?
The tax code is Congress’ fault. Congress is our fault. Therefore the tax code is our fault. We should all choose to elect a congress to fix it, but in the meantime, here are some modest proposals.
- Congressmen must do their own taxes. On paper.
- All congressmen will be audited. Their mistakes get put on a web site.
- Tax law must be delivered in source code form
And yes, I was recently trying to figure out whether I owe any estimated taxes.
(Ideas from conversations with Win.)
The Bully Pulpit
Here’s my first political post.
I think President Elect Obama is missing a bet on the Blagojevich affair. After the Rev. White incident, Obama took the opportunity to present a very good speech on race relations. He didn’t hide hide his connection to White, or excuse the man. He took the moment of public attention and turned it to good purpose.
In the case of Blagojevich, Obama has merely tried to distance himself from it. I think that is the wrong approach. Obama has an opportunity here to present a very effective discussion of service, responsibility, integrity, and character.
Drain Grates
I thought these were all removed in the previous century!
The problem with these is that the slots run along the riding direction, and are just wide enough to catch a road bike tire. If your front wheel is unlucky, it will wedge into a slot and stop dead. You will not stop, but according to Newton’s first law, continue moving forward over the handlebars and onto the street.
I noticed this one on the driveway in front of the Wayland Middle School.
If the thing is square, at least it could be put in so the slots run across the road instead of along it.
Update, (March 2009)
I could be in serious trouble now.
http://www.boingboing.net/2009/03/03/manchester-man-arres.html
Is a story about a man in the UK arrested and held for two days as a terrorist for photographing a sewer grate.
Leaf Peeping
This is New England. It’s fall. That means it is time for the annual trek to New Hampshire to look at the trees.
Or you can just step out the front door.
Here’s the Great Wall of Flame out back:
Here’s the Lesser Wall of Flame out front:
Here’s the Maple we put in outside the living room window:
And here are the three Maples in front. Someday these will hold up some hammocks. Don’t expect me to get up after that.
This area has seasons. I enjoy them.
Bicycle commuting notes – Sidewalks
In May 2007, I started occasional commuting from my home in Wayland, MA to the offices of SiCortex in Maynard. This is just about nine miles each way. In 2007 I ran up about 1000 miles on my 30 year old Nishiki Olympic. I think I am on pace to do a bit better in 2008.
I’m not very fast. When I started it took almost an hour to get to work. On the way home the read derailleur cable broke, leaving me stuck in 9th gear (of 10), so it also took a long time to get home. My colleague David shook his head at the condition of my bike, with its steel rims and puncture proof (heavy) tires, and talked me into some upgrades. With new alloy rims and new tires (and getting in better shape) I can now get to work in a little under 40 minutes and home in about 35. The elevation gain is near zero, since I am on the Sudbury river in Wayland, and the office is on the Assabet river in Maynard. The Sudbury and Assabet merge in Maynard. If anything, Maynard must be at a lower elevation, but something about the topography makes it seem like more uphill going in, and more downhill coming back. There is definitely a high point in the middle, at Sudbury Center.
It was really this commuting that pushed me into starting this blog. I would spend the trip composing elaborate letters to the editor of the local paper, often about the poor conditions for cycling. This week I finally took a bunch of photos along my route, and I am ready to start ranting.
Continue reading “Bicycle commuting notes – Sidewalks”