Replacing a Live System RAID On The Fly

Well, this is one of those crazy fun things that I had to attempt. My old system drive consisted of two 80 GB IDE drives in a mirror configuration. Pretty standard, but they were getting sluggish and I happen to have a few 320GB SATA drives doing nothing.

This is not a tutorial. This is a step by step account of a process I took to replace my disks; This is to be taken only as a suggestion as your configuration and mileage may vary. There may also be a better way of doing this.

So, lo and behold, I plugged them in to my eSATA ports and started rolling over!  For the sake of this exercise the original drives will be sda, sdb. The new drives will be sdc and sdd. The raid device is md0.

  1. BACKUP YOUR DATA
  2. Partition the new drives using your favorite partition manager, create a matching partition on both sdc and sdd to be near the maximum.
  3. Force a failure and remove the first drive:
    mdadm -f /dev/md0 /dev/sda1
    mdadm -r /dev/md0 /dev/sda1
  4. Add the first of the new drives:
    mdadm –add /dev/md0 /dev/sdc1
    wait for it to rebuild/resync.  You can check the status by typing ‘cat /proc/mdstat’ from a command line.  This took about 45 minutes for me.
  5. Force a failure and remove the second drive:
    mdadm -f /dev/md0 /dev/sdb1
    mdadm -r /dev/md0 /dev/sdb1
  6. Add the second of the new drives:
    mdadm –add /dev/md0 /dev/sdd1
    and wait for it to rebuild/resync. In my case the two new drives are considerably faster so this sync only took about 25 minutes.
  7. Your system is now running on your brand new mirror, but the mirror is still only as large as the original disks.  So, you need to resize the md0 device:
    mdadm –grow /dev/md0 –size=max
    Then wait for it to rebuild/resync to the new size.  This part took 2 hours.
  8. After that is done you need to grow your filesystem.  Depending on which filesystem you’re using, there are tools for it.  I’m using ext4; so I used the standard ext resize utility:
    resize2fs /dev/md0
    And it went into an online resize mode and worked fine.  Other utilities are: xfs_growfs and resize_reiserfs – if you’re using another filesystem or an LVM you may want to read the friendly manual.
  9. And it’s done!

If you’re using a production server and need to upgrade disks this is a pretty smooth method of doing it. While everything remains online for the long portions of the process things are a bit slower but not painfully slow.  The resync processes play nice and only take available power.  Idle servers rebuild faster and your times may vary on the steps.

After doing this, you may also want to make sure GRUB or your boot loader is in properly configured on the new drives.

And then, enjoy your new faster and bigger disks!

Published by Darryl Clarke

random text goes here, i guess