Grow mdadm RAID1 to RAID5

Discussion in 'Other Operating Systems' started by daehenoc, Oct 23, 2020.

  1. daehenoc

    daehenoc Member

    Joined:
    Nov 4, 2005
    Messages:
    2,702
    Location:
    Mt Gravatt E, BNE, QLD
    Hi everyone. I have a two disk RAID1 array with two 5Tb drives:
    Code:
    $ cat /proc/mdstat
    Personalities : [raid1] [linear] [multipath] [raid0] [raid6] [raid5] [raid4] [raid10]
    md0 : active raid1 sdb1[3] sda1[2]
          4883638471 blocks super 1.2 [2/2] [UU]
          bitmap: 0/19 pages [0KB], 131072KB chunk
    I've found a few pages on converting to a RAID5 array:
    https://stevesubuntutweaks.blogspot.com/2015/06/converting-raid1-array-to-raid5-using.html
    https://dowe.io/shrink_mdadm_linux_raid_lvm_logical_volumes/
    https://raid.wiki.kernel.org/index.php/A_guide_to_mdadm#Growing_an_array

    I've definitely got 3 drives all the same size:
    Code:
    $ sudo fdisk -l /dev/sda
    The backup GPT table is not on the end of the device. This problem will be corrected by write.
    Disk /dev/sda: 4.56 TiB, 5000981078016 bytes, 9767541168 sectors
    Disk model: ST5000LM000-2AN1
    Units: sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 4096 bytes
    I/O size (minimum/optimal): 4096 bytes / 4096 bytes
    Disklabel type: gpt
    Disk identifier: 75E6E491-EDCB-4E5A-88AE-61B637923B28
    
    Device     Start        End    Sectors  Size Type
    /dev/sda1   2048 9767541133 9767539086  4.6T Linux LVM
    Code:
    $ sudo fdisk -l /dev/sdb
    The backup GPT table is not on the end of the device. This problem will be corrected by write.
    Disk /dev/sdb: 4.56 TiB, 5000981078016 bytes, 9767541168 sectors
    Disk model: ST5000LM000-2AN1
    Units: sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 4096 bytes
    I/O size (minimum/optimal): 4096 bytes / 4096 bytes
    Disklabel type: gpt
    Disk identifier: 2DB3B486-48E1-4641-8167-6FD83954099C
    
    Device     Start        End    Sectors  Size Type
    /dev/sdb1   2048 9767541133 9767539086  4.6T Linux RAID
    Code:
    $ sudo fdisk -l /dev/sdc
    Disk /dev/sdc: 4.56 TiB, 5000981078016 bytes, 9767541168 sectors
    Disk model: ST5000LM000-2AN1
    Units: sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 4096 bytes
    I/O size (minimum/optimal): 4096 bytes / 4096 bytes
    Disklabel type: gpt
    Disk identifier: 5A98802E-3615-774C-95F9-29B95CB6A7D1
    
    Device     Start        End    Sectors  Size Type
    /dev/sdc1   2048 9767541134 9767539087  4.6T Linux RAID
    I just created the partition on sdc, and I note that the partition on sda is of type Linux LVM (duur, it's been like that for ... two years), and the statement for sda/b about the backup GPT table is in dark red.

    When I do this step:
    Code:
    $ sudo mdadm --grow /dev/md0 --level=5
    mdadm: Cannot convert RAID1 of this size - reduce size to multiple of 4K first.
    I don't know how to change the size of the RAID1 array to a multiple of 4K!

    I've also found this:
    https://documentation.suse.com/sles/15-SP1/html/SLES-all/cha-raid-resize.html

    But I'm unsure if these instructions are correct.

    The /dev/md0 device has space left on it:
    Code:
    $ sudo pvscan
      PV /dev/md0         VG vg_storage      lvm2 [<4.55 TiB / 458.91 GiB free]
    So does this mean I can safely resize /dev/md0 to a multiple of 4k?

    I'd really like to resize it all online, rather than have to copy all the data off to an external drive, and create a RAID5 from scratch... (given the partition type on sda... I may have to!)

    (muse: add sdc, wait for sync, remove sda, change the partition type, then add sda back to grow to RAID5?)
     
  2. OP
    OP
    daehenoc

    daehenoc Member

    Joined:
    Nov 4, 2005
    Messages:
    2,702
    Location:
    Mt Gravatt E, BNE, QLD
    In the meantime, I'm going to:
    1. (zeroth step) TAKE A BACKUP OF THE DATA I CARED ABOUT
    2. fail sda so that my array is degraded running off sdb, (zero hours)
    3. sync sdb to sdc to make a healthy RAID1 array, (12 hours)
    4. fail sdb to have a degraded RAID1 running off sdc, (zero hours!)
    5. fix sda and sdb with new gpt partition tables and correctly typed partitions, (a few minutes)
    6. make a degraded RAID5 array out of sda, sdb and 'missing', (... more than 12 hours? Nope, created instantly, because there's nothing on /dev/md1 to create parity of!)
    7. do the LVM magic to move everything off RAID1 (sdc) onto the RAID5 (sda+sdb+missing), (err... depends on how large the LVs are, but my 4.6Tb RAID1 has about 4Tb of LVs... probably another 12 hours - I didn't count)
    8. once everything is moved, stop the degraded RAID1 and release sdc, (a few minutes)
    9. then add sdc to the RAID5 to make a healthy RAID5! (... more than step 5? yeah, it's taken more than 18 hours so far...)
    Job done.
     
    Last edited: Oct 27, 2020
  3. demiurge3141

    demiurge3141 Member

    Joined:
    Aug 19, 2005
    Messages:
    2,192
    Location:
    Melbourne 3073
    Much easier to just backup, create array with 3 drives and restore from backup.
     
  4. OP
    OP
    daehenoc

    daehenoc Member

    Joined:
    Nov 4, 2005
    Messages:
    2,702
    Location:
    Mt Gravatt E, BNE, QLD
    If I had a spare HDD to copy everything to, and didn't mind the downtime of copying everything back off that HDD to the RAID array, absolutely!

    With my approach above, I get to keep everything online, and SWMBO and offspring don't have a go at me for the media being unavailable :p
     
  5. Primüs

    Primüs Member

    Joined:
    Apr 1, 2003
    Messages:
    3,418
    Location:
    CFS
    Can't you mount the media...on the backup drive? So its still available while new stuff creating/copying?

    I just find your approach a largely risky idea - if you don't have the media already backed up then there's a lot of steps where 'something' might go wrong, and it 'may' end up with an unrecoverable array.
     
  6. OP
    OP
    daehenoc

    daehenoc Member

    Joined:
    Nov 4, 2005
    Messages:
    2,702
    Location:
    Mt Gravatt E, BNE, QLD
    Haha, yes, you are correct, there are several places here where it could have all gone wrong - I've added the zeroth step of taking a backup of the critical data (helpfully renumbered to step 1 by the list code), so that if something did go wrong, I had a backup of the important data. The thing I didn't state above was that I wanted to keep the LVs online during all this shuffling around the place, hence the approach.

    All the LVs are now in the correct place, I've stopped /dev/md0 and released /dev/sdc, wrote zeros to the first couple Mb of the drive, created a new gpt partition table, created a new partition for the full drive of type 'Linux RAID', and have added that into /dev/md1 and the re-sync is chugging along :)

    Man the block-based movement of pvmove is slow!
     

Share This Page

Advertisement: