12 to 16 drives, RAID6 or 50?

Discussion in 'Storage & Backup' started by kripz, Dec 15, 2011.

  1. kripz

    kripz Member

    Joined:
    Sep 29, 2004
    Messages:
    2,834
    Location:
    Near Frankston
    I dont like the rebuild times on such a big RAID6 so i want to split them to two RAID5 volumes but i like having one big volume so i plan on putting those two in RAID0. I've also read on here that lots of drives results in higher chance of failure.

    Is this recommended? Mathematically which is safer? What would you do?
     
  2. wardenm

    wardenm Member

    Joined:
    Jul 11, 2002
    Messages:
    348
    Location:
    Brisbane
    I'd still go with RAID6. RAID6 should be more reliable since you can lose any 2 drives.

    With RAID50 you can lose any 1 drive, but if a second drive in the same RAID5 set fails during the rebuild, the whole array is gone.

    I don't know if controllers actually do this, but another possible benefit is that with RAID6 you have enough parity to properly correct errors. If you get bad data from a drive in RAID5 the controller can't tell the difference between the real data and the damaged data. With RAID6 the controller could re-create 3 copies and then know to throw away the one that doesn't match the other two.
     
  3. Axe

    Axe Member

    Joined:
    May 7, 2002
    Messages:
    357
    Location:
    brisbane
    if 2 drives go down in one of the 50 arrays, you lose the whole lot.
    ie, 2 arrays of 8 drives in a 0 stripe (5+0), one of those arrays fail, lose the lot

    not the same in the raid 6 tho, lose 2 and you can still rebuild.

    personally i'd go with raid 6 :)
     
  4. bobrandom

    bobrandom Member

    Joined:
    Oct 9, 2002
    Messages:
    988
    RAIDZ3 get a look in?
     
  5. terrastrife

    terrastrife Member

    Joined:
    Jun 2, 2006
    Messages:
    18,818
    Location:
    ADL/SA The Monopoly State
    there is no real difference in rebuild times on raid 5 or 6, youre basically limited by drive performance... unless your controller is ASS.

    i would make 3 separate raid5's or something.
     
  6. wardenm

    wardenm Member

    Joined:
    Jul 11, 2002
    Messages:
    348
    Location:
    Brisbane
    The RAID5's would only be half the size of the RAID6, so the rebuild should be quicker if you only lose a single drive.
     
  7. power

    power Member

    Joined:
    Apr 20, 2002
    Messages:
    67,347
    Location:
    brisbane
    Considered raid10? Can handle multiple simultaneous drive failures.
     
  8. noobmastery

    noobmastery Member

    Joined:
    Dec 23, 2008
    Messages:
    888
    Location:
    Sydney, 2111
    If it were me i'd put the drives in 2 RAID6 sets, that way you can't lose all your data in one hit which is always a good thing.
    RAID6 is safer than RAID50, no question.
     
  9. xenorg

    xenorg Member

    Joined:
    Feb 14, 2003
    Messages:
    384
    Location:
    Cranbourne North, Vic
    Yes, but not any 2 drives. If 2 go from the same mirror, you're screwed.
     
  10. damox

    damox Member

    Joined:
    Aug 22, 2011
    Messages:
    304
    You could loose any two drives as long as the drives are not the mirrored drive.

    Say you have;
    1A-1B-1C-1D
    2A-2B-2C-2D

    You could loose any combination of drives expect 1A && 2A or 1B && 2B ...

    If you loose two from the same mirror 1A+1B you can recover from 2A-2B you could even loose the entire A mirror and recover from B.

    ---

    I personally have a 20 disk setup. 2x raidz2

    It can loose any two from a single raidz2 (Raid6) and up to 4 (2 from each raidz2)

    The recovery time is limited by the performance of the drive to be recovered. A single drive can only write so fast, you wont see any gains in recovery time over different raid levels.

    Code:
    config:
    
    	NAME          STATE     READ WRITE CKSUM
    	tank          ONLINE       0     0     0
    	  raidz2-0    ONLINE       0     0     0
    	    c10t12d0  ONLINE       0     0     0
    	    c10t11d0  ONLINE       0     0     0
    	    c10t13d0  ONLINE       0     0     0
    	    c10t14d0  ONLINE       0     0     0
    	    c10t39d0  ONLINE       0     0     0
    	    c10t40d0  ONLINE       0     0     0
    	    c10t41d0  ONLINE       0     0     0
    	    c10t42d0  ONLINE       0     0     0
    	    c7t17d0   ONLINE       0     0     0
    	    c7t18d0   ONLINE       0     0     0
    	  raidz2-1    ONLINE       0     0     0
    	    c7t19d0   ONLINE       0     0     0
    	    c7t20d0   ONLINE       0     0     0
    	    c8t38d0   ONLINE       0     0     0
    	    c8t39d0   ONLINE       0     0     0
    	    c7t35d0   ONLINE       0     0     0
    	    c7t34d0   ONLINE       0     0     0
    	    c8t29d0   ONLINE       0     0     0
    	    c8t30d0   ONLINE       0     0     0
    	    c8t31d0   ONLINE       0     0     0
    	    c8t32d0   ONLINE       0     0     0
    
     
  11. cvidler

    cvidler Member

    Joined:
    Jun 29, 2001
    Messages:
    15,457
    Location:
    Canberra
    keep RAID6 + backup.

    No worry about data loss, only rebuild/restore time.

    Seriously, everyone needs to get it that RAID can not prevent data loss like a backup, it's just more convenient if the drive failure is within the capabilities of the RAID set.
     
  12. TRAG!C

    TRAG!C Member

    Joined:
    Apr 18, 2007
    Messages:
    1,755
    Location:
    Sydney
    What are you using the volume for? have you looked at unraid as an option?
     
  13. terrastrife

    terrastrife Member

    Joined:
    Jun 2, 2006
    Messages:
    18,818
    Location:
    ADL/SA The Monopoly State
    to rebuild you need to read every disk sequentially at once raid 5 or 6, hence, it depends on the performance of your raid card

    rebuilding from 10 disks or 3 disks should be very very similar in times, if the disks are the same capacity.
     
  14. MetalPhreak

    MetalPhreak Member

    Joined:
    Apr 8, 2007
    Messages:
    1,763
    Location:
    Perth
    When you replace one drive, you only have to replace the data on that one drive.... regardless of what raid type you use. As long as there is no bottleneck simultaneously reading from the other drives and calculating any necessary parity info, the rebuild time will simple be how long it takes to fill up a drive of that size :lol: 2TB drives take a while even at 80MB/s average speed :p


    I believe ZFS only writes data to parts where actual files are stored, so if your array isn't full the rebuild time will be quicker. Only possible because its both the filesystem and the raid software.
     
  15. rowan194

    rowan194 Member

    Joined:
    Jan 5, 2009
    Messages:
    2,047
    The downside is that because it's stepping through the entire file system, rather than just a sequential read/write of sectors, resilvering on a nearly full file system, or one with lots of small files, may actually take longer than a straight clone.
     
  16. xenorg

    xenorg Member

    Joined:
    Feb 14, 2003
    Messages:
    384
    Location:
    Cranbourne North, Vic
    I'm pretty sure thats what i said....not any 2 drives....:confused:


    Thats right. Fragmentation also has a huge effect on the resilver speed. On the upside...if you are resilvering to a drive that was pulled from the existing vdev, it'll only resilver the missing data. eg. if you offlined, then onlined a drive, it would only resilver the changes that happened between the time it was offlined, and the time it was onlined.
     
    Last edited: Dec 16, 2011
  17. OP
    OP
    kripz

    kripz Member

    Joined:
    Sep 29, 2004
    Messages:
    2,834
    Location:
    Near Frankston
    Weird, with mdadm, more drives = longer it takes to rebuild for me.
     
  18. terrastrife

    terrastrife Member

    Joined:
    Jun 2, 2006
    Messages:
    18,818
    Location:
    ADL/SA The Monopoly State
    are you at your bus limit?

    ie my unraid has a total of about 400MB/sec of throughput over the various sata controllers.

    divide by 15 hdds and of course overall speed will be slow.

    if you cannot read at full speed from every disk that is required for parity AT THE SAME TIME, its gonna be slow.
     
  19. OP
    OP
    kripz

    kripz Member

    Joined:
    Sep 29, 2004
    Messages:
    2,834
    Location:
    Near Frankston
    Shouldnt be?

    6 drives on the onboard, 3x pci-e 1x 2 port controllers.
     
  20. terrastrife

    terrastrife Member

    Joined:
    Jun 2, 2006
    Messages:
    18,818
    Location:
    ADL/SA The Monopoly State
    well most onboard tops around 500MB/sec and, no idea what the throughput of cheap 2 port cards are.

    so if the limitation isnt the 2 port cards at most you will get around 80MB/sec rebuilds.
     

Share This Page

Advertisement: