Replacing a failed drive in a RAIDZ2 array – Mixing 4k sector drives issues?

Discussion in 'Storage & Backup' started by frozencrystal, Oct 2, 2011.

  1. frozencrystal

    frozencrystal Member

    Joined:
    Feb 12, 2009
    Messages:
    30
    Background Info
    Running FreeBSD 8.2-STABLE
    8x Hitachi 2TB 5K3000 connected via AOC-USAS2-L8i
    Drives given a GEOM label, then whole drive given to ZFS

    Story
    So I’ve had two drives in my RAIDZ2 array fail simultaneously, and after panicking for a while I managed to get my array functioning in a degraded state, backed up the most important data to another place and then shut it down. I have RMA’d the two drives, but won’t be able to get replacement drives for another 2-4 weeks.

    Now I’d like the use of my fileserver again whilst waiting for the RMA'd drives, but I’m extremely hesitant to run it without any redundancy whatsoever. As the 2Tb Hitachis are not in stock anywhere, I’ve come up with the following alternatives and would like your ideas on which would be the best thing to do:

    1. Buy a new (4k sector) 2tb drive and use it to replace one of the failed drives.

    • Potential performance issues as it’s a 4k drive mixed in with 512 sector drives.
    • Inability to use the gnop trick to align the drive because that might reduce the available capacity of the drive below that of the other drives? Perhaps this could be counterbalanced by not assigning the replacement drive a GEOM label which would give me an extra 512 bytes of leeway?
    2. Buy a 3tb Hitachi drive (512 byte sectors) and use that to replace one of the failed drives.
    • Wasted $$$
    • Not in stock anywhere locally, so making option 1 work would be preferable.

    Any input you could provide would be much appreciated, or if there's a better 3rd alternative i'd love to hear it!
     
  2. rowan194

    rowan194 Member

    Joined:
    Jan 5, 2009
    Messages:
    2,046
    I find that hard to believe... where have you looked so far? I just checked a couple of MSY stores and both are showing stock of the 5400RPM 2TB model. Perhaps someone from here may be able to give you tips on where to find them.

    What happened to the drives exactly? Did you test them after they were dropped from the array?
     
  3. terrastrife

    terrastrife Member

    Joined:
    Jun 2, 2006
    Messages:
    18,817
    Location:
    ADL/SA The Monopoly State
    is your raidz raiding partitions or drives? you should always raid partitions, then it doesnt matter.
    so if you did, it doesnt matter XD
     
  4. OP
    OP
    frozencrystal

    frozencrystal Member

    Joined:
    Feb 12, 2009
    Messages:
    30
    Thanks rowan194, completely overlooked MSY :p Having returned a number of hdds over the years, the potential possibility of dealing with MSY's warranty department isn't very appealing but I guess I don't really have much of a choice.

    Both drives had unrecoverable read errors - one failed SMART testing on boot (warning of imminent failure), and both had reallocated sector counts of >1000 (from memory, anyway).

    Raiding entire drives unfortunately, because that was what I read as best practice when I built my array. Will definitely be raid-ing partitions next time to avoid this situation.
     
  5. sub.mesa

    sub.mesa Member

    Joined:
    Jun 23, 2010
    Messages:
    271
    Location:
    Europe
    Did you flash your controller into IT-mode firmware or are you running IR (RAID) mode firmware?

    Unless you mean something else, this is the wrong way to do it. The proper way to do it is to give the labeled device to ZFS, so not /dev/ada4 since that would overwrite the label, but rather /dev/label/disk4 or something.

    I recommend to use GPT partitions and leaving some space unused at the end of the capacity.

    Yes, because you are not using the recommended optimal number of disks, for RAID-Z2 that is either 6 disks or 10 disks. For 512-byte disks this does not matter, but it does when using 4K sector disks.

    The geom_nop (GNOP) provider does not consume any space, transforming the sectorsize does not alter capacity in any way. But you still cannot add a disk with higher sectorsize to an existing pool. That will NEVER work. If your pool is used for 512-byte sectorsize (ashift=9) you can never add a disk with a higher sector size; on any platform. You can do it the other way around, when having a pool with ashift=12 (4K sectorsize) and add a 512-byte sectorsize disk. So you can add disks with a lower sectorsize, but never with a higher sectorsize.

    Not sure what your best option is. I think you should be able to get a Hitachi 5K3000 2TB disk soon. That really would be your best option. Are there no webshops in Australia that can ship the drive with a 'next-day' service?
     
  6. rowan194

    rowan194 Member

    Joined:
    Jan 5, 2009
    Messages:
    2,046
    MSY got into trouble with the ACCC earlier this year (from memory) regarding warranties, so things may have changed since your last experience, or reading of others experiences.

    I've had faulty drives replaced on the spot before. Apparently this was not normal policy but they allowed it at my local store. You never know.
     
  7. OP
    OP
    frozencrystal

    frozencrystal Member

    Joined:
    Feb 12, 2009
    Messages:
    30
    Flashed to IT-mode firmware

    My bad, yeah I gave the labelled device to ZFS/

    I've been following your many posts on ZFS (both here and hardforum) and thought about doing that originally, but unfortunately 6 drives was too few for what I wanted and 10 drives wouldn't fit in the case I had in mind. Should have just forked out for one of those Norco cases in hindisight :p

    Thanks for clarifying that - I really don't know too much about what i'm doing, i've just been following what other people have posted online ^^

    You wouldn't happen to have bought it from a NSW-based store, would you? ^^ But cheers, will probably head off to a MSY sometime later this week to get one. Will have to think about what to do with the drive once the RMA'd ones return (would love to have it as a hot spare but for insufficient drive bays + no spare power plugs) but alas life is never easy :p
     
  8. ctn

    ctn Member

    Joined:
    Dec 24, 2007
    Messages:
    123
    Location:
    Sydney, NSW
    Is there a guide on how to set up raidz for raiding partitions?
     
  9. davros123

    davros123 Member

    Joined:
    Jun 18, 2008
    Messages:
    3,048
    interested in this...I was under the impression it was zfs best practice to pass the whole drive and NOT use partitions.

    terra, can you expand on your suggestion to use partitions?
     
  10. sub.mesa

    sub.mesa Member

    Joined:
    Jun 23, 2010
    Messages:
    271
    Location:
    Europe
    That's Solaris-specific advice, could be due to some limitations on that platform. On FreeBSD the best practice is to always use partitions. For example, because if you need to replace a disk you can get a replacement which is a few kilo/megabytes smaller than your original, causing the replacement disk to be rejected. This is highly undesirable.

    Besides, having a device name like /dev/gpt/samsung-disk3 is alot better than c8t20d0 or some other dynamically assigned device name. And you need partitions in order to be bootable.
     

Share This Page

Advertisement: