1. OCAU Merchandise is available! Check out our 20th Anniversary Mugs, Classic Logo Shirts and much more! Discussion in this thread.
    Dismiss Notice

How to setup ZFS share?

Discussion in 'Storage & Backup' started by Multiplexer, Jul 1, 2015.

  1. Multiplexer

    Multiplexer Member

    Joined:
    Feb 26, 2002
    Messages:
    2,135
    Location:
    Home
    I am not too familiar with FreeNAS and ZFS. I want to setup RaidZ1. I follow this guide but did not create zvol.
    • Is my setup correct and complete?
    • What else do I need to do?
    • How much space do I have?
    • Running the raid calculator, I should have 36000GB.
    The drive is 7 * 6TB HGST NAS 0S03841

    [​IMG]
    [​IMG]
    [​IMG]
     
    Last edited: Jul 1, 2015
  2. NSanity

    NSanity Member

    Joined:
    Mar 11, 2002
    Messages:
    18,651
    Location:
    Brisbane
    don't use Z1.
    don't use Z1.
    don't use Z1.
    don't use Z1.
    don't use Z1.
    don't use Z1.
    don't use Z1.
    don't use Z1.
    don't use Z1.
    don't use Z1.
    don't use Z1.
    don't use Z1.
    don't use Z1.
    don't use Z1.
    don't use Z1.
    don't use Z1.
    don't use Z1.
    don't use Z1.
    don't use Z1.
    don't use Z1.
    don't use Z1.
    don't use Z1.
    don't use Z1.
    don't use Z1.
    don't use Z1.
    don't use Z1.
    don't use Z1.
    don't use Z1.
    don't use Z1.

    Also Metadata and checksumming is a thing. You will have less space than n-2 disks.

    zpool status will give you a better idea than zfs status.

    Also, never ever ever ever fill a zfs pool. Infact budget 7-10% space free being "full".

    Finally, create a dataset, goto cifs, create a share - assign to dataset.
     
    Last edited: Jul 1, 2015
  3. Rezin

    Rezin Member

    Joined:
    Oct 27, 2002
    Messages:
    9,485
  4. passivekid

    passivekid Member

    Joined:
    Sep 10, 2003
    Messages:
    332
    Location:
    Perth, WA
    You recommending mirrors by this? :)
     
  5. NSanity

    NSanity Member

    Joined:
    Mar 11, 2002
    Messages:
    18,651
    Location:
    Brisbane
    No. Z2 or more correctly Z3 for members of that size. Scrub and Resilver times will be literally forever - even with tweaking of the Resilver/Scrub priority/performance, it will still be a very very long time.

    Mirrors are better for iops WITH throughput, however this isn't typically needed for a basic CIFS share - really only Virtualisation shared storage, iSCSI mounts etc.

    You will still get nice throughput from a single vdev Raidz1/2/3 - just don't bomb it with large io requests.
     
    Last edited: Jul 2, 2015
  6. passivekid

    passivekid Member

    Joined:
    Sep 10, 2003
    Messages:
    332
    Location:
    Perth, WA
    Recommended disk vdev size as follows from research yeah?

    Raidz1 - 5 disks
    Raidz2 - 6 disks
    Raidz3 - 7 disks

    My understanding is without the redundant disk you want to keep 128 evenly by the number of disks eg. 128/4=32 or bigger Raidz1 with 9 disks 128/8=16
     
  7. NSanity

    NSanity Member

    Joined:
    Mar 11, 2002
    Messages:
    18,651
    Location:
    Brisbane
    Not really - its not just "simple" like that. Depending on what attributes you want out of your Raid-Z, you might go wider or narrower. Also things change depending sector/record size (less important now that everything is 4k sectors).

    http://blog.delphix.com/matt/2014/06/06/zfs-stripe-width/

    Not really - the above link better explains.
     
  8. davros123

    davros123 Member

    Joined:
    Jun 18, 2008
    Messages:
    3,222
    "No. Z2 or more correctly Z3 for members of that size. Scrub and Resilver times will be literally forever - even with tweaking of the Resilver/Scrub priority/performance, it will still be a very very long time."

    Hi NSanity, This differs from my experience - admiedly on Solaris, not freebsd - but I can prob ask a mate what his times are for his 50TB system on freebsd...

    Yes, if your definition of literally forever as < 30hours.

    Scrub time primarily depends upon the data on the disk as it only scubs the data - not the empty part of a pool (as well as the read speed).

    On my (rather old nas - shaed resources as a vm esxi with 10 other vm's and a fair amount of concurrent io) scrubs take 26h for 20+TB of data in raidz2 across 10 3TB hitachi's.

    I'd expect your newer drives to be faster than my old faithful and slow Hitachi's.

    That's not too bad for once a fortnight or once a month scrub imho.
     
    Last edited: Jul 2, 2015
  9. NSanity

    NSanity Member

    Joined:
    Mar 11, 2002
    Messages:
    18,651
    Location:
    Brisbane
    http://serverfault.com/questions/499739/tuning-zfs-scrubbing-141kb-s-running-for-15-days
    http://broken.net/uncategorized/zfs-performance-tuning-for-scrubs-and-resilvers/

    This is not uncommon in the real world.

    Scrub/Resilver priority is generally less than actual service priority. By Default - If you can't give ZFS a break to actually scrub, it will never do this. Tiny, random - but continuous - io is enough to stop it from happening inside sane completion times.

    Ultimately there is a lot of variables here - but my production boxes (16TB and 25TB usable) take 2-3 days to scrub with no errors.

    The whole point of this is - if you're actually resilvering a member of your Z1 vdev, and then you have an URE, you lose the whole vdev. Z2 and Z3 will give you extra insulation against this.
     
    Last edited: Jul 2, 2015
  10. OP
    OP
    Multiplexer

    Multiplexer Member

    Joined:
    Feb 26, 2002
    Messages:
    2,135
    Location:
    Home
    I want to maximise the amount of available space. Maybe I should setup Raid 5 with the 7 disk?
     
  11. NSanity

    NSanity Member

    Joined:
    Mar 11, 2002
    Messages:
    18,651
    Location:
    Brisbane
  12. HobartTas

    HobartTas Member

    Joined:
    Jun 22, 2006
    Messages:
    1,160
    I find this hard to believe as your describing behaviour consistent with hardware raid 5 where any other filesystem usually cannot recover from the loss of a stripe.

    ZFS does both the raid and filesystem and I understand that it will clearly regard it as a broken stripe as it is below minimum redundancy and from that point on it will work out what files are affected and will tell you they are damaged and need to be restored from backup, the resilver will then continue and complete as normal.

    Cheers
     
  13. OP
    OP
    Multiplexer

    Multiplexer Member

    Joined:
    Feb 26, 2002
    Messages:
    2,135
    Location:
    Home
    I have the following Scrub default, what is the schedule?

    Does it mean it run every 35 days on a Sunday at mid night?

    Volume
    Threshold days: 35
    Minute: 00
    Hour: 00
    Day of Month: everyday
    Month: every month
    Day of week: Sunday
    Enabled checkbox: true
     
  14. Rezin

    Rezin Member

    Joined:
    Oct 27, 2002
    Messages:
    9,485
    http://olddoc.freenas.org/index.php/ZFS_Scrubs

    Edit:

    Not exactly. Well, I mean, I think anything but the days doesn't matter.

     
    Last edited: Jul 2, 2015
  15. OP
    OP
    Multiplexer

    Multiplexer Member

    Joined:
    Feb 26, 2002
    Messages:
    2,135
    Location:
    Home
    I read that doc and is a bit confuse.

    So it run every 35 days?
     
  16. Rezin

    Rezin Member

    Joined:
    Oct 27, 2002
    Messages:
    9,485
    Reading it again, me too. :)
     
    Last edited: Jul 2, 2015
  17. davros123

    davros123 Member

    Joined:
    Jun 18, 2008
    Messages:
    3,222
    No, you may lose the block/file but not the pool (not a vdev).
     
  18. NSanity

    NSanity Member

    Joined:
    Mar 11, 2002
    Messages:
    18,651
    Location:
    Brisbane
  19. davros123

    davros123 Member

    Joined:
    Jun 18, 2008
    Messages:
    3,222
    Pretty (aka) extremely unlikely to lose all the metadata for the pool!

    needle in haystack stuff.

    As for a single URE killing the pool on a resilver of a raidz...I'll need to have a read up on that...I would have expected it's just lose the block...not give up altogether!

    In either case, absolutely agree you want raidz2, not z1.

    edit:
    Dasun contradicts cyberjock and says you will not lose the pool here...
    https://forums.freenas.org/index.ph...ountered-during-mirrored-vdev-resilver.15174/
    This is what i would expect.

    Personally, having read hundreds of cyberjock's posts I have come to understand his statements of fact are to be taken with a large dose of salt.
     
    Last edited: Jul 2, 2015
  20. NSanity

    NSanity Member

    Joined:
    Mar 11, 2002
    Messages:
    18,651
    Location:
    Brisbane
    I won't say its equally likely - but its certainly possible and it will end your day.

    Having spoken to him in a professional setting - he is speaking from experience.

    1 test != all instances.
     
    Last edited: Jul 2, 2015

Share This Page

Advertisement: