How to setup ZFS share?

I am not too familiar with FreeNAS and ZFS. I want to setup RaidZ1. I follow this guide but did not create zvol.

• Is my setup correct and complete?
• What else do I need to do?
• How much space do I have?
• Running the raid calculator, I should have 36000GB.

The drive is 7 * 6TB HGST NAS 0S03841

 

don't use Z1.
don't use Z1.
don't use Z1.
don't use Z1.
don't use Z1.
don't use Z1.
don't use Z1.
don't use Z1.
don't use Z1.
don't use Z1.
don't use Z1.
don't use Z1.
don't use Z1.
don't use Z1.
don't use Z1.
don't use Z1.
don't use Z1.
don't use Z1.
don't use Z1.
don't use Z1.
don't use Z1.
don't use Z1.
don't use Z1.
don't use Z1.
don't use Z1.
don't use Z1.
don't use Z1.
don't use Z1.
don't use Z1.

Also Metadata and checksumming is a thing. You will have less space than n-2 disks.

zpool status will give you a better idea than zfs status.

Also, never ever ever ever fill a zfs pool. Infact budget 7-10% space free being "full".

Finally, create a dataset, goto cifs, create a share - assign to dataset.

 

Good to read (this).

 

You recommending mirrors by this?

 

No. Z2 or more correctly Z3 for members of that size. Scrub and Resilver times will be literally forever - even with tweaking of the Resilver/Scrub priority/performance, it will still be a very very long time.

Mirrors are better for iops WITH throughput, however this isn't typically needed for a basic CIFS share - really only Virtualisation shared storage, iSCSI mounts etc.

You will still get nice throughput from a single vdev Raidz1/2/3 - just don't bomb it with large io requests.

 

Recommended disk vdev size as follows from research yeah?

Raidz1 - 5 disks
Raidz2 - 6 disks
Raidz3 - 7 disks

My understanding is without the redundant disk you want to keep 128 evenly by the number of disks eg. 128/4=32 or bigger Raidz1 with 9 disks 128/8=16

 

Not really - its not just "simple" like that. Depending on what attributes you want out of your Raid-Z, you might go wider or narrower. Also things change depending sector/record size (less important now that everything is 4k sectors).

http://blog.delphix.com/matt/2014/06/06/zfs-stripe-width/

Not really - the above link better explains.

 

"No. Z2 or more correctly Z3 for members of that size. Scrub and Resilver times will be literally forever - even with tweaking of the Resilver/Scrub priority/performance, it will still be a very very long time."

Hi NSanity, This differs from my experience - admiedly on Solaris, not freebsd - but I can prob ask a mate what his times are for his 50TB system on freebsd...

Yes, if your definition of literally forever as < 30hours.

Scrub time primarily depends upon the data on the disk as it only scubs the data - not the empty part of a pool (as well as the read speed).

On my (rather old nas - shaed resources as a vm esxi with 10 other vm's and a fair amount of concurrent io) scrubs take 26h for 20+TB of data in raidz2 across 10 3TB hitachi's.

I'd expect your newer drives to be faster than my old faithful and slow Hitachi's.

That's not too bad for once a fortnight or once a month scrub imho.

 

http://serverfault.com/questions/499739/tuning-zfs-scrubbing-141kb-s-running-for-15-days
http://broken.net/uncategorized/zfs-performance-tuning-for-scrubs-and-resilvers/

This is not uncommon in the real world.

Scrub/Resilver priority is generally less than actual service priority. By Default - If you can't give ZFS a break to actually scrub, it will never do this. Tiny, random - but continuous - io is enough to stop it from happening inside sane completion times.

Ultimately there is a lot of variables here - but my production boxes (16TB and 25TB usable) take 2-3 days to scrub with no errors.

The whole point of this is - if you're actually resilvering a member of your Z1 vdev, and then you have an URE, you lose the whole vdev. Z2 and Z3 will give you extra insulation against this.

 

I want to maximise the amount of available space. Maybe I should setup Raid 5 with the 7 disk?

 

this does not insulate you from URE's during a resilver.

http://www.zdnet.com/article/why-raid-5-stops-working-in-2009/

 

I find this hard to believe as your describing behaviour consistent with hardware raid 5 where any other filesystem usually cannot recover from the loss of a stripe.

ZFS does both the raid and filesystem and I understand that it will clearly regard it as a broken stripe as it is below minimum redundancy and from that point on it will work out what files are affected and will tell you they are damaged and need to be restored from backup, the resilver will then continue and complete as normal.

Cheers

 

I have the following Scrub default, what is the schedule?

Does it mean it run every 35 days on a Sunday at mid night?

Volume
Threshold days: 35
Minute: 00
Hour: 00
Day of Month: everyday
Month: every month
Day of week: Sunday
Enabled checkbox: true

 

http://olddoc.freenas.org/index.php/ZFS_Scrubs

Edit:

Not exactly. Well, I mean, I think anything but the days doesn't matter.

 

I read that doc and is a bit confuse.

So it run every 35 days?

 

Reading it again, me too.

 

No, you may lose the block/file but not the pool (not a vdev).

 

No.

https://forums.freenas.org/index.php?threads/zfs-and-raidz1-resilvering.17335/#post-91907

 

Pretty (aka) extremely unlikely to lose all the metadata for the pool!

needle in haystack stuff.

As for a single URE killing the pool on a resilver of a raidz...I'll need to have a read up on that...I would have expected it's just lose the block...not give up altogether!

In either case, absolutely agree you want raidz2, not z1.

edit:
Dasun contradicts cyberjock and says you will not lose the pool here...
https://forums.freenas.org/index.ph...ountered-during-mirrored-vdev-resilver.15174/

This is what i would expect.

Personally, having read hundreds of cyberjock's posts I have come to understand his statements of fact are to be taken with a large dose of salt.

 

I won't say its equally likely - but its certainly possible and it will end your day.

Having spoken to him in a professional setting - he is speaking from experience.

1 test != all instances.