Large Storage Build

We are both wrong. it's not 1:1 as I said, not 1:5...but rather depends upon the block size.

http://www.oracle.com/technetwork/articles/servers-storage-admin/o11-113-size-zfs-dedup-1354231.html

 

where's this stated?

 

*sings* in a world of imagination

 

So.

I've had major discussions with various Nexenta technicians (both here and in the US) about the topic over the past 2 years - across medium to large to stupid huge deployments with dedupe.

They've told me what works and what doesn't - and in terms of what is safe to "install and not monitor on a daily basis".

Here's an Oracle consultant of 20 Years - http://constantin.glez.de/blog/2011/07/zfs-dedupe-or-not-dedupe

Here's FreeNAS 9.3 Documentation - http://doc.freenas.org/9.3/freenas_storage.html#deduplication

This guy is saying that if you care about performance 20GB of Ram / TB - http://permabit.com/zfs-dedupe-just-doesnt-make-any-sense/

Here's some maths on a real world 10TB scenario - https://pthree.org/2013/12/18/zfs-administration-appendix-d-the-true-cost-of-deduplication/

Heres a guy saying 6GB/1TB - http://jrs-s.net/2015/02/24/zfs-dedup-tested-found-wanting/

 

Man 20GB per TB is crazy though. My 100TB would require 2TB of Ram, madness.

Bit rot is not really an issue with Media Files anyway.You might end up with a slight stutter in a movie or something like that but otherwise its fine. I doubt corporates would need to take a machine to a LAN so the whole 2in1 PC requirement I have is not an issue for them.

Also I can't fit 100GB of Ram into my PC (or Any non Server board) so ZFS is not workable based on your statements.

 

You can get away with a lot less ram these days, it used to have high utilization in early builds of FreeNAS but seems to have settled down as it has matured. I only give 6gb to my NAS4Free VM with 20TB and shes been happy for a loooooong time over many drive failures, scrubs etc. I don't even see full RAM usage on the system as is and it will happily saturate my gigabit link.

Anyone who says there are issues with running FreeNAS / NAS4Free in a VM environment is talking out of their ass, I have been doing it for so long that I cant even remember when I started now on ESXi.

 

Again, on the topic of Ram and ZFS today.

We're upgrading our 16TB Usable Nexenta box from 64GB to 128GB next week. Why? Because it the arc metadata overflowed onto disk and thus all operations required a page to disk. This happened over a period just 16 days;

Code:

root@zeus:/# uptime
13:43:11    up 15 day(s), 18:36,  1 user,  load average: 0.81, 0.77, 0.76

This isn't specifically a Nexenta/Illumnos thing - this is a ZFS thing.

And sure, you probably wouldn't notice it if you just had CIFS shares - because who gives a fuck if you have a 150-400ms read/write latency when you're copying some files. But for VM Usage this pretty much spells death to any kind of performance.

We could probably get along with 96GB of ram just nicely - but Nexenta recommends 128GB, and its reasonably cheap.

 

LMAO @ this thread.

OP asked a question done to death and likely never bothered searching before posting it. Thread gets derailed quickly and goes on and on, no return of the OP.

 

Yep. Seems strange to decide on CPU before motherboard, and I would suggest that motherboard and HBA are the two most important things for a storage build.

 

Server boards are pretty good typically. You don't have fuckery where HBA's don't work because of limited option rom and other such bullshit.

 

True, but for 24 drives I'd be thinking more than 32GB ECC ram, if not straight away then at least down the road. However the E3 CPU is limited to max 32GB ram...

 

I said this earlier.

FreeNAS zealots all claim that 8GB is overkill.

 

Word.

Give it 8G or 32, 96GB and you'll see no material difference in a home environment - except to your wallet!

Commercial server (pending usage pattern) would be a different kettle of fish.

 

fixed that for you.

 

yep. the mob rules over there....and the mob has drunk all the coolaid.

 

I would suggest LSI 9211 8i cards with a Intel expander.

On the tpoic of memory sizing, I had 24gb allocated to the vm hosting ominos and nappit. I didnt however enable dedupe. Performance was quite good.