Overclockers Australia Forums
OCAU News - Wiki - QuickLinks - Pix - Sponsors  

Go Back   Overclockers Australia Forums > Specific Hardware Topics > Storage & Backup

Notices


Sign up for a free OCAU account and this ad will go away!
Search our forums with Google:
Reply
 
Thread Tools
Old 9th September 2010, 7:45 PM   #1
fredhoon Thread Starter
Member
 
fredhoon's Avatar
 
Join Date: Jun 2003
Location: Brisbane
Posts: 1,970
Question Home Server - unRAID, linux mdraid or FreeBSD ZFS?

It's been a while coming, I've finally put my rack and home server back together. It's time to trial a few OS's and plan out my storage OS and filesystem for the box. I've essentially ruled out OpenSolaris and FreeBSD for ZFS due to Oracle and a lack of time to learn BSD. Now I'm stuck with whether I should go down the linux mdraid path, or move away from KISS and run a virtual instance of unRAID.

I have 8x 1.5TB Samsung 5400rpm 'green' drives and desire 9TB usable space, n-2. I love the unRAID concept of RAID3 style parity disk (via softRAID1) without the striped data - only spin up what's required, data loss restricted to failed hardware, data still readable when degraded or failed. My perception of a lack of flexibility (due to unfamiliarity) with unRAID discourages me from running it as the main server OS is. I would much prefer to run unRAID in a VM so I can choose what software I use to fulfil other roles, either directly on the host or in other VM's.

Home server intended usage....
9TB storage (Media & backup)
scheduled Torrent or Newsgroup downloads
Media streamer to PS3/Amp/TV, which ever works the best
ftp server
remote backup via ftp & rsycn to hosted storage and small office
VM sandpit to play with things (if I ever find the time)
Quiet and lowish power (need to replace the hotswap bay fans)

Hardware
Tyan Thunder s??97 i3010W motherboard
Intel Q6600
8G RAM non-ECC (eventually), some cheap ECC RAM I purchased was not compatible with the useless motherboard
Intel on-board nics
Highpoint software RAID cards
2x 320G 5400rpm 2.5" RAID1 OS
8x 1.5TB Samsung 5400rpm 'green' hdds


I'm going to give the free version of unRAID a bash this weekend to see what it's like, however I'm concerned that running it in a VM is overcomplicating things and will lead to performance issues (depending on virtualisation platform & my skills in setting it up).


Can anyone offer advice, opinions or tales of their experience for the above scenario? Are there alternatives to unRAID with similar storage features?
__________________
Quote:
Originally Posted by Andres View Post
Come-on guys, this is a motoring enthusiast forum, not a Harold Scruby fan club. Lighten up a little
Go soothingly on the grease mud as there lurks a skid demon.
fredhoon is offline   Reply With Quote

Join OCAU to remove this ad!
Old 9th September 2010, 10:55 PM   #2
sub.mesa
Member
 
Join Date: Jun 2010
Location: Europe
Posts: 271
Default

No ECC, no Checksums for your data; how important is your data anyway?

I think you wrote off ZFS too quickly; it's the best filesystem at the moment. But you don't have real HBA controller; so a proper ZFS server wouldn't be possible without swapping that Highpoint for an LSI HBA, for example.

Q6600 is not low power at all; this is not a low-power build. It's quite a mainstream build with some older components. But still should make a great NAS.

Can you explain "Media streamer"; what kind of protocol/port is that? Don't you just mean CIFS access?
__________________
I will gladly help you with any questions related to ZFS. I'm currently working on my own ZFS platform called ZFSguru.
sub.mesa is offline   Reply With Quote
Old 9th September 2010, 11:22 PM   #3
chancey
Member
 
Join Date: Jan 2005
Location: Sydney
Posts: 1,415
Default

I love ZFS and all its features but unless your using a solaris system I wouldn't trust the fact that oracle won't pull the source code or do something tricky with the licence - it wouldn't be the first time.

Linux software raid (or in combination with LVM) has pretty much all the main features of ZFS, it rock solid and never runs the risk of being pulled out of the kernel. LVM in a way is better than ZFS because you can "remove vdevs from a pool" in zfs-speak.

I haven't tried unraid myself but i've read a bit about it. Don't have enough info to comment on it.
__________________
MusicBrainz AutoEditor | Last.fm: chancey (170,000+ plays)
Laptop: Unibody MacBook 2.4ghz, 4GB DDR3, 750GB
Desktop: Mac Pro (8-core) 2.8Ghz, 6GB ECC RAM
Server: Norco 4220, Q9550, 4GB 1066Mhz, 3ware 9650SE-16ML, 31TB, Fedora 10 x64
chancey is offline   Reply With Quote
Old 9th September 2010, 11:50 PM   #4
sub.mesa
Member
 
Join Date: Jun 2010
Location: Europe
Posts: 271
Default

Quote:
Originally Posted by chancey View Post
I love ZFS and all its features but unless your using a solaris system I wouldn't trust the fact that oracle won't pull the source code or do something tricky with the licence - it wouldn't be the first time.
That's why you run FreeBSD; they clearly stated ZFS is going to stay in FreeBSD and the V28 patchset also means it's getting the greatest ZFS implementation. CDDL license is not something that can be revoked by Oracle; though they can stop releasing new code under this license.

Either way, FreeBSD would make a solid ZFS platform both now and especially in the near future. FreeBSD 9, when it releases, might have usable V28 implementation. Though it's at least half a year away.

Quote:
Linux software raid (or in combination with LVM) has pretty much all the main features of ZFS
Cool! How do i get checksums on Linux? Ahh yes i have to use Btrfs; but that's also owned by Oracle (how ironic) and it's not stable and usable yet. And self-healing? Nah; you have to unmount your filesystems and check them with an offline utility.

I consider checksumming critical. But even with Btrfs+LVM i'm not sure if Btrfs can actually access the redundant information and use that instead if one disk has bad data. This is the critical part of ZFS and why it is SO MUCH BETTER to have both RAID engine and Filesystem in one package. With those separated; you lowered your potential reliability considerably. Checksums and access to redundant copies is KEY for any 'next-gen' filesystem.
__________________
I will gladly help you with any questions related to ZFS. I'm currently working on my own ZFS platform called ZFSguru.
sub.mesa is offline   Reply With Quote
Old 10th September 2010, 7:02 PM   #5
Zer0
Member
 
Zer0's Avatar
 
Join Date: Jun 2001
Location: Singapore/Brisbane
Posts: 910
Default

Try nexentastor community edition.

Packaged ZFS with a nice web interface to admin. Works well for me so far. Add in a small cheap SSD and use it as a cache to accelerate your read or write speed.
__________________
OCAU ANIME CLUB #22
Zer0 is offline   Reply With Quote
Old 10th September 2010, 11:58 PM   #6
fredhoon Thread Starter
Member
 
fredhoon's Avatar
 
Join Date: Jun 2003
Location: Brisbane
Posts: 1,970
Default

Quote:
Originally Posted by sub.mesa View Post
No ECC, no Checksums for your data; how important is your data anyway?

I think you wrote off ZFS too quickly; it's the best filesystem at the moment. But you don't have real HBA controller; so a proper ZFS server wouldn't be possible without swapping that Highpoint for an LSI HBA, for example.

Q6600 is not low power at all; this is not a low-power build. It's quite a mainstream build with some older components. But still should make a great NAS.

Can you explain "Media streamer"; what kind of protocol/port is that? Don't you just mean CIFS access?
You're right, I have written off ZFS too quickly. It's mainly due to my leanings towards unRIAD, in a toss up between mdraid + lvm and ZFS I would explore the options much more thoroughly. Of the data that I am storing, I am only concerned with the integrity of a few hundred Gb's and it will be backed up accordingly in multiple places. The rest of the data would be inconvenient to lose, but not irreplaceable.

Running non-ECC RAM and the rest o the mix of old hardware is purely a cost issue. When I can find some decently priced RAM compatible with my motherboard (or a replacement board) i'll definitely be using it. And by lowish power I was mainly referring to the hard-drives. Although it would be nice to save power, I'm more concerned about heat as it has a tendency to accompany noise. It would be interesting to see what the idle and typical load power draw of the system is with and without hdd's spinning away.

By media streamer I desire something along the lines of Playstation Media server / XMBC for spitting out video to the TV (probably via the PS3). Currently I'm using a laptop as a HTPC, which makes it less than portable with HDMI & Cat5 cables plugged into it. I need a media streamer rather than just presenting the devices with a SMB share as they have limited file and codec support and I don't want to go transcoding things.

Quote:
Originally Posted by Zer0
Try nexentastor community edition.

Packaged ZFS with a nice web interface to admin. Works well for me so far. Add in a small cheap SSD and use it as a cache to accelerate your read or write speed.
Cheers, I'll give it a look. Although if I go down the ZFS path I want to install it on a standard FreeBSD platform for flexibility and the learning experience.
__________________
Quote:
Originally Posted by Andres View Post
Come-on guys, this is a motoring enthusiast forum, not a Harold Scruby fan club. Lighten up a little
Go soothingly on the grease mud as there lurks a skid demon.
fredhoon is offline   Reply With Quote
Old 11th September 2010, 12:05 AM   #7
thebranded
Member
 
Join Date: Nov 2006
Location: Sydney
Posts: 2,128
Default

Have you looked at FreeNAS?

Has ZFS or linux, your choice. Supports all your requirements and has a web interface.

I run it on two boxes, one a backup of the other. plays fine with everything from xbox1- mac (via VM) does not support online expansion though.
thebranded is offline   Reply With Quote
Old 11th September 2010, 12:40 AM   #8
sub.mesa
Member
 
Join Date: Jun 2010
Location: Europe
Posts: 271
Default

Quote:
Originally Posted by fredhoon View Post
You're right, I have written off ZFS too quickly. It's mainly due to my leanings towards unRIAD, in a toss up between mdraid + lvm and ZFS I would explore the options much more thoroughly.
I don't know unRAID well, but it's not ZFS. In terms of reliability, ZFS employs the most protections for your data of any local filesystem.

Quote:
Of the data that I am storing, I am only concerned with the integrity of a few hundred Gb's and it will be backed up accordingly in multiple places. The rest of the data would be inconvenient to lose, but not irreplaceable.
Consider using copies=2 on your most important data. And the filesystem checksums done by ZFS would make a formidable defense against corruption. Even when it occurs, ZFS let's you know exactly which files are corrupt beyond ZFS' self-healing capabilities. Metadata always has at least two copies.

Quote:
Running non-ECC RAM and the rest o the mix of old hardware is purely a cost issue.
Reason the more you need ZFS; here corruption is spotted and you know about it; prompting you to do another round of memtest86 which can make your life easier really. The fact that you KNOW all data to be intact is invaluable. That makes backups also more reliable, especially if you combine them with snapshots so you get incremental backups.

The features of ZFS are very nice to explore, and discover how they can benefit your way of storing files without having to do alot of micro-managing. ZFS is pretty maintenance-free too.

Quote:
It would be interesting to see what the idle and typical load power draw of the system is with and without hdd's spinning away.
I can highly recommend an AC power meter; like Kill-a-Watt but many variants exist. That way you can measure power consumption, and each watt consumed basically means one watt in heat generation.

Quote:
By media streamer I desire something along the lines of Playstation Media server / XMBC for spitting out video to the TV (probably via the PS3). Currently I'm using a laptop as a HTPC, which makes it less than portable with HDMI & Cat5 cables plugged into it. I need a media streamer rather than just presenting the devices with a SMB share as they have limited file and codec support and I don't want to go transcoding things.
Interesting! I believe MediaTomb on FreeBSD is used for this; though i didn't really know what for and how it works; i don't own a Xbox or PS3. I guess it transcodes movies on-the-fly to a format that the PS3 can understand? You could test MediaTomb if you choose FreeBSD route.

Quote:
Cheers, I'll give it a look. Although if I go down the ZFS path I want to install it on a standard FreeBSD platform for flexibility and the learning experience.
I can recommend FreeBSD 8.1. I have some install guides on my website as well including many screenshots. They can be found here:

http://submesa.com/data/bsd/install
http://submesa.com/data/bsd/setup

But if you like i can lend assistance with your first steps into FreeBSD territory; just to get you familiar. So feel free to contact me about FreeBSD stuff. Asking can' t hurt. ;-)
__________________
I will gladly help you with any questions related to ZFS. I'm currently working on my own ZFS platform called ZFSguru.
sub.mesa is offline   Reply With Quote
Old 11th September 2010, 9:53 AM   #9
Betzie
Member
 
Betzie's Avatar
 
Join Date: Aug 2004
Posts: 753
Default

I'm looking at doing somthing similar to the op so I will be watching this thread.

Although I cant help with OS selection as I am still working that out myself I can suggest www.vuze.com as your torrent client and media server, from my googling it will install on BSD and opensolaris
Betzie is offline   Reply With Quote
Old 11th September 2010, 10:12 AM   #10
VenomousDesigns
Member
 
Join Date: Jan 2009
Posts: 94
Default

I've been reading a number of threads on OC Aus and have come to the conclusion, as I have never seen it mentioned recently, that everyone has either not heard of FlexRAID or it's like one of those family members no one mentions haha.

FlexRAID encompasses everything unRAID does and more for FREE. Although I haven't done alot of reach into ZFS I also see it as a competitor.

It's ability to be able to be used under any OS, fantastic power consumption, it combats H/W RAID in the fact that an entire Array isn't lost if more DRUs are lost than PPUs (just the disks itself thereafter) and ability to be able to remove HDDs and share Data and then place back is ASTOUNDING.

Again all comes down to what one is actually using his/her server for. A static Media/File such as mine, FlexRAIDs 'snapshot' feature-set is perfect (work on FlexRAID Live is also steaming along).

Check it out: http://www.openegg.org/FlexRAID.curi
VenomousDesigns is offline   Reply With Quote
Old 11th September 2010, 10:15 PM   #11
sreg0r
Member
 
sreg0r's Avatar
 
Join Date: Jul 2001
Location: Melbourne
Posts: 1,100
Default

Quote:
Originally Posted by Betzie View Post
I'm looking at doing somthing similar to the op so I will be watching this thread.

Although I cant help with OS selection as I am still working that out myself I can suggest www.vuze.com as your torrent client and media server, from my googling it will install on BSD and opensolaris
I, like most people here (except sub.mesa) are more comfortable using windows than opensolaris/freebsd. So I have opensolaris as my host o/s to make use of zfs and then simply run a windows XP VM for running native windows apps like media streaming, usenet, torrent etc.
sreg0r is offline   Reply With Quote
Old 11th September 2010, 10:52 PM   #12
Betzie
Member
 
Betzie's Avatar
 
Join Date: Aug 2004
Posts: 753
Default

Quote:
Originally Posted by sreg0r View Post
I, like most people here (except sub.mesa) are more comfortable using windows than opensolaris/freebsd. So I have opensolaris as my host o/s to make use of zfs and then simply run a windows XP VM for running native windows apps like media streaming, usenet, torrent etc.
I have set mine up the other way around with vista as a host and vm of whs for shares, I have it working but it is quite slow and it has been tedious to setup and maintain.

My server box is a part of the home thertre setup so i want a front end for it to display on my loungeroom screen, but essentially it will just serve media to my ps3, vuse will let me set it up as a torrent server and is a network streamer aswel and works with ps3

At this stage I am leaning toward opensolaris aslong as I can get a the vuze client to work as my frontend and upnp to serve media working and a lovely zfs storage pool

the rest I can remote desktop or do remotely with vm server
Betzie is offline   Reply With Quote
Old 12th September 2010, 1:21 AM   #13
sub.mesa
Member
 
Join Date: Jun 2010
Location: Europe
Posts: 271
Default

Quote:
Originally Posted by sreg0r View Post
I, like most people here (except sub.mesa) are more comfortable using windows than opensolaris/freebsd. So I have opensolaris as my host o/s to make use of zfs and then simply run a windows XP VM for running native windows apps like media streaming, usenet, torrent etc.
I don't use FreeBSD as desktop OS; only as server OS.

You want a hybrid server/workstation running on one PC. I generally recommend to avoid that; it complicates your setup and problems might cause both systems to be unavailable. But it does allow you to use less hardware. If it works well for you that's great; keep it that way.

But generally, a NAS is a dedicated computer serving only files and not used as workstation. This setup is simpler, faster and less prone to failures/errors i believe, but it does require more hardware: a dedicated server and a separate workstation.

The workstation can be anything; Windows, Mac OSX or Linux. I use Ubuntu as primary OS. Things like torrents can be offloaded to the server, and you connect with a frontend application from your (Windows) workstation to your torrent daemon running on the server. Transmission can do this and others as well i believe.

I'll look into Vuze when i have some spare time.
__________________
I will gladly help you with any questions related to ZFS. I'm currently working on my own ZFS platform called ZFSguru.
sub.mesa is offline   Reply With Quote
Old 12th September 2010, 2:26 AM   #14
sreg0r
Member
 
sreg0r's Avatar
 
Join Date: Jul 2001
Location: Melbourne
Posts: 1,100
Default

I was referring to nas/server usage only. Although I don't mind working with VMs on the server I wouldn't use one for my primary machine.
sreg0r is offline   Reply With Quote
Old 12th September 2010, 2:40 AM   #15
sub.mesa
Member
 
Join Date: Jun 2010
Location: Europe
Posts: 271
Default

Oh sorry for the misunderstanding; you in essence use Windows in a VM as a server not as workstation?

The torrent stuff could be easily offloaded to the FreeBSD server. It would run a 'daemon' without any interface, and on your windows desktop you use a graphical frontend to connect to that interface. But the torrent application would actually be running on the server not on your windows desktop.

As for other things, what are the things you use? I would like to add useful stuff to my ZFS interface which are popular. The mediastreaming to PS3/XBOX should work with MediaTomb, but could you define usenet? You don't mean an NNTP server, do you?
__________________
I will gladly help you with any questions related to ZFS. I'm currently working on my own ZFS platform called ZFSguru.
sub.mesa is offline   Reply With Quote
Reply

Bookmarks

Sign up for a free OCAU account and this ad will go away!

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +10. The time now is 8:11 PM.


Powered by vBulletin® Version 3.8.4
Copyright ©2000 - 2014, Jelsoft Enterprises Ltd. -
OCAU is not responsible for the content of individual messages posted by others.
Other content copyright Overclockers Australia.
OCAU is hosted by Internode!