Home Server - unRAID, linux mdraid or FreeBSD ZFS?

Discussion in 'Storage & Backup' started by fredhoon, Sep 9, 2010.

  1. fredhoon

    fredhoon Member

    Joined:
    Jun 27, 2003
    Messages:
    2,187
    Location:
    Brisbane
    It's been a while coming, I've finally put my rack and home server back together. It's time to trial a few OS's and plan out my storage OS and filesystem for the box. I've essentially ruled out OpenSolaris and FreeBSD for ZFS due to Oracle and a lack of time to learn BSD. Now I'm stuck with whether I should go down the linux mdraid path, or move away from KISS and run a virtual instance of unRAID.

    I have 8x 1.5TB Samsung 5400rpm 'green' drives and desire 9TB usable space, n-2. I love the unRAID concept of RAID3 style parity disk (via softRAID1) without the striped data - only spin up what's required, data loss restricted to failed hardware, data still readable when degraded or failed. My perception of a lack of flexibility (due to unfamiliarity) with unRAID discourages me from running it as the main server OS is. I would much prefer to run unRAID in a VM so I can choose what software I use to fulfil other roles, either directly on the host or in other VM's.

    Home server intended usage....
    9TB storage (Media & backup)
    scheduled Torrent or Newsgroup downloads
    Media streamer to PS3/Amp/TV, which ever works the best
    ftp server
    remote backup via ftp & rsycn to hosted storage and small office
    VM sandpit to play with things (if I ever find the time)
    Quiet and lowish power (need to replace the hotswap bay fans)

    Hardware
    Tyan Thunder s??97 i3010W motherboard
    Intel Q6600
    8G RAM non-ECC (eventually), some cheap ECC RAM I purchased was not compatible with the useless motherboard
    Intel on-board nics
    Highpoint software RAID cards
    2x 320G 5400rpm 2.5" RAID1 OS
    8x 1.5TB Samsung 5400rpm 'green' hdds


    I'm going to give the free version of unRAID a bash this weekend to see what it's like, however I'm concerned that running it in a VM is overcomplicating things and will lead to performance issues (depending on virtualisation platform & my skills in setting it up).


    Can anyone offer advice, opinions or tales of their experience for the above scenario? Are there alternatives to unRAID with similar storage features?
     
  2. sub.mesa

    sub.mesa Member

    Joined:
    Jun 23, 2010
    Messages:
    271
    Location:
    Europe
    No ECC, no Checksums for your data; how important is your data anyway?

    I think you wrote off ZFS too quickly; it's the best filesystem at the moment. But you don't have real HBA controller; so a proper ZFS server wouldn't be possible without swapping that Highpoint for an LSI HBA, for example.

    Q6600 is not low power at all; this is not a low-power build. It's quite a mainstream build with some older components. But still should make a great NAS.

    Can you explain "Media streamer"; what kind of protocol/port is that? Don't you just mean CIFS access?
     
  3. chancey

    chancey Member

    Joined:
    Jan 5, 2005
    Messages:
    1,415
    Location:
    Sydney
    I love ZFS and all its features but unless your using a solaris system I wouldn't trust the fact that oracle won't pull the source code or do something tricky with the licence - it wouldn't be the first time.

    Linux software raid (or in combination with LVM) has pretty much all the main features of ZFS, it rock solid and never runs the risk of being pulled out of the kernel. LVM in a way is better than ZFS because you can "remove vdevs from a pool" in zfs-speak.

    I haven't tried unraid myself but i've read a bit about it. Don't have enough info to comment on it.
     
  4. sub.mesa

    sub.mesa Member

    Joined:
    Jun 23, 2010
    Messages:
    271
    Location:
    Europe
    That's why you run FreeBSD; they clearly stated ZFS is going to stay in FreeBSD and the V28 patchset also means it's getting the greatest ZFS implementation. CDDL license is not something that can be revoked by Oracle; though they can stop releasing new code under this license.

    Either way, FreeBSD would make a solid ZFS platform both now and especially in the near future. FreeBSD 9, when it releases, might have usable V28 implementation. Though it's at least half a year away.

    Cool! How do i get checksums on Linux? Ahh yes i have to use Btrfs; but that's also owned by Oracle (how ironic) and it's not stable and usable yet. And self-healing? Nah; you have to unmount your filesystems and check them with an offline utility.

    I consider checksumming critical. But even with Btrfs+LVM i'm not sure if Btrfs can actually access the redundant information and use that instead if one disk has bad data. This is the critical part of ZFS and why it is SO MUCH BETTER to have both RAID engine and Filesystem in one package. With those separated; you lowered your potential reliability considerably. Checksums and access to redundant copies is KEY for any 'next-gen' filesystem.
     
  5. Zer0

    Zer0 Member

    Joined:
    Jun 28, 2001
    Messages:
    907
    Location:
    Singapore/Brisbane
    Try nexentastor community edition.

    Packaged ZFS with a nice web interface to admin. Works well for me so far. Add in a small cheap SSD and use it as a cache to accelerate your read or write speed.
     
  6. OP
    OP
    fredhoon

    fredhoon Member

    Joined:
    Jun 27, 2003
    Messages:
    2,187
    Location:
    Brisbane
    You're right, I have written off ZFS too quickly. It's mainly due to my leanings towards unRIAD, in a toss up between mdraid + lvm and ZFS I would explore the options much more thoroughly. Of the data that I am storing, I am only concerned with the integrity of a few hundred Gb's and it will be backed up accordingly in multiple places. The rest of the data would be inconvenient to lose, but not irreplaceable.

    Running non-ECC RAM and the rest o the mix of old hardware is purely a cost issue. When I can find some decently priced RAM compatible with my motherboard (or a replacement board) i'll definitely be using it. And by lowish power I was mainly referring to the hard-drives. Although it would be nice to save power, I'm more concerned about heat as it has a tendency to accompany noise. It would be interesting to see what the idle and typical load power draw of the system is with and without hdd's spinning away.

    By media streamer I desire something along the lines of Playstation Media server / XMBC for spitting out video to the TV (probably via the PS3). Currently I'm using a laptop as a HTPC, which makes it less than portable with HDMI & Cat5 cables plugged into it. I need a media streamer rather than just presenting the devices with a SMB share as they have limited file and codec support and I don't want to go transcoding things.

    Cheers, I'll give it a look. Although if I go down the ZFS path I want to install it on a standard FreeBSD platform for flexibility and the learning experience.
     
  7. thebranded

    thebranded Member

    Joined:
    Nov 30, 2006
    Messages:
    2,210
    Location:
    Sydney
    Have you looked at FreeNAS?

    Has ZFS or linux, your choice. Supports all your requirements and has a web interface.

    I run it on two boxes, one a backup of the other. plays fine with everything from xbox1- mac (via VM) does not support online expansion though.
     
  8. sub.mesa

    sub.mesa Member

    Joined:
    Jun 23, 2010
    Messages:
    271
    Location:
    Europe
    I don't know unRAID well, but it's not ZFS. In terms of reliability, ZFS employs the most protections for your data of any local filesystem.

    Consider using copies=2 on your most important data. And the filesystem checksums done by ZFS would make a formidable defense against corruption. Even when it occurs, ZFS let's you know exactly which files are corrupt beyond ZFS' self-healing capabilities. Metadata always has at least two copies.

    Reason the more you need ZFS; here corruption is spotted and you know about it; prompting you to do another round of memtest86 which can make your life easier really. The fact that you KNOW all data to be intact is invaluable. That makes backups also more reliable, especially if you combine them with snapshots so you get incremental backups.

    The features of ZFS are very nice to explore, and discover how they can benefit your way of storing files without having to do alot of micro-managing. ZFS is pretty maintenance-free too.

    I can highly recommend an AC power meter; like Kill-a-Watt but many variants exist. That way you can measure power consumption, and each watt consumed basically means one watt in heat generation.

    Interesting! I believe MediaTomb on FreeBSD is used for this; though i didn't really know what for and how it works; i don't own a Xbox or PS3. I guess it transcodes movies on-the-fly to a format that the PS3 can understand? You could test MediaTomb if you choose FreeBSD route.

    I can recommend FreeBSD 8.1. I have some install guides on my website as well including many screenshots. They can be found here:

    http://submesa.com/data/bsd/install
    http://submesa.com/data/bsd/setup

    But if you like i can lend assistance with your first steps into FreeBSD territory; just to get you familiar. So feel free to contact me about FreeBSD stuff. Asking can' t hurt. ;-)
     
  9. Betzie

    Betzie Member

    Joined:
    Aug 17, 2004
    Messages:
    774
    I'm looking at doing somthing similar to the op so I will be watching this thread.

    Although I cant help with OS selection as I am still working that out myself I can suggest www.vuze.com as your torrent client and media server, from my googling it will install on BSD and opensolaris
     
  10. VenomousDesigns

    VenomousDesigns Member

    Joined:
    Jan 30, 2009
    Messages:
    98
    I've been reading a number of threads on OC Aus and have come to the conclusion, as I have never seen it mentioned recently, that everyone has either not heard of FlexRAID or it's like one of those family members no one mentions haha.

    FlexRAID encompasses everything unRAID does and more for FREE. Although I haven't done alot of reach into ZFS I also see it as a competitor.

    It's ability to be able to be used under any OS, fantastic power consumption, it combats H/W RAID in the fact that an entire Array isn't lost if more DRUs are lost than PPUs (just the disks itself thereafter) and ability to be able to remove HDDs and share Data and then place back is ASTOUNDING.

    Again all comes down to what one is actually using his/her server for. A static Media/File such as mine, FlexRAIDs 'snapshot' feature-set is perfect (work on FlexRAID Live is also steaming along).

    Check it out: http://www.openegg.org/FlexRAID.curi
     
  11. sreg0r

    sreg0r Member

    Joined:
    Jul 9, 2001
    Messages:
    1,146
    Location:
    Melbourne
    I, like most people here (except sub.mesa) are more comfortable using windows than opensolaris/freebsd. So I have opensolaris as my host o/s to make use of zfs and then simply run a windows XP VM for running native windows apps like media streaming, usenet, torrent etc.
     
  12. Betzie

    Betzie Member

    Joined:
    Aug 17, 2004
    Messages:
    774
    I have set mine up the other way around with vista as a host and vm of whs for shares, I have it working but it is quite slow and it has been tedious to setup and maintain.

    My server box is a part of the home thertre setup so i want a front end for it to display on my loungeroom screen, but essentially it will just serve media to my ps3, vuse will let me set it up as a torrent server and is a network streamer aswel and works with ps3

    At this stage I am leaning toward opensolaris aslong as I can get a the vuze client to work as my frontend and upnp to serve media working and a lovely zfs storage pool

    the rest I can remote desktop or do remotely with vm server
     
  13. sub.mesa

    sub.mesa Member

    Joined:
    Jun 23, 2010
    Messages:
    271
    Location:
    Europe
    I don't use FreeBSD as desktop OS; only as server OS.

    You want a hybrid server/workstation running on one PC. I generally recommend to avoid that; it complicates your setup and problems might cause both systems to be unavailable. But it does allow you to use less hardware. If it works well for you that's great; keep it that way.

    But generally, a NAS is a dedicated computer serving only files and not used as workstation. This setup is simpler, faster and less prone to failures/errors i believe, but it does require more hardware: a dedicated server and a separate workstation.

    The workstation can be anything; Windows, Mac OSX or Linux. I use Ubuntu as primary OS. Things like torrents can be offloaded to the server, and you connect with a frontend application from your (Windows) workstation to your torrent daemon running on the server. Transmission can do this and others as well i believe.

    I'll look into Vuze when i have some spare time. :)
     
  14. sreg0r

    sreg0r Member

    Joined:
    Jul 9, 2001
    Messages:
    1,146
    Location:
    Melbourne
    I was referring to nas/server usage only. Although I don't mind working with VMs on the server I wouldn't use one for my primary machine.
     
  15. sub.mesa

    sub.mesa Member

    Joined:
    Jun 23, 2010
    Messages:
    271
    Location:
    Europe
    Oh sorry for the misunderstanding; you in essence use Windows in a VM as a server not as workstation?

    The torrent stuff could be easily offloaded to the FreeBSD server. It would run a 'daemon' without any interface, and on your windows desktop you use a graphical frontend to connect to that interface. But the torrent application would actually be running on the server not on your windows desktop.

    As for other things, what are the things you use? I would like to add useful stuff to my ZFS interface which are popular. The mediastreaming to PS3/XBOX should work with MediaTomb, but could you define usenet? You don't mean an NNTP server, do you?
     
  16. sreg0r

    sreg0r Member

    Joined:
    Jul 9, 2001
    Messages:
    1,146
    Location:
    Melbourne
    Yeah, running opensolaris as a host and winXP as a guest OS. I only really use opensolaris for zfs and sharing but everything else I do the VM.

    As for usenet I meant client not server. SABnzbd and Sick Beard are what I use, and although they can be compiled for *nix I just find it easier to install / update under windows and access the programs using their web interfaces on my workstation. I hear utorrent is getting a *nix daemon soon as well.
     
  17. OP
    OP
    fredhoon

    fredhoon Member

    Joined:
    Jun 27, 2003
    Messages:
    2,187
    Location:
    Brisbane
    To update my progress thus far, my ad-hoc hardware is being a pain in the arse. Looking at the cost of getting an 8-port LSI HBA, ECC RAM, possibly a replacement motherboard (and CPU if I go down the Xeon path) and a few other bits and pieces (taking into account the resale value of the 2nd hand unsued gear), I have decided to make do with what I've got.

    This means (unfortunatly) FreeBSD and/or xenserver + unRAID is out. Next attempt will be to run a full slackware buid including unRAID and KVM or VMWare on top. If i can get that working i'll migrate the data, leave it for a few months and when time presents itself play with FreeBSD in a VM along with a few other ideas running aorund in my head.
     

Share This Page