OpenSolaris/Solaris 11 Express/ESXi : BYO Home NAS for Media and backup images etc

Discussion in 'Storage & Backup' started by davros123, Dec 14, 2009.

  1. davros123

    davros123 Member

    Joined:
    Jun 18, 2008
    Messages:
    2,837
    NOTE : The below in way out of date now...check out the later posts for more recent info...ie. I am now running Solaris 11 Express under ESXi.


    See here for the new setup....http://forums.overclockers.com.au/showpost.php?p=13492597&postcount=544

    Hi guys,
    After reading myself into confusion, I though I'd ask my question here. Yes, I know there have been other threads on a home NAS, but as they say in the army, there may be many like it, but this one is mine.

    So, my requirements are for a "pure" NAS with the following attributes:
    * Build it myself from appropriate "old" parts - I have a number of "spare PC's" about and will augment the 6 onboard SATA ports with PCI cards + operate on a Gigabit LAN
    * Fault tolerant - I need a level of redundancy where I can sustain a single HDD failure (I think that's raid 5)
    * Dynamically grow the volumes - either by swapping larger HDD's (2TB) or adding an extra HDD (I can accept downtime during this operation)
    I will initially use 5 x 1.5TB drives and dynamically grow from there.
    * Software Raid (cards are too expensive for my needs)
    * Serving 6+PC MS Windows PC environment (XP + 7) for media + HDD backup images etc. (Potentially at some later stage IP security cams. although they may remain local to that server due to volume of data?)
    * Operation will be 24x7 - but as it's a home NAS, some downtime for maintenance is acceptable.
    * Real time alerting/reporting on disk issues (eg. SNMP or email via SMTP)
    * Will be serviced by an appropriate UPS
    * Do not require hot standby HDD.
    * Prefer a web interface to a CLI interface - My experience with linux is limited but it gets me by so I guess a user friendly linux is ok...but Solaris may be stretching it.
    *Set and forget solution although I do want a strong support community.
    * Free

    Having seen some other threads degrade, I feel I need to add please be kind and no thread crapping.

    Suggestions for solution welcome.

    EDIT: And here is the final outcome.

    Click to view full size!

    OpenSolaris 2009.6
    I now have a new LianLi case (PC-A71A),
    8G ECC ram on an Intel S3210SHLC mobo.
    Q6600Xeon 3370 (3Ghz Quad)
    2 x 8-Port SUPERMICRO AOC-USASLP-L8I UIO SAS
    Corsair HX-850
    2 x 250G laptop drives in a mirrored rpool (ie. boot disks)
    8 x 1.5TB WD Green EADS in raidz2
    1 x 1TB WD Green for dvr duties.

    14 x 1.5 TB HDD's in 2 x 7 disk raidz2 vdev's.
    2 x 40Gb OCZ Vertex 2 in mirror as ZIL and L2ARC


    If you have ever used any sort of linux/unix, you'll find OpenSolaris easy to use and actually very friendly...heck most of the time I do not notice any real diff. between this and Ubuntu or CentOS etc. after all it uses Gnome so looks and feels the same.

    Click to view full size!


    The only requirement I had to compromise on was the dynamic growth...I can certainly swap out the HDD"s with 2TB drives, but need to do all 8 to get the increase...In reality, i would not want the single array to be larger than 8 drives, so I'm happy with this compromise.

    Here are some links I used to help me install and config. opensolaris....again, they may not be of much relevance to you...but they were to me :)

    SETTING UP AN OPENSOLARIS NAS BOX: FATHER-SON BONDING - a very simple/easy guide..but was all I needed to get a ZFS server up and running.
    A Home Fileserver using ZFS - A comprehensive guide to building an opensolaris nas. Highly recommend this one.
    zfs tutorial part 1
    Setting up a static network configuration with NWAM
    How do you configure OpenSolaris to automatically login a user?
    How to Set Up Samba in the OpenSolaris 2009.06 Release - Information Resources - wikis.sun.com
    Getting Started With the Solaris CIFS Service
    Mount NTFS / Ext2 / Ext3 / FAT 16 / FAT 32 in Solaris and also here - note 2G limit.
    Seven Useful OpenSolaris ZFS Home Server/NAS Tips <--- highly recommend looking at this.
     
    Last edited: Jan 4, 2012
  2. Rampage101

    Rampage101 Member

    Joined:
    Jun 27, 2001
    Messages:
    2,032
    Location:
    Country NSW
    Basically just done that (well I re-did my server anyway)

    * Running Centos 5.4
    * Setup Webmin and Samba (Webmin for web management of machine, add users, change services etc)
    * Xen (Running Windows 2008 R2 Server for uTorrent :rolleyes:)
    * Onboard SATA running 4x750gb Samsungs in RAID5 (can have monitoring but I haven't set that up yet) using MDADM.

    I think it pretty much covers all your points.
     
  3. greenhawk

    greenhawk Member

    Joined:
    Dec 18, 2003
    Messages:
    789
    That pretty much removes most of the items from the market as they are off the shelf units.

    raid 5 can do it, but it can cause issues with expanding the storage space. Another option is to just use lots of raid 1 paired drives. Or use windows home server (does something similar to raid 1 at the file level for data protection).

    That is where motherboard raid and most software raids fall over (exception being linux). Most of these would need a backup / distory / recreate to expand.

    can be a good way to keep proces down, but it effects the dynamic growth needs

    if just being used to serve data, then it does not matter what is on the network. with windows home server it can help with doing backing up window systems.

    not sure of what can do that nativly, but I suspect that there several add-ons that could be used.

    that basically only leaves existing copies of windows or some form of linux.
     
  4. jmannik

    jmannik Member

    Joined:
    Jun 10, 2005
    Messages:
    731
    Location:
    Albury, NSW
    By the look of your requirements it looks like freenas would be a good fit for you, web based management and all the features you want.

    http://www.freenas.org/

    Have a look.
     
  5. OP
    OP
    davros123

    davros123 Member

    Joined:
    Jun 18, 2008
    Messages:
    2,837
    Thanks Rampage101
    Thump...that's overkill me thinks ;) But hey, each to their own. Thinking about it thiough perhaps I'll run up an XP on there to give me the flexability to run some windows app's....hmmmm....that can be rekease 2.0 of the NAS.

    I am not familiar with Xen...but it looks very solid...and there is even a wiki on how to install it under CentOS...however I have a feeling I might need more help that that.

    Thanks for the webmin suggestion...I just had a look at it and it's nice...they even have a demo login (in getman or something but looks the business).
    They also have step by step instructions on how to install samba on the webmin site...I am liking this even more!

    mdadm was what I thought would be the best option...can easily grow the array. Good.

    greenhawk,
    Thanks for the ply. I understand that mdadm can dynamically grow arrays...I came across this example when googling.

    jmannik,
    Thanks for the link to freenas. I did look at that but I understood that it was end of life....but I found this which indicates it will change, but live on...sort of. Do you know if it does everything I need...esp. the dynamic growing of arrarys? (it does have ZFS, but that does not currently allow dynamic growing).


    I want to go home and set it up now! But before I do...

    3 Questions:
    1) Rampage101 - Do you have a separate physical drive for the o/s?
    2) Rampage101 - What throughput speed are you getting to the array (over gigabit)? I heard some conflicting opinions re. performance of this setup.
    3) All - What happens if the mobo/hardware fails and I need to replace it...can I remount the array on diff. hardware? (ie. say I am running an AMD Athlon 2500+ and it fails and I then replace it with a Core2.
    a) will I need to rebuild/reinstall the o/s
    b) can I easily remount the array on the diff h/w?

    Dav
     
  6. Herballizard

    Herballizard Member

    Joined:
    Oct 9, 2002
    Messages:
    1,533
    Location:
    .
    Last edited: Dec 15, 2009
  7. Rampage101

    Rampage101 Member

    Joined:
    Jun 27, 2001
    Messages:
    2,032
    Location:
    Country NSW
    OK,
    1. Yep, CentOS is running off an 80gb Sata.
    2. Throughput so far is better than my old windows server, getting average of 60mb/s from old seagate 7200.7 320gb to the array, and about the same back from it. Keep in mind, the 320gb I believe is slow (havn't had a chance to test my sammy backup drives) and the array is 3/4 full. Running a dd test to create a 8gb file I got 110mb/s or thereabouts.
    3. Supposedly, I had my array all setup on Ubuntu 9.1 but hated many little things about the O/S, so I formatted the 80gb and installed CentOS, picked up the array no worries. I've booted up with extra sata drive attached which moved 1 of my raid drives to another letter (ie. /dev/sdb to /dev/sdc) and the raid autodetected fine. I'm intending to move my array onto a Supermicro UIO 8 port SAS card as soon as it and some cables turn up.
     
  8. OP
    OP
    davros123

    davros123 Member

    Joined:
    Jun 18, 2008
    Messages:
    2,837
    Rampage101,
    Thanks for that. Absolutely brilliant!

    Re. the Supermicro UIO 8 port SAS....just doing some digging and it's looking good...we might have NAS clones :)

    Assume you got the cables from deal Extreme? That's pretty cheap!

    Is this theone? It says raid card, but I assume they can do non-raid (with raid in software)?

    Other than e-bay, do you have advice on a best source for the card?

    Ok, I think that's all for the questions...I have pestered you enough.

    I might just go home and start putting together a test server tonight. Just to have a play with...just hope the wife does not want to use her PC in the next few days :D

    Thanks so much for taking the time to help me out.
    Dav
     
    Last edited: Dec 15, 2009
  9. Rampage101

    Rampage101 Member

    Joined:
    Jun 27, 2001
    Messages:
    2,032
    Location:
    Country NSW
    Thats definetly the card I ordered. I wanted low profile in case I go with a 2RU case later on. In terms of it being a raid card, in "IT" mode is supposedly just acts as a controller afaik.
    With the cables, I ordered 2 off ebay, about $16 each. SFF 8087 to 4x sata.

    I'm still waiting on both of the above before I know for sure how they all work and how well they work. But I'll let you know.
     
  10. ice_cool

    ice_cool Member

    Joined:
    Nov 9, 2005
    Messages:
    2,845
    Location:
    Sydney
    forgive me if this question is retarded (i don't really know that much about linux/xen), but here goes anyways....

    ok i have a windows 2003 server running for WSUS services at home.
    Now i want to create a NAS/file server, and after doing some reading it seems that Samba is ideal for my needs. would it be possible to run samba and then run windows server 2003 inside a Xen to provide my WSUS service?
    Also if i run utorrent inside my ws2003, what is the best way to transfer data over to the samba file server? (Just over the network?)

    also sorry mate for hijacking your thread
     
  11. OP
    OP
    davros123

    davros123 Member

    Joined:
    Jun 18, 2008
    Messages:
    2,837
    :D All are welcome.

    If I understand what you are saying...it is exactly what Rampage101 is doing (and what I intend to test/play with once I find a freakingHDD in my ancient collection that's not borked! Freaking ancient 30GB HDD's! ::upset:)

    You install CentOS, then install Xen (or other Virtualisation software) and create a VM running Windows Server 2003 and install whatever you want inside this VM.

    So, in short, yes.
     
  12. Rampage101

    Rampage101 Member

    Joined:
    Jun 27, 2001
    Messages:
    2,032
    Location:
    Country NSW
    Yep, although I have since found out that Xen has given me performance problems, and when trying to upgrade to 3.4.2 it breaks the full virtualisation capabilities (kernel boots with no VMX/SVM flags).

    I have since moved to qemu/kvm and am running a Smoothwall VM for my internet server, and also the 2008 Server for uTorrent and WSUS (when I get is setup again). My previous server ran 2008 Server with VMWare Server running the Smoothwall VM.

    One thing to note with WSUS, you cannot store the updates on a mapped drive (unless there is a hack etc). I've made a VHD in 2K8 disk management and stored it on a mapped samba share (ie the raid array). I'm hoping I don't have to re-download 35gb of updates :shock:
     
  13. ice_cool

    ice_cool Member

    Joined:
    Nov 9, 2005
    Messages:
    2,845
    Location:
    Sydney
    cool. thanks for the advice!

    I might keep coming back to this thread
     
  14. OP
    OP
    davros123

    davros123 Member

    Joined:
    Jun 18, 2008
    Messages:
    2,837
    Quick update on how this has been going for me...

    * Installed CentOS 5.4 32 & 64 a few times and finally selected 32 as I am unlikely to use over 4GB and it's possibly more compatible.
    * Installed to a old AMD 3500+(?) for a test, played about and eventually got moth things running (Samba, mdadm and looked into Elastix)....Thought I might want to do a VM at some stage later on, so played with setting up a VM... Discovered it did not do virtualisation (VT), so then moved to my newer E8400 C2D rig...but had issues with the BIOS not allowing VT even thought the CPU does...(many hours spent on that one!).

    There went a day and about 10 installs.

    Decided to go with a whole new rig - bought a E6300($83) and 2GB ram($54) and matched with a P5Q Pro I had laying about and other bits. Added 3x 1.5TB WD Green @$140ea to kick off the Raid5 array(gee they run cool!) and will add another 2 after playing about a bit. An hour to format and all good...until I discover that the Atheros nic does not work under CentOS...so grab a spare 10/100 nic and install that, find the package, install etc etc....Ahhh, the joys of Linux.

    Ok, now tried to create raid5 array and dicovered that the partitions were in use as I has the disks connected when I installed the o/s and it seems to have claimed them as it's own...reinstall #5 billion :~ in progress.

    I now understand why people go out and buy a Nas :) But hey, what else would I be doing if not this...Oh yeah, time to put the washing out.:D
     
    Last edited: Dec 19, 2009
  15. OP
    OP
    davros123

    davros123 Member

    Joined:
    Jun 18, 2008
    Messages:
    2,837
    Hmm....I think I have some network tweaking to do...

    I am getting 20-35MB/s over Gigabit to either of my 2 Windows PC's (One W7 and the other XP SP2) - all using onboard LAN.

    dd test shows decent numbers (away now so can not recall the exact number)...so LAN is the issue.

    I think I get 40's between the PC's....will do some more tests and install CentOS on the other PC and see what speed I get and then try some direct connects...different cables etc.

    I'll get there.
     
  16. Rampage101

    Rampage101 Member

    Joined:
    Jun 27, 2001
    Messages:
    2,032
    Location:
    Country NSW
    My Supermicro Card turned up Friday, but no cables:mad:

    As to performance, with my 4 drive raid, my network so far can sustain high 60's. I haven't done any tweaking at all.
     
  17. Bthree

    Bthree Member

    Joined:
    Jan 5, 2002
    Messages:
    82
    Location:
    Country Vic 3500
    At the risk of repeating myself .... look at Windows Home server

    WHS will do all you want and very simple to use

    Great for backing up your 6 PC's

    Just add another HDD when full and selectively duplicate shares

    Worth a look and will run on any decent hardware ( I am using an atom 330 with 1gb ram and two 1TB drives )
     
  18. Rampage101

    Rampage101 Member

    Joined:
    Jun 27, 2001
    Messages:
    2,032
    Location:
    Country NSW
    At the risk of repeating the OP,

    • Free

    :shock:
     
  19. OP
    OP
    davros123

    davros123 Member

    Joined:
    Jun 18, 2008
    Messages:
    2,837
    Thanks guys appreciate the support..

    The NAS is operational now (after rebuilding the array overnight following my muching about) and all working well, the issue at the moment is the network throughput....which I suspect is the nic's (and perhaps some Samba overheads...may look into getting NFS working on Windows...but that's another day).

    Bthree, I am keen to keep at the linux angle as I also run an Asterisk server (Elastix also on CentOS distro.) and I plan to integrate it into the one PC.

    Rampage101, I'd be happy with 60 as that's close to the limit of my PC's (non-raid) HDD's anyway.

    I've installed iperf and will do some more testing. At the moment the best I can get is ~40...I suspect the onboard nic's are the issue (crappy Realtek and Atheros on the nas).

    I've posted over in the networking forum re. this and it might be time for some Intel Pro 1000 nic's...just do not know which to buy or how to best configure them.
     
    Last edited: Dec 21, 2009
  20. Herballizard

    Herballizard Member

    Joined:
    Oct 9, 2002
    Messages:
    1,533
    Location:
    .
    Have you got room to smack in some decent intel cards that support teaming, if the rest of your gear supports it and you can confirm thats the bottleneck.

    Write Cache enabled by any chance
     

Share This Page