"Budget", Quiet, ECC, LGA1155 ubuntu NAS

Discussion in 'PC Build Logs' started by T1tan, Sep 3, 2019.

  1. T1tan

    T1tan Member

    Joined:
    May 8, 2002
    Messages:
    5,950
    Location:
    2605 / caaaaanbwa
    OK I'll start this by saying, excluding HDD's, this can be done for under AUD$650, and with comparatively "lower end" bits, even less. The idea behind this build was to get into ECC RAM territory without forking out a fortune to start up, so I went second hand. LGA1155 xeon related stuff is pretty prominent on the ebay market right now. I used the following websites as my guides:

    Server build guide:
    serverbuilds.net

    More specifically: https://forums.serverbuilds.net/t/nas-killer-4-0-build-guide-125-400/667

    Software Setup:
    linuxserver.io Perfect Media Server NAS

    2016: https://blog.linuxserver.io/2016/02/02/the-perfect-media-server-2016/
    2017: https://blog.linuxserver.io/2017/06/24/the-perfect-media-server-2017/
    2019: https://blog.linuxserver.io/2019/07/16/perfect-media-server-2019/

    My current, ~8yr old QNAP 439 has 4x2tb hdd's, giving me about 5.5tb in usable space, it's an atom with about 400 passmarks. A very-affordable $250 HDD these days is 6tb. Second hand E3 Xeons are cheap as hell right now - between $60-100 shipped. So, I can get about 6-8000 passmarks for pretty cheap and the power management on these CPUs for idle time is pretty good to my knowledge. The hardware is so cheap that buying extra parts for redundancy is feasible. I bought 32gb ECC RAM just so I had redundant modules.

    Desired Capability:
    I wanted the following capability list that begins to depart from low end off-the-shelf NAS's as the list goes down:
    1. ECC RAM - More or less just to keep writes clean.
    2. Quiet - Raid5 is noisier than I'd like. I will spin disks down when they are not in use. This NAS will be used for storage, plex, rutorrent.
    3. Cheap to start out with - I don't want to have to fork out for $1000+ for 4x HDD's just to get started and be forced into RAID1+0. This isn't a business, I'm not loaded, and I'm married. I'm more concerned with a PC that shuts up. And I don't want a 4 bay limit, I want to be able to expand every few months/year with a new HDD.
    4. Easily replaced, cheap parts - If something happens/happened to my qnap hardware, I don't think I'd be able to easily repair it.
    The solution was in the mergerfs/snapraid combo noted in the linuxserver.io links above:
    • MergerFS more or less merges disks to look like one big volume - so disk1, disk2, disk3 can look like mergeddisk.
    • Snapraid more or less manages parity for the underlying disks, so disk1, disk2, disk3 can have a parity disk on pairity1 (and parity2, ... etc).

    Cost:
    As it has gone, I've spent about $1000 give or take, before HDDs. This is because I've decided to get a bunch of slightly more expensive stuff to make it quieter, also some cheap UPS from gumtree. If I were more patient, I could have cut costs down more, but I'm not, so I didn't. Also, people here will be able to source better deals than I as they know what they're doing :D I'm way out of the loop on all this stuff these days (first full build in 15years or so) so I have spent a fair amount of time getting back on the research bandwagon. I hadn't even seen a PCIe slot in the flesh until the other day when the motherboard arrived. I've managed to get by on laptops or OPS - Other Peoples Stuff - for about that time.

    upload_2019-9-3_21-43-47.png

    The above config is good to go for about 14x disks, with a SAS Expansion card down the line it should be good for another lot. I cut down costs by signing up to a free trial of eBay plus, and already being a member of amazon prime, which both allow for free postage a lot of the time. Also, I made use of make-an-offer on ebay, which is surprisingly effective. The one thing this is missing is USB 3.x, I'll fix that with a card and a 5.25" bay slot at some point in the future. My other 5.25" drive bays will be used to mount SSDs. I could have got the motherboard for about $50 or an alternative model for less but this one was in brand new condition and has 4 PCIe ports, and 5x PWM fan controllers, takes ECC RAM, and is micro ATX.

    I spent a little extra money where I thought it would be more useful long term like keeping fan noise to a minimum, and with 5 year warranties on those fans, I expect them to be good for a while. The Noctua fans are replacing stock case fans, so there's savings to be made there if desired. The PSU is a tad more expensive than I'd have bought if sound wasn't a concern. Also, the 32gb RAM allows me to run VMs if required. I'm undecided right now whether or esxi server will run on the bare metal. The 2x UPS for $40 were also hard to ignore.

    A key point with this build that I need to stress is that I can build it over time, so the up-front costs are not so hurty to the wallet. I sourced these parts over 2-3 months as a bit of a side project.

    I will eventually put this into a 12u Open Rack that will become an ikea desk leg @ 710mm high on castors.

    The Initial Disks:
    Next Disks:
    • tba: 1x 10tb shucked disk for snapraid parity disk #2
    • as required.

    The HDD array will be managed by mergerFS. Snapraid will control the parity stuff. See the linuxserver.io stuff above for more detail. Really this combination is just an economical way of getting HDDs to look like one array (mergerfs) and to provide parity of the underlying disks (snapraid). There is no striping of bits in mergerfs - it's more or less a mask that sits above your array of hdd's and full files get written to one hdd or another. Disks can be added or removed as required, with or without data present. Interesting stuff.

    To keep noise down
    • The OS will run on SSDs, and
    • Temporary downloads etc will run on the 2tb SSD.
    • Media will be copied to HDD with the most free space (managed by mergerFS)
    • Spindle HDDs will spool up every monday for a parity check, and
    • When nobody is home HDDs will spin up for a Plex library update.
    • HDDs will spin down after ~90mins of inactivity. Spin up delay is acceptable.
    • Any constant read/write, small file requirements etc will be done with SSDs.
    Keep in mind that this is the plan, not necessarily what will eventuate. I'm trying to figure out a way of using symlinks for plex (symlinks to sit on the SSDs) but I'm not sure this is possible.

    To do:
    Case purchase is coming next week.
    Build will begin shortly after.

    Updates and pics to follow. :) I put my decent phone in for repair last night, it won't be back for a week.
     
    Last edited: Sep 3, 2019
  2. demiurge3141

    demiurge3141 Member

    Joined:
    Aug 19, 2005
    Messages:
    1,606
    Location:
    Melbourne 3073
    This makes little sense, data should be on ZFS, not the OS.
     
    wwwww and T1tan like this.
  3. OP
    OP
    T1tan

    T1tan Member

    Joined:
    May 8, 2002
    Messages:
    5,950
    Location:
    2605 / caaaaanbwa
    Cheers! Happy to take feedback on board. :)
     
  4. terrastrife

    terrastrife Member

    Joined:
    Jun 2, 2006
    Messages:
    18,626
    Location:
    ADL/SA The Monopoly State
    honestly it sounds like unraid would suit you better for low noise low power as it only needs to spin a single disk for media playback and has basically everything you want to do (in your "To keep noise down" list) built in.
     
    Last edited: Sep 4, 2019
  5. garnet

    garnet Member

    Joined:
    Nov 10, 2009
    Messages:
    366
    Location:
    3020
    perhaps looking LGA2011, those chinese x79 are getting pretty decent.
     
  6. wwwww

    wwwww Member

    Joined:
    Aug 22, 2005
    Messages:
    5,273
    Location:
    Melbourne
    Don't whitebox it if reliability is any concern. You could build a file server with greater expandability, performance, reliability for cheaper with old enterprise equipment, e.g.:

    DL380 G6 - $50 (you can find these for free if you look hard enough)
    2x Xeon X5650 because you need 12 cores/24 threads - $40
    48GB(12x4GB) DDR3 Reg Ecc - $100
    Taotronic Active noise cancelling headphones - $70 (I just googled these, no idea if these are sufficient to block out the noise but you can always get better ones)
    = $260

    Add in P822s (about $80 each with a cache and BBU) as your storage needs increase (you can add in up to 6). Each card is good for 3.4 petabytes so maximum expandability of the system is about 20 petabytes.
     
    T1tan likes this.
  7. OP
    OP
    T1tan

    T1tan Member

    Joined:
    May 8, 2002
    Messages:
    5,950
    Location:
    2605 / caaaaanbwa
    danyell likes this.
  8. wwwww

    wwwww Member

    Joined:
    Aug 22, 2005
    Messages:
    5,273
    Location:
    Melbourne
    danyell and terrastrife like this.
  9. ae00711

    ae00711 Member

    Joined:
    Apr 9, 2013
    Messages:
    1,448
    utter bullshit
     
    terrastrife likes this.
  10. terrastrife

    terrastrife Member

    Joined:
    Jun 2, 2006
    Messages:
    18,626
    Location:
    ADL/SA The Monopoly State
    yup, with the exception of the bad cap era, everything is reliable. Trying to repurpose old hardware not designed for the current task is pretty stupid imo, massive wast of power and noise for starters.

    I've been running unraid for over a decade on outdated desktop hardware, while also not suited to the task, it's the specific hardware designed for the task. At first it was AM2, currently B75, and I'm looking to retire my desktop which is X99. It's not often it gets upgraded.
     
    Last edited: Sep 8, 2019
    T1tan and ae00711 like this.
  11. OP
    OP
    T1tan

    T1tan Member

    Joined:
    May 8, 2002
    Messages:
    5,950
    Location:
    2605 / caaaaanbwa
    I'm rolling with supermicro / xeon for reliability and redundancy (or the perception of). I'm specifically not getting old racked servers as noise isn't a consideration in the design. Can't really work around that, also, the depth is usually more than I'd like to accomodate.

    I did consider unraid, i tried almost all nas options in virtual, but i couldn't try out unraid. Is it easy enough to run containers?

    Going to order the case on tuesday night. Then it should be build time.
     
    Last edited: Sep 8, 2019
  12. terrastrife

    terrastrife Member

    Joined:
    Jun 2, 2006
    Messages:
    18,626
    Location:
    ADL/SA The Monopoly State
    Dockers and VMs are literally 1-3 clicks max operations.
     
  13. demiurge3141

    demiurge3141 Member

    Joined:
    Aug 19, 2005
    Messages:
    1,606
    Location:
    Melbourne 3073
    Except onboard NICs. Get an Intel network card. The realtex onboard crap will lose it during sustained transfers.
     
    wwwww likes this.
  14. OP
    OP
    T1tan

    T1tan Member

    Joined:
    May 8, 2002
    Messages:
    5,950
    Location:
    2605 / caaaaanbwa
    In that case I'll give it a look at before I head down the ubuntu route.
     
  15. terrastrife

    terrastrife Member

    Joined:
    Jun 2, 2006
    Messages:
    18,626
    Location:
    ADL/SA The Monopoly State
    I've never had this issue, not even when using a combination fo Realtek and Inel nics in a LACP with constant usage and hundreds of open connections under Windows.

    I often do transfers of around 1TB to my NAS no hiccups there either considering it's a way lighter load.
     
  16. die_piggy

    die_piggy Member

    Joined:
    Feb 18, 2008
    Messages:
    631
    +1 for Unraid, my box now hosts over 70TB across 14 Hdd's with very few issues
     
    danyell likes this.
  17. ae00711

    ae00711 Member

    Joined:
    Apr 9, 2013
    Messages:
    1,448
    There's a G7 going for $175 in the sale section
     

Share This Page

Advertisement: