Fileserver Project - 24TB FlexRAID Build

Discussion in 'Storage & Backup' started by VenomousDesigns, Oct 5, 2010.

  1. VenomousDesigns

    VenomousDesigns Member

    Joined:
    Jan 30, 2009
    Messages:
    98
    So after numerous posts across here, HF, FlexRAID forums and alike I can successfully say I have decided that FlexRAID fits what I need perfectly, and now have began the next steps of putting the actually server together.

    Please keep in mind all prices are in AUD - HDDs are already in my procession - Samsung F3 1TB 7200RPMs

    Option 1:

    CPU: AMD Phenom II X6 1055T(2.8GHz) - AM3, 9MB Cache,125W - AUD $220
    MB: Asus M4A89GTD-Pro USB3 AM3 890GX 4DDR3 2PCIE RAID VGA GLAN 2FW ATX HDMI - AUD $195
    RAM: Kingston KVR1333D3E9SK2/4G ValueRAM (ECC Un-Buffered) - AUD $135
    PSU: Corsair HX-850 ATX Power Supply w 140mm Modular Cables - AUD $222
    Controller: SuperMicro AOC-USAS2-L8e - Included with HP SAS Price below
    Misc: HP SAS Expander - AUD $650
    Case: Norco 4224 - AUD $459

    Total: AUD $1880

    Option 2:

    CPU: Intel Xeon E5620 Westmere 2.4GHz 12MB L3 Cache LGA 1366 80W Quad-Core Server Processor - AUD $400
    MB: SuperMicro X8STi-3F - AUD $345
    RAM: Kingston KVR1333D3E9SK2/4G ValueRAM (ECC Un-Buffered) - AUD $135
    PSU: Corsair HX-850 ATX Power Supply w 140mm Modular Cables - AUD $222
    Controller: LSI 1068E 8-Port SAS Controller Onboard
    Misc: HP SAS Expander - AUD $375
    Case: Norco 4224 - AUD $459

    Total: AUD $1930


    RAID Setup:

    As you may have noticed the HDDs are not included in the above costs as I already have them, this is one of the main reasons for my move to create a server as I just do NOT have the room in my Main PC for 20+ HDDs.

    I will be running FlexRAID in a T2+ (RAID6) Array with 21xDRUs and 3xPPUs - I will also have a spare 350GB Samsung running as the main OS drive (Possible upgrade to an SSD in the near future).

    All RAID HDDs are Samsung F3 1TB 7200RPM. My OS of choose for the above will be Windows Server 2008 R2 64Bit


    The Battle:

    To be used as a Fileserver - it will be access by my Main PC and also our HTPC in the living room, along my wife and I to watch/view Films and Photos whenever we like, without having to switch Discs/Risk Damage and ensure with have some sort of redundancy on Family Pictures/Movies.

    This PC will NOT be used for Gaming, Programming or any other sort of work. That is what my Main PC is. I would like to reduce costs as much as possible, but also not to the point where it is unusable for the above tasks.

    Parity generation will be ran on a weekly basis when/if new files are added - will be run during the wee hours of a weekday so not to overlap with anything that might slow performance.


    Conclusion:

    The issue I'm facing is the bloody expense/performance - this thing is ending up more powerful then my Core 2 Duo GAMING PC, and will end up being used solely as a Fileserver. The other option I can add is to wait till the new Sandy Bridge CPUs are released, upgrade my Main PC and retire my current PC into a server...

    Any pointers would be lovely, suggestions, or new builds that are brought up will be considered and advantages/disadvantages of the above all take to heart. Please do contribute.

    Regards,
    VD
     
  2. bobrandom

    bobrandom Member

    Joined:
    Oct 9, 2002
    Messages:
    988
    I got to ask why not zfs raidz2
     
  3. noobmastery

    noobmastery Member

    Joined:
    Dec 23, 2008
    Messages:
    888
    Location:
    Sydney, 2111
    Slightly off topic
    Where are you getting those HP SAS expanders?
     
  4. Quoccus

    Quoccus Member

    Joined:
    Mar 23, 2009
    Messages:
    387
    Location:
    Perth
    where did you get the supermicro I had to buy it from the US :mad:
     
  5. tylerplowright

    tylerplowright Member

    Joined:
    Jun 21, 2010
    Messages:
    603
    Location:
    Newcastle
    get a mountain mods case, they can hold heaps of hard drives. I know of one that will hold up to 32, there is probably a bigger one which will hold even more
     
  6. chancey

    chancey Member

    Joined:
    Jan 5, 2005
    Messages:
    1,415
    Location:
    Sydney
    I hadn't heard about FlexRAID until now, but whats the benefit of using FlexRAID over hardware/software/zfs raid?

    These are hard cases to get in australia. The only store I know that stocks them is techbuy and they won't be getting any more stock for a long while. I ended up buying the 4240 from impatience and its suits my needs better anyway.

    Clearly you need a linux/unix solution.

    This is the part I don't get. The whole point of redundant drives in an array is that if any drive at any moment dies there is a persistent parity block to rescue it. If the parity is deferred then how can you guarantee data integrity at any given time? Parity calculations aren't CPU intensive, I see no good reason why the developer would move them to a later time.

    Tone down the hardware, if its file server it doesn't need very much power at all. A lot of commercial expensive NAS boxes run on intel atom cpus - granted you have a lot more drives but a quad core is over the top.

    I'm sure you've got your reasons for having your heart set on windows, but thats seems a very bad choice to me.
     
  7. noobmastery

    noobmastery Member

    Joined:
    Dec 23, 2008
    Messages:
    888
    Location:
    Sydney, 2111
    I don't see any reason why windows is a particularly bad choice for a file server
     
  8. Dropbear

    Dropbear Member

    Joined:
    Jun 27, 2001
    Messages:
    9,720
    Location:
    Brisbane
    Unless you're serving an office tower, this seems like ridiculously over the top specced hardware to me ..

    Not going to be very power efficient either.

    I can see you spending thousands on this and never wanting to turn / leave it on.
     
  9. OP
    OP
    VenomousDesigns

    VenomousDesigns Member

    Joined:
    Jan 30, 2009
    Messages:
    98
    Couple things to cover:

    1. FlexRAID suits me perfectly - if you are unaware of it, or haven't looked at it for a while, I eagerly direct you to; http://www.openegg.org/FlexRAID.curi - Brahim, myself and a few others have been vigorously testing new BETAs and polishing off new/old features.

    In a nutshell its a perfect system for someone using their Fileserver for static files - for example; Movies, Photos and Data that remains the same/stored. Effectively FlexRAID works by making a 'Snapshot' and everything included in there will/can be recovered.

    For example I have all my Blu-Rays stored on my HDDs, generate a new 'Snapshot', I purchase a new Blu-Ray and want it added. UNTIL I re-generate a 'Snapshot' that new one I've added won't be able to be recovered - which is fine because I'm not adding hundreds of new files everyday and the 'Snapshot' doesn't have to REDO the entire HDDs, only newly added things.

    Please keep in mind this is a 'Real-Time' system and isn't suitable for someone looking for a cheaper/free option.

    2. To answer the question related to attaining the SuperMicro/HP SAS Expander; I have a mate travelling back from the US who can/will be purchasing them for me.

    3. I don't see any issues using a Windows based OS; mainly one can be rest assured Drivers/Compatibility will be good in the way of RAID/HBA Cards. FlexRAID does support Linux, and also looked into ZFS/Solaris and it didn't align with my needs.

    4. In the way of toning down Hardware; would love to here alternative builds/options. Just saying that is a lot easier then making sure everything is going to be compatible/efficient.

    VD
     
  10. chancey

    chancey Member

    Joined:
    Jan 5, 2005
    Messages:
    1,415
    Location:
    Sydney
    Yes, I read this page before making my original comment. That quote above is untrue, if it was real-time then all the data and parity information would be real time - like any other raid solution.

    I still don't see any benefit. If you think there is a real risk that someone in your household will purposely or accidentally delete files and you want the snapshots to help recover then thats even more reason why you should be using *nix with permissions.

    Theres so many reasons why there is a better solution, from cost, stability, reliability, upgradability. Your using expensive (comparatively) server grade hardware to do a task thats incredibly mundane for a computer.

    If your looking for reassurance that you've made all the right choices and someone else with a similar setup will popup and say "Yes, its all been working fantastically for me." Don't hold your breathe.

    I'm not intending to be rude or condescending but the choices you are making across the board are not the wisest. I'm sure everything will in fact work fine, and I hope it does.

    The rule of thumb for a home file server is "spend all your money on the drives" the other hardware is unimportant. A lot of people opt for a cheap AMD, midrange mobo, cheap ram etc.

    Whatever hardware you end up buying, plug it all in, install linux with software raid (that way you don't have to buy an expensive raid card.) Turn on samba and you done.

    Claims like "Faster than many hardware RAID implementations" are bogus for 2 reasons;
    1. Overall hardware raid will always be faster than a software implementation when using it in a generic manner.
    2. It makes no difference if your raid is 500mb/s or 1500mb/s. You are still only using a single ethernet.

    Call me old fashioned or set in my ways but i've been building file, web and database servers for a long time and FlexRAID provides no benefit that I can see.
     
  11. OP
    OP
    VenomousDesigns

    VenomousDesigns Member

    Joined:
    Jan 30, 2009
    Messages:
    98
    I apologise, in my haste it was mean't to read ISN'T a substitute for someone looking for a 'Real-Time' RAID option.


    I'm in no way reading you as rude, I like to hear everyones opinion.

    From the brief look I gave to ZFS I only saw negatives in MY (please read that it says MY haha) situational setup. Apparently you are unable to add single drives, must add new arrays etc etc.

    The KEY advantages for me with FlexRAID are:

    • Reduced power consumption - one is able to have their HDDs power down;
    • Provides the same redundancy of Hardware RAID at a cheaper cost (For Hardware, Power Consumption);
    • Unlike Hardware RAID, if more then the number of PPUs (Parity HDDs fail), you do not lose your ENTIRE array; only lose the additional HDDs after the number of PPUs (KEY SELLING POINT);
    • With FlexRAID HDDs are still able to be 'hot-swappable' - I can give rip out my Photo HDD for example, move data to another PC and simply put in back without the entire RAID array exploding.

    The list goes on...


    As I have all the HDDs atm, I'm both lucky and cursed. The curse is the fact I have no where I can RUN all the HDDs at once to gather/sort/watch something if so desired. I need to hot-swap them.

    The positive is the cost of HDDs can be removed from the Build setup.

    I hear what you're saying about reducing costs, getting low-power hardware, and I'm all for it. As you can see from my AMD Build (Option 1); the CPU, RAM and MB are all 'cheap'. The thing knocking the dam cost up are the ONE OFF HDB + SAS Controller Cards and Norco Case - the rest can be swapped/upgrade as time goes on no - don't you agree?

    If you can find me a cheaper CPU/RAM/MB etc I will take all suggestions on board, and no doubt switch them in if it can be proved to be more beneficial etc. Better yet, if you can show me a low powered Atom/i3/i5 or AMD equivalent Motherboard that has enough PCI-E slots for the HBA, SAS Controller I will happily kiss you :D

    Cheer,
    VD
     
    Last edited: Oct 6, 2010
  12. fredhoon

    fredhoon Member

    Joined:
    Jun 27, 2003
    Messages:
    2,643
    Location:
    Brisbane
    Have you looked into unRAID?

    Also I hear your point about hardware comparability, dealing with issues myself.
     
  13. chancey

    chancey Member

    Joined:
    Jan 5, 2005
    Messages:
    1,415
    Location:
    Sydney
    unRAID isn't without its faults either;
    1. Single parity only, which would be very unwise given the number of disks you have.
    2. Adding a drive or replacing a drive cannot be bigger than your parity drive.
    3. It's slow because it doesn't use striping or mirroring, so even with a bunch of disks you only really get the speed of one disk.
    4. Even the Pro version (which costs $139) only supports 12 disks.

    For the people who don't want or can't afford a hardware raid card, linux software raid and zfs raidz are still the best options. If any other 'semi-raid' product worked as well more people would use them instead.
     
  14. OP
    OP
    VenomousDesigns

    VenomousDesigns Member

    Joined:
    Jan 30, 2009
    Messages:
    98
    Don't want to start a pissing contest here, but can you point out FlexRAIDs fault - love to let you know if they have been corrected in the newer beta builds.

    Was looking around i3/Atom Motherboards last night and have come up with nothing that has enough PCI-E slots to support HBA + SAS Controller. Open for suggestions...
     
  15. chancey

    chancey Member

    Joined:
    Jan 5, 2005
    Messages:
    1,415
    Location:
    Sydney
    I don't know enough about flexraid to get into detail. The overriding point i'm trying to make is that 24TB is a lot of data to entrust to a non mature product. A non mature product that doesn't provide any real world benefit. If you started off with only a few disks and needed flexraid because it was the only solution that let you expand the way you wanted to or there was some special feature of flexraid you couldn't get anywhere else then maybe it would be a good idea.

    I stick with the known methods because I know they work. Often with other products you get a honeymoon period, that is you set it all up and its working and your so happy. Then you start to realise all the faults, the performance sucks, it doesn't do a feature properly like you expected, you start hearing about rare but still prominent data corruption cases, you realise the lack of restore tools if it happened you and you get nervous etc etc.

    I call this the "blue car syndrome." When you buy a blue car suddenly you notice a lot more blue cars on the road. The same theory applying to FlexRAID, unRAID where suddenly you start noticing all the things you didn't think of before choosing it as your permanent product.

    Everyones needs are different, but linux software raid provides the best set of features, the least maintenance and the most reliability overall. If your desperate for snapshots you can have that as well but I wouldn't recommend it.
     
  16. OP
    OP
    VenomousDesigns

    VenomousDesigns Member

    Joined:
    Jan 30, 2009
    Messages:
    98
    More then happy to give it ZFS a stab once I can get over this dam Hardware hurdle haha.
     
  17. doodz

    doodz Member

    Joined:
    Nov 6, 2002
    Messages:
    5,025
    Location:
    Sydney
    Looked at a Intel Server S775 board? Similar to my setup. Works well.
     
  18. fredhoon

    fredhoon Member

    Joined:
    Jun 27, 2003
    Messages:
    2,643
    Location:
    Brisbane
    I totally agree, I was just putting unRAID out there as it has some similar features to flexRAID from my understanding. I too haven't looked into it in any detail, from my limited understanding it handles redundancy in the same manner as WHS. Duplicate copies of data can also be setup under several alternate methods and you still end up with n/2 data capacity (I may have missed the point of FlexRAID with this statement).
     
  19. OP
    OP
    VenomousDesigns

    VenomousDesigns Member

    Joined:
    Jan 30, 2009
    Messages:
    98
    Haha, what a coincidence, I JUST finished reading your thread and it seems you have/went through the exact same issues I am currently facing. Problem is, I'm in Australia so in regards to return polices its GG.

    With that said, I've spent the last 5hrs (at work haha) pulling through hundreds of pages to determine where I want to end up. I went through pulling my hair out, to joy and back again. I've looked at the i3 to i5, to Xeon back to AMD and now back to Intel because frankly, AMD in the way of AM3, server related products and PCI-E is CRAP.

    From suggestions/pieces of information I've picked up I'm looking at the following revised builds:

    Option 1:

    CPU: Intel Core i3 530 Processor LGA1156 2.93GHz 4MB Cache CPU - AUD $122
    MB: Intel S3420GPLC - AUD $250
    RAM: Kingston 4GB(2 X 2GB) DDR3-1333MHZ - AUD $95
    PSU: Corsair HX-750 ATX Power Supply w 140mm Modular Cables - AUD $195
    Controller: SuperMicro AOC-USAS2-L8e (See below - cost included with HP SAS Expander)
    Misc: HP SAS Expander - AUD $650
    Case: Norco 4224 - AUD $459

    Total: AUD $1770


    Option 2:

    CPU: Intel Core i5 650 Processor LGA1156 3.2GHz 4MB Cache CPU - AUD $215
    MB: Intel S3420GPLC - AUD $250
    RAM: Kingston 4GB(2 X 2GB) DDR3-1333MHZ - AUD $95
    PSU: Corsair HX-750 ATX Power Supply w 140mm Modular Cables - AUD $195
    Controller: SuperMicro AOC-USAS2-L8e (See below - cost included with HP SAS Expander)
    Misc: HP SAS Expander - AUD $650
    Case: Norco 4224 - AUD $459

    Total: AUD $1860


    Option 3:

    CPU: Intel Quad-Core Xeon X3440 2.53 GHz, 8MB Cache, 2.5 GT/Sec, Socket LGA 1156 - AUD $265
    MB: Intel S3420GPLC - AUD $250
    RAM: Kingston KVR1333D3E9SK2/4G ValueRAM (ECC Un-Buffered) - AUD $135
    PSU: Corsair HX-750 ATX Power Supply w 140mm Modular Cables - AUD $195
    Controller: SuperMicro AOC-USAS2-L8e (See below - cost included with HP SAS Expander)
    Misc: HP SAS Expander - AUD $650
    Case: Norco 4224 - AUD $459

    Total: AUD $1950

    Basically its the same repeated but with switching the CPUs around. The costs remain so damn high due to the Case, HBA and SAS Card - its a killer but I also guess its a one time investment no? I should effectively be able to move all 3 parts into future Motherboards/CPUs.

    The other thing to consider is this system will now be exceptionally power concise, as opposed to the previous Builds I had listed. I spent a good 2hrs comparing the X6 + ASUS MB and saw they came out ALOT further ahead in regards to Power Consumption, listed are below are for people interested:

    http://www.silentpcreview.com/article1054-page4.html
    http://www.legitreviews.com/article/1414/12/
    http://www.xbitlabs.com/articles/cpu/display/core-i5-760_8.html#sect0

    Other then that it basically comes down to; do I want a IGP to be able to hook this up to my TV in the lounge and get away with no having to build/buy a HTPC (Option 1 + 2) or use it as a dedicated Fileserver and reap the performance benefits of a Xeon (Option 3)?

    Further reading for those interested in Power Consumption in simiular setups:

    http://www.servethehome.com/intel-xeon-x3440-windows-home-server-minireview/
    http://www.wegotserved.com/2010/06/...s-home-server-from-scratch/#IDComment78535181

    And a review of the Intel S3420GPLC Motherboard:

    http://www.servethehome.com/intel-s3420gplc-motherboard-review/


    Questions:

    1. I do also have a question, as the Intel S3420GPL have they're on 'Integrated Graphics', can it still make use of the i3/i5 one?
    2. I've read as much as I can in regards to the Supermicro range of Motherboards vs Intel Server Boards - for example looking at a direct comparison between the SuperMicro X8SIL-F and Intel S3420GPLC only difference I can see is the Intel doesn't have; Remote KVM over IP, Management NIC, nor WebGUI. And for a price jump of approximately AUD $150 I don't see it worth it TBH?

    Interested to hear feedback on the updated builds/power consumption....
     
    Last edited: Oct 7, 2010
  20. Rampage101

    Rampage101 Member

    Joined:
    Jun 27, 2001
    Messages:
    2,033
    Location:
    Country NSW
    Keep in mind there have been reported issues with data (corruption??) with ZFS and SAS Expanders.
     

Share This Page

Advertisement: