New 10GbE NAS Build

Discussion in 'Storage & Backup' started by jpasint, Jun 21, 2013.

  1. OP
    OP
    jpasint

    jpasint Member

    Joined:
    Mar 4, 2012
    Messages:
    129
    Location:
    Coconut Creek, Florida
    Lol, no rack here!
    But here are a few pics of the build. I never was one to be able to stop the build process to take lots of pics and document my steps.

    Joe

    [​IMG]

    [​IMG]

    [​IMG]


    You can see my real beast in the background!
    [​IMG]

    [​IMG]

    [​IMG]

    And these are the best part.
    [​IMG]

    [​IMG]
     
    Last edited: Jul 5, 2013
  2. Munity

    Munity Member

    Joined:
    Jan 13, 2006
    Messages:
    363
    /needs more money......
     
  3. zero_velocity

    zero_velocity Member

    Joined:
    Sep 16, 2010
    Messages:
    2,858
    Location:
    QLD
    Looks absolutely beasty! I would love 10Gbit in my house to take full advantage of my servers raidz arrays (capable of about 300Mb/s peak).

    That lian li case is only tiny though, hope you dont require any expansion any time soon
     
  4. fuXK

    fuXK Member

    Joined:
    Jun 10, 2005
    Messages:
    551
    Location:
    Melb
    Really not economical to do it.. unless you do transfer file around a lot eg like at lan

    My beast 60TB norco box will eat 10giga alive but spending 1500 for the 10gig setup is not worth the hassle really. 1GB is more then enough for my needs at home
     
  5. OP
    OP
    jpasint

    jpasint Member

    Joined:
    Mar 4, 2012
    Messages:
    129
    Location:
    Coconut Creek, Florida
    Holy crap what do you do with 60TB of storage space?
     
  6. shane41

    shane41 Member

    Joined:
    Nov 10, 2008
    Messages:
    6,979
    Location:
    dice que no sabe
    I think the 10gb setup is awesome, nice speed there OP :thumbup:
     
  7. fuXK

    fuXK Member

    Joined:
    Jun 10, 2005
    Messages:
    551
    Location:
    Melb
    Download the internet
     
    Last edited: Jul 5, 2013
  8. zero_velocity

    zero_velocity Member

    Joined:
    Sep 16, 2010
    Messages:
    2,858
    Location:
    QLD
    Hence why I havent done it. If it were considerably affordable by my standards I wouldnt be saying I want It now...
     
  9. archersaurus

    archersaurus (Banned or Deleted)

    Joined:
    Nov 19, 2012
    Messages:
    321
    Why do you need to spend 1500 for 10gb? if your happy with CX-4 you can do a setup for upto 4 machines on 10gb + 48 gbit ports for around the 700 dollar mark using a Woven switch.
     
  10. BlueSteel

    BlueSteel Member

    Joined:
    Feb 24, 2005
    Messages:
    290
    Location:
    Petrie QLD 4502
    OP: Nice setup you have there.

    I've been holding off on a 10Gbit switch for a while now.. as they are either too expensive or don't have the features I want. I namely want a switch with some SFP+ slots and a reasonable number of 1Gbit ports, that won't break the bank (something like 2-4 SFP+ ports and 24 x 1Gbit ports that I could get for $400-500 would be ideal).

    Currently I've just got 10Gbit running between my desktop and iSCSI box. I got the cards for ~$120 each and the fibre SFP+ transceivers for $50 a pop iirc.

    I do also have 48 and 24 port gigabit switches, however when I use the SFP port on these I am limited to 1Gbit, so these only tend to get used when I host a LAN.

    So currently my storage setup currently is:

    Windows 7 Desktop (iSCSI initiator):
    • Core i7 3930k @ stock
    • 48GB DDR3
    • 512GB Samsung 830 SSD for OS
    • several mechanical disks setup in various ways (varying sizes from 1.5TB-3TB)
    • Neterion X3120 dual 10gbit (SFP+ with fibre transceivesr)

    iSCSI host:
    • Q6600 @ stock
    • 4GB DDR2 800
    • Asus P5Q-Deluxe
    • Neterion X3120 dual 10gbit (SFP+ with fibre transceivers)
    • 8 * 3TB Toshiba 7200RPM (rebadged Hitachi drives) in software RAID6 via MDADM
    • Neterion X3120 dual 10gbit (SFP+ with fibre transceiver)

    Performance is nothing to write home about; I mainly chose mdadm and iSCSI for simplicity (so I can move the drives around to different boxes, without being tied to a hardware raid controller etc.)

    Windows simply sees an 18TB iSCSI device, and treats it as a hard drive (albeit with speeds of 300-400MB/sec, depending on the type of IO etc).
    On the linux host I can get single threaded reads of 600-700MB/sec to the array (with DD and an appropriately block size), so I suspect iSCSI is the biggest bottleneck currently. As a quick benchmark I just copied some 20GB of large files from the iSCSI host to the Samsung SSD and saw 310MB/sec for the duration of the transfer.

    I know I can get much greater throughput from these drives. Individually they benchmark at ~180MB/sec for sequential reads, so I'll probably spend some time optimising or rebuilding the array in the future, but for now it works fine and offers decent enough speed for my bulk storage requirements (namely it just has to be able to saturate a single gigabit link out to the rest of the network)... for now :Paranoid:
     
  11. archersaurus

    archersaurus (Banned or Deleted)

    Joined:
    Nov 19, 2012
    Messages:
    321
    Whats the specs?

    My 72tb box (24x3tb in raid 6 on a 1882ix-24) will do a single sustained transfer of around 1600-1700MB's over 200gb with agg'ed 10gb links (Intel X520 SFP+) on both a Dell 8024 and a Cisco Nexus

    As for 10gb cheap, its not overly difficult to get a 24 or 48 port switch with 2-4 SFP+ slots, just got to hunt for them, The Dell 5524/5548 come with 2 SFP+ ports native and run around 900 bucks brand new from Dell, and you can get the 15% autoclub discount ontop of that price, Dell 6224/6248 can be had for <500 each and the SFP+ modules are <100 bucks each from ebay, which will give you 4 SFP+ modules.

    You then have the Netgear XS078 which is what the OP is using, gives you 8 10g base t and 1 SFP+

    You can also get the HP 1910 for the 200 mark and get the modules for it, however the modules can be exxy, around the 200-300 a pop.

    Just got to watch all the usual sites and jump on them when they come up, they do come up every now and again, just got to watch.

    Im waiting personally for a 8024 POE to come up in the 2k range and ill jump on that, 24 ports of 10g base t (far far more flexible for home use then SFP+) and 4 SFP+
     
  12. fuXK

    fuXK Member

    Joined:
    Jun 10, 2005
    Messages:
    551
    Location:
    Melb
    switch cost 800-900

    Network card? you need 2 which cost around 250-300 each
     
  13. BlueRaven

    BlueRaven Brute force & optimism

    Joined:
    Jul 29, 2010
    Messages:
    4,894
    Location:
    2076
    Nice setup OP. :shock:

    I'm continually amazed at the technological advancement and cost reductions in high speed networking, seeing home server storage capacities and network bandwidth now rivaling enterprise infrastructure of five years ago for a fraction of the cost.

    Amazing stuff. God bless progress.
     
  14. OP
    OP
    jpasint

    jpasint Member

    Joined:
    Mar 4, 2012
    Messages:
    129
    Location:
    Coconut Creek, Florida
    Thanks bro!

    Definitely nothing wrong with those speeds. My last tests where I got the high numbers were with a single 100GB .ZIP file so I'm sure if I broke that down to lots of little files my numbers would have dropped like a rock.
    You must have been pretty amazed (as I was) to see the meter go above the standard 100MBps mark.
    I've been drooling over 10Gb for years.

    Cheers to progress. I remember 2Mb Lantastic coax setups. In fact that was my first network setup at my house.
     
    Last edited: Jul 6, 2013
  15. Grvm

    Grvm (Banned or Deleted)

    Joined:
    Oct 26, 2012
    Messages:
    141
    A woven 48 port was 300 last time I checked and cx-4 cards were <100 each
     
  16. fuXK

    fuXK Member

    Joined:
    Jun 10, 2005
    Messages:
    551
    Location:
    Melb
    if i go 10gig.. i go buy the best not the cheap ones
     
  17. Grvm

    Grvm (Banned or Deleted)

    Joined:
    Oct 26, 2012
    Messages:
    141
    They were 5k+ brand new. It's cheap because its a dead interface cx-4, enterprise has moved to SFP+ or base t.

    Nothing wrong with it, except its cheap.
     
  18. fuXK

    fuXK Member

    Joined:
    Jun 10, 2005
    Messages:
    551
    Location:
    Melb
    Link me to one... u make it like it easy to find one
     
  19. Grvm

    Grvm (Banned or Deleted)

    Joined:
    Oct 26, 2012
    Messages:
    141
  20. BlueRaven

    BlueRaven Brute force & optimism

    Joined:
    Jul 29, 2010
    Messages:
    4,894
    Location:
    2076
    I got all excited a few years ago when I found a stack of decommissioned Alcatel-Lucent 24 port managed switches in the rack room of a large auditing firm that I was doing some work for.

    Since they were no doubt going to be written off, I was tempted to "borrow" a pair of them to test but then realised there were no GBICs in any of them.
    Getting them out of the building probably would have been interesting too.... :lol:

    Ah well, the noise probably would have driven me mad eventually.
    Sad to see such nice hardware sitting around gathering dust though.

    How do you find that Netgear? Reviews/Comments suggest that it's a noisy little bugger, not relevant for many but would probably be a significant factor for home use.
     
    Last edited: Jul 7, 2013

Share This Page

Advertisement: