6Gb/s SAS Expander Cards

Discussion in 'Storage & Backup' started by snick, May 18, 2011.

  1. snick

    snick Member

    Joined:
    Sep 8, 2001
    Messages:
    114
  2. GoofyHSK

    GoofyHSK Bracket Mastah

    Joined:
    Mar 3, 2002
    Messages:
    1,568
    Location:
    Adelaide Hills
  3. Stanza

    Stanza Member

    Joined:
    Jun 27, 2001
    Messages:
    2,903
    Location:
    Adelaide
  4. OP
    OP
    snick

    snick Member

    Joined:
    Sep 8, 2001
    Messages:
    114
    Also posted this on hardforum..

    So I have had a bit of a play with this card now. in particular duallinking

    Please bear in mind, I have never played with duallinking before, nor do I really know anything about it, I was hoping some people might help me understand it better.. and if Im on the right track.

    When running a single external cable from my 9210-8i (m1015 flashed to it mode) to the chenbro sas expander's external input, the following is shown using lsi's util.

    [​IMG]

    When running 2 external cables from the 9210-8i to the 2 external connectors on the sas expander (even tho the "Quick Guide" says the 2nd port is Ext. Output to JBOD Unit), the output of the lsi util is shown in the following :

    [​IMG]

    is that all that needs to be done? does this even do anything? I noticed that I am getting a slightly faster read speed from my 20 disk zpool (2x10vdev) up from 1.17G/s to 1.29G/s but I am unsure if that was due to other circumstances, or actually because of the 2 cables plugged in make a difference..

    I noticed in the lsi util program that there is an option to enable multipath. when i toggled it on or off (rebooting in between) i noticed no difference at all.
     
  5. Stanza

    Stanza Member

    Joined:
    Jun 27, 2001
    Messages:
    2,903
    Location:
    Adelaide
    Multipath will only benefit you with dual port SAS drives....;)
     
  6. OP
    OP
    snick

    snick Member

    Joined:
    Sep 8, 2001
    Messages:
    114
  7. bobrandom

    bobrandom Member

    Joined:
    Oct 9, 2002
    Messages:
    988
    Which expander did you get snick and where from?
     
  8. OP
    OP
    snick

    snick Member

    Joined:
    Sep 8, 2001
    Messages:
    114
    sorry, i should have actually stated that in my post lol

    Chenbro CK23601

    a company called MXITS is where I ordered it from. I was put onto them by digicor (chenbro distributor).
     
  9. bobrandom

    bobrandom Member

    Joined:
    Oct 9, 2002
    Messages:
    988
    How much was it? If you don't mind me asking?
     
  10. OP
    OP
    snick

    snick Member

    Joined:
    Sep 8, 2001
    Messages:
    114
    $440 + gst
     
  11. OP
    OP
    snick

    snick Member

    Joined:
    Sep 8, 2001
    Messages:
    114
    took less than 5 working days to get.. was really fast service, considering how long people seemed to have waited for these
     
  12. cynix

    cynix Member

    Joined:
    Jun 28, 2001
    Messages:
    6
    Does using expanders mean lower performance, because there are more drives than the actual number of ports on your HBA?
     
  13. OP
    OP
    snick

    snick Member

    Joined:
    Sep 8, 2001
    Messages:
    114
    not necessarily.. depends on the expander, the hba and to a lesser extent, the hdds used.
     
  14. cynix

    cynix Member

    Joined:
    Jun 28, 2001
    Messages:
    6
    I guess I don't understand how these expanders work. Say if I have 24 HDDs, is it not reasonable to assume that performance will be lower if I use a 8-port HBA + expander, and higher if I use a 24-port HBA directly?
     
  15. OP
    OP
    snick

    snick Member

    Joined:
    Sep 8, 2001
    Messages:
    114
    ok think of it this way..

    each sas connector has 4 lanes.. each 6G/s i.e. sata III speeds

    Sata III provides peak throughput of about 600 MB/s (Megabytes per second) including the protocol overhead (10b/8b coding with 8 bits to one byte).

    so that is 4 x 600MB/s = 2400MB/s

    Now lets say you have 24 hard drives, all accessing at the exact same time.. now not including the overheards.. this allows for 100MB/s for every hard drive on the sas expander. which is the average speed of most hard drives anyway... sure, some hdds are faster at the beginning of the hdd, and slow down the fuller they are.. so you would potentially be capping yourself at the faster end of the scale..

    but lets face it, the chances of runnign 24 hard drives, at the exact same time is pretty rare. some people might, but the majority of people dont.

    now.. from what ive read about dual linking.. this actually gives you the bandwidth of 8 lanes from the HBA.. ive read info saying that the 24 hdds are divided between the lanes evenly, and ive read that the bandwidth is just increased as one big pool.. this.. i have yet to understand/find out.

    I hope this helps to explain why there is no real noticible slowdown by using a sas expander.. in fact, if you dont use something like ZFS, which uses all the drives at once, and use just a bunch of disks in windows, or something like unraid etc.. chances are, there wouldnt be a single bit of noticible slowdown.
     
  16. cynix

    cynix Member

    Joined:
    Jun 28, 2001
    Messages:
    6
    Thanks for the explanation. I plan on using ZFS so I guess I'll have to live with slighly lower performance.
     
  17. XZed

    XZed New Member

    Joined:
    Sep 27, 2011
    Messages:
    2
    CK23601 : noisy fan !

    Hello,

    I just bought the expander CK23601 that is a nice piece of hardware :).

    But, as soon as powered : i noticed that the heatsink fan is very noisy !!!

    But i also noticed the huge overheat.

    So, i'd like to know if someone has already tried it fanless or used, instead, a chassis fan ?

    Thank you.

    Sincerely,

    XZed
     
  18. OP
    OP
    snick

    snick Member

    Joined:
    Sep 8, 2001
    Messages:
    114
    The fan being noisy is an understatement. I do plan, in the future to do something with it to silence it.

    I havnet noticed the temperature on the card.. I should grab my infrared thermometer and see what temp its running at.. I havnet had an issue with it tho.. since it pretty much has 2x 120mm noctua fans blowing air straight over the expander (since i only run hdds, psu, sas expander in the norco case)
     
  19. XZed

    XZed New Member

    Joined:
    Sep 27, 2011
    Messages:
    2
    I'm using it w/ a LSI 9260 RAID card in a tower case. You can imagine the heat :)...

    By the way, the side panel isn't aerated, so i can't extract/blow from/to this bottom part of the case :/ ....

    Thank you.
     
  20. sub.mesa

    sub.mesa Member

    Joined:
    Jun 23, 2010
    Messages:
    271
    Location:
    Europe
    Not sure whether this is helpful. But for ZFS using expanders or port multipliers is not the 'smart choice'. More than any solution, ZFS very much desires to have dedicated bandwidth to each disk, and is extremely sensitive to latencies. So it prefers a real HBA controller without any RAID layer, expander, port multiplier, extra bridges, or other chips in between. Just a direct dedicated pipe to each disk would be the best solution.

    Expanders also can consume a decent amount of extra energy compared to a plain HBA solution. For example, consider SuperMicro motherboards with an integrated SAS controller so you start with 14 ports. Then add two normal 6Gbps HBAs and you have 16 + 14 = 30 SATA ports. Then you may still have two PCI-express slots free for 10 gigabit ethernet or another controller, etc. Such a solution is often cheaper or about the same price as when using expanders, consumes less energy, and especially on ZFS should perform much better.

    For things like UnRAID, sharing bandwidth has the least effect (next to none). So depending on the solution using expanders may or may not be very attractive. Personally I recommend to avoid them, because I can see little benefits. If you already have a motherboard and have limited PCI-express or other reasons why you cannot use multiple HBAs, then expanders might be the only viable solution. Otherwise, multiple HBAs probably are your best choice.
     

Share This Page

Advertisement: