6Gb/s SAS Expander Cards

Just a heads up, that Chenbro have added two new sas expander cards to their lineup, tho I have not located anywhere to buy them from.

http://www.chenbro.com/corporatesite/products_cat.php?pos=37

 

Intel also have one out - http://www.intel.com/Products/Server/RAID-controllers/re-res2sv240/RES2SV240-Overview.htm

 

good reading for some in the manual.

will explain a lot of SAS stuff if you don't quite understand SAS

http://download.intel.com/support/motherboards/server/sb/e93121002_res2sv240_hwug.pdf

 

Also posted this on hardforum..

So I have had a bit of a play with this card now. in particular duallinking

Please bear in mind, I have never played with duallinking before, nor do I really know anything about it, I was hoping some people might help me understand it better.. and if Im on the right track.

When running a single external cable from my 9210-8i (m1015 flashed to it mode) to the chenbro sas expander's external input, the following is shown using lsi's util.



When running 2 external cables from the 9210-8i to the 2 external connectors on the sas expander (even tho the "Quick Guide" says the 2nd port is Ext. Output to JBOD Unit), the output of the lsi util is shown in the following :



is that all that needs to be done? does this even do anything? I noticed that I am getting a slightly faster read speed from my 20 disk zpool (2x10vdev) up from 1.17G/s to 1.29G/s but I am unsure if that was due to other circumstances, or actually because of the 2 cables plugged in make a difference..

I noticed in the lsi util program that there is an option to enable multipath. when i toggled it on or off (rebooting in between) i noticed no difference at all.

 

Multipath will only benefit you with dual port SAS drives....

 

ok, well i have it disabled again... thanks for that info

http://hardforum.com/showpost.php?p=1037440118&postcount=21

it was this post that had me curious to try this out

and the following thread.. http://hardforum.com/showpost.php?p=1037650065&postcount=38

seems to suggest this person is at the same point I am.

 

Which expander did you get snick and where from?

 

sorry, i should have actually stated that in my post lol

Chenbro CK23601

a company called MXITS is where I ordered it from. I was put onto them by digicor (chenbro distributor).

 

How much was it? If you don't mind me asking?

 

$440 + gst

 

took less than 5 working days to get.. was really fast service, considering how long people seemed to have waited for these

 

Does using expanders mean lower performance, because there are more drives than the actual number of ports on your HBA?

 

not necessarily.. depends on the expander, the hba and to a lesser extent, the hdds used.

 

I guess I don't understand how these expanders work. Say if I have 24 HDDs, is it not reasonable to assume that performance will be lower if I use a 8-port HBA + expander, and higher if I use a 24-port HBA directly?

 

ok think of it this way..

each sas connector has 4 lanes.. each 6G/s i.e. sata III speeds

Sata III provides peak throughput of about 600 MB/s (Megabytes per second) including the protocol overhead (10b/8b coding with 8 bits to one byte).

so that is 4 x 600MB/s = 2400MB/s

Now lets say you have 24 hard drives, all accessing at the exact same time.. now not including the overheards.. this allows for 100MB/s for every hard drive on the sas expander. which is the average speed of most hard drives anyway... sure, some hdds are faster at the beginning of the hdd, and slow down the fuller they are.. so you would potentially be capping yourself at the faster end of the scale..

but lets face it, the chances of runnign 24 hard drives, at the exact same time is pretty rare. some people might, but the majority of people dont.

now.. from what ive read about dual linking.. this actually gives you the bandwidth of 8 lanes from the HBA.. ive read info saying that the 24 hdds are divided between the lanes evenly, and ive read that the bandwidth is just increased as one big pool.. this.. i have yet to understand/find out.

I hope this helps to explain why there is no real noticible slowdown by using a sas expander.. in fact, if you dont use something like ZFS, which uses all the drives at once, and use just a bunch of disks in windows, or something like unraid etc.. chances are, there wouldnt be a single bit of noticible slowdown.

 

Thanks for the explanation. I plan on using ZFS so I guess I'll have to live with slighly lower performance.

 

CK23601 : noisy fan !

Hello,

I just bought the expander CK23601 that is a nice piece of hardware .

But, as soon as powered : i noticed that the heatsink fan is very noisy !!!

But i also noticed the huge overheat.

So, i'd like to know if someone has already tried it fanless or used, instead, a chassis fan ?

Thank you.

Sincerely,

XZed

 

The fan being noisy is an understatement. I do plan, in the future to do something with it to silence it.

I havnet noticed the temperature on the card.. I should grab my infrared thermometer and see what temp its running at.. I havnet had an issue with it tho.. since it pretty much has 2x 120mm noctua fans blowing air straight over the expander (since i only run hdds, psu, sas expander in the norco case)

 

I'm using it w/ a LSI 9260 RAID card in a tower case. You can imagine the heat ...

By the way, the side panel isn't aerated, so i can't extract/blow from/to this bottom part of the case :/ ....

Thank you.

 

Not sure whether this is helpful. But for ZFS using expanders or port multipliers is not the 'smart choice'. More than any solution, ZFS very much desires to have dedicated bandwidth to each disk, and is extremely sensitive to latencies. So it prefers a real HBA controller without any RAID layer, expander, port multiplier, extra bridges, or other chips in between. Just a direct dedicated pipe to each disk would be the best solution.

Expanders also can consume a decent amount of extra energy compared to a plain HBA solution. For example, consider SuperMicro motherboards with an integrated SAS controller so you start with 14 ports. Then add two normal 6Gbps HBAs and you have 16 + 14 = 30 SATA ports. Then you may still have two PCI-express slots free for 10 gigabit ethernet or another controller, etc. Such a solution is often cheaper or about the same price as when using expanders, consumes less energy, and especially on ZFS should perform much better.

For things like UnRAID, sharing bandwidth has the least effect (next to none). So depending on the solution using expanders may or may not be very attractive. Personally I recommend to avoid them, because I can see little benefits. If you already have a motherboard and have limited PCI-express or other reasons why you cannot use multiple HBAs, then expanders might be the only viable solution. Otherwise, multiple HBAs probably are your best choice.