1. OCAU Merchandise is available! Check out our 20th Anniversary Mugs, Classic Logo Shirts and much more! Discussion in this thread.
    Dismiss Notice

RAID for OCAU's new server

Discussion in 'Storage & Backup' started by Agg, Aug 11, 2003.

  1. Agg

    Agg Lord of the Pings

    Joined:
    Jun 16, 2001
    Messages:
    33,774
    Location:
    A Reported Post Near You
    OCAU's new server is taking shape and I am currently pondering the storage subsystem. Currently we are on a single 10k-rpm U160 drive and as you've no doubt noticed, at about 10pm most weeknights the system slows to a crawl.

    Firstly: what's a good way to see if the hard drive is a bottleneck, on a running Linux system? It seems likely to me that it is, with many small random updates to the huge forum database swamping the disk channel. Keep in mind that even just viewing a thread causes a write to the database, with the thread view counter being incremented and Who's Online being updated etc.

    Secondly, assuming the disk subsystem is a bottleneck, what should we upgrade to? The motherboard we have in mind supports a Zero Channel Raid upgrade. From what I have found in my wanderings around the net it seems that a RAID0+1 (or RAID10, or RAID0/1, they all seem to be the same thing) array is ideal. 2 10k-rpm U320 drives striped, then that array mirrored on another two drives.

    Does anyone have any good links (I've googled without much success) for benchmarks of the various RAID configurations? Does RAID10 read from all four disks in this configuration or only two? I assume writing is done to both arrays almost simultaneously.

    Anyone got any experience and other suggestions of what config we should run for a random-write-intensive database and general webserver?
     
  2. AlanC

    AlanC Member

    Joined:
    Apr 9, 2002
    Messages:
    60
    Location:
    Melbourne
    I think if you are using Raid10 with 4 x 10K-rpm HDD. PCI64 is the only to go.

    Looking at testing done by "Chainbolt" Western Digital Raptor 10k-RPM HDD in RAID 0x4.It reach speed of 105MB/Sec. Only 2XMB/sec for network....etc.
     
    Last edited: Aug 12, 2003
  3. Audiobuzz

    Audiobuzz Member

    Joined:
    Jun 27, 2001
    Messages:
    952
    Location:
    Adelaide
    My understanding of raid 1 (mirror) is that it stripes the reads across the sub units but it may depend on the controller. It'd be silly not to really as it's free bandwidth :)

    AB
     
  4. eva2000

    eva2000 DDR1/DDR2/DDR3 Addict

    Joined:
    Jun 27, 2001
    Messages:
    21,959
    Location:
    Brisbane, Australia
    raid 1+0 would be ideal compromise but even raid 5 is good, vBulletin.com itself is on a single dual p3 1.4ghz tualatin with 2-3GB memory and 3x 36GB 10k scsi in raid 5 and it handled a new record 1400 guests online recently running vB 3 beta 5 of course :D

    vB 2.3.x branch wouldn't be able to handle that many guests + the regular members due to as Agg mentioned, the thread view updates, counters and who's online writing constantly to the database and eventually table locking the database.
     
  5. BlueSmurf

    BlueSmurf Dream it. Build it.

    Joined:
    Aug 2, 2001
    Messages:
    2,374
    Location:
    Baldivis, WA
    Allow me to offer some advice. :D

    If you'd care to peruse this document: http://members.iinet.net.au/~winndyne/images/forums/SCSI/RAID Benchmarks.doc

    You'll find the IOmeter testing results I colated from a benchmarking exercise a couple of years ago. It was done on a Compaq single channel RAID controller with a bunch of 18GB 10k RPM disks. Varying RAID types, varying tests and varying numbers of disks. I didn't get into the cluster/stripe size tweaking and you could probably improve those figures but it gives you an even comparison between the different arrays.

    The testing was done to look at the most effective 4 disk arrays, IIRC, and the green highlights the best performing 4 disk array in any category and the Red the worst. You'll note that a 4 disk 0+1 comes out on top most of the time, often by a big margin.

    For OCAU's purposes, consider the OLTP figures, probably the 2k or 4k as forum useage, and the file server as an indicator for web. IOmeter has a web benchmark but we weren't building web servers at the time. :)

    So my advice is 4 disks, RAID 0+1. The number of channels on the controller isn't critical with SCSI, especially for small reads/writes but obviously more is better. Also don't get hungup on U320 over U160, won't make much of a difference in this application.

    If the new server supports hotplug SCSI - I have 3x 18GB 10k RPM disks leftover from this sale which I'm prepared to donate to OCAU if you want them.

    Cheers,

    BlueSmurf.
     
  6. =BF=

    =BF= Member

    Joined:
    Jun 27, 2001
    Messages:
    1,238
    Location:
    BrisVegas
    Raid 10 is a pair of mirrored disks mirrored on a striped pair.
    Raid 0+1 is a stripped pair mirrored on a stripped pair.
    These diagrams show it better.
     
    Last edited: Aug 12, 2003
  7. saba

    saba Evil Vizier

    Joined:
    Aug 4, 2001
    Messages:
    2,709
    Location:
    Melbourne
    My IOMeter tests at work also confirm that ZCR is a POS. Have to invest in a real controller I'm afraid.
     
  8. OP
    OP
    Agg

    Agg Lord of the Pings

    Joined:
    Jun 16, 2001
    Messages:
    33,774
    Location:
    A Reported Post Near You
    Yes, that's not in dispute..?

    Thanks for the offer, but they may be too small for our needs. I had decided on the controller but this thread has made me undecided again. We were planning on using this card:

    http://www.adaptec.com/worldwide/pr...-2010S&cat=/Technology/Ultra320/Ultra320+RAID

    But if ZCR sucks as much as people in this thread say, we will have to pursue another option. Upaboveit/Saba, what cards are you using for your benchies? Could the poor performance be card-specific?

    Yes, but cost is a concern. Based on this pricing I guess we could move from our current Seagate 10k plan to the Maxtor 15k, but it's ending up at over $2200 for the 4 drives.
     
    Last edited: Aug 12, 2003
  9. eva2000

    eva2000 DDR1/DDR2/DDR3 Addict

    Joined:
    Jun 27, 2001
    Messages:
    21,959
    Location:
    Brisbane, Australia
    For Aussie vendor, have you checked out Secretnet's scsi drive pricing ?

    For controllers, maybe Schwein, Upaboveit or Voodoo could hook you up or recommend something ?

    what about these babies ?

    Mylex AcceleRAID 400 PCI-X Ultra320 SCSI RAID Controller w/32MB DDR RAM
    http://scsi4me.com/?menu=menu_scsi&pid=3082

    Mylex AcceleRAID 500 PCI-X Dual Ultra320 SCSI RAID Controller w/64MB DDR RAM
    http://scsi4me.com/?menu=menu_scsi&pid=3078&display=Mylex500.htm

    Mylex AcceleRAID 600 PCI-X Dual Ultra320 SCSI RAID Controller w/ 128MB DDR RAM
    http://scsi4me.com/?menu=menu_scsi&pid=3079&display=Mylex600.htm

    edit: didn't realise Pluscorp's 10k scsi were cheaper, but Secretnet's 15k seagate are cheaper hehe

    But really vB itself should do well on 10k scsi drives.. particular if you decide to separate to 2 server setup rather than 1 server for web and database
     
    Last edited: Aug 12, 2003
  10. Doso

    Doso Member

    Joined:
    Jul 16, 2002
    Messages:
    6,079
    Location:
    [Vic]Melbourne
    do you have any price ristrictions at all?? or can we end up spending 10k +
     
  11. OP
    OP
    Agg

    Agg Lord of the Pings

    Joined:
    Jun 16, 2001
    Messages:
    33,774
    Location:
    A Reported Post Near You
    I can't buy through you, Upaboveit.. the majority of this server is being provided by PlusCorp with OCAU buying a few extra bits (from them). So we have to go off PlusCorp's pricing. Doso - it'll be cost pricing anyway so cheaper than the website prices, but still, we are not of infinite resources. :)
     
    Last edited: Aug 12, 2003
  12. BlueSmurf

    BlueSmurf Dream it. Build it.

    Joined:
    Aug 2, 2001
    Messages:
    2,374
    Location:
    Baldivis, WA
    What's wrong with it?

    Single disk, same redundancy as an (n) disk RAID5 or a 2 disk RAID1 array.

    Just how many drives are we really expecting to fail simulataneously and how long with the box need to operate before a replacement is sourced?

    It's all about risk vs. $$ - in the extremely unlikely event of two drives failing within the replacement window, Agg does have the video tape replay he can go back to.

    BlueSmurf.
     
  13. BlueSmurf

    BlueSmurf Dream it. Build it.

    Joined:
    Aug 2, 2001
    Messages:
    2,374
    Location:
    Baldivis, WA
    It's obvious. Performance. ;)

    BlueSmurf.
     
  14. Iceman

    Iceman Member

    Joined:
    Jun 27, 2001
    Messages:
    6,647
    Location:
    Brisbane (nth), Australia
    So, ZCR is a little like AMR / CNR.. a useless novelty that never really took off because it skimped in the wrong place?
     
  15. stalin

    stalin (Taking a Break)

    Joined:
    Jun 26, 2001
    Messages:
    4,581
    Location:
    On the move
    To continue the ZCR bashing, i tried it out on a few newish servers, ran IOmeter on them, thought it was rather poor and replaced it with a Dual Channel U320 128MB IBM RAID (sorry, didnt look at the manu.) performance was significantly better, even for a single disk.

    As for RAID type i would run it either RAID 10 or 0+1, the redundancy call is up to you.

    As for spindle speed, if you can stretch for the 15K drives, do it without a question.

    I had 4x10K disks RAID 5 + hotspare and replaced them all with 15K disks (array rebuild to preserve the data) server performance was noticbly more responsive. I have IOmeter results of before and after, if i find em and you want them, i can post em.

    www.google.com

    :)

    j/k
     
  16. Whisper

    Whisper Member

    Joined:
    Jun 27, 2001
    Messages:
    8,297
    Location:
    Sydney
    This is just me, but I would have though that for the Ratio of Reads to Writes, that the best solution would be a RAID-5 system.

    RAID 0+1 or RAID 10 requires so many drives and unless you mirror across 2 seperate storage controllers, you still stuck with having your disk controller being a single point of failure, in which case you're stuffed no matter which version of RAID you go with.

    With RAID5 you can have an online redundant hotswappable drive and deal with replacement of the drive at your lesiure, so to speak, i.e. You don't have to deal with the replacement hardware immediately.

    I want to see Crusher and Chris Nolans posts anyway before I would decide anything.
     
    Last edited: Aug 13, 2003
  17. =BF=

    =BF= Member

    Joined:
    Jun 27, 2001
    Messages:
    1,238
    Location:
    BrisVegas
    Raid 5 requires a resonse from each stripe write request, including the crc, before it unblocks so writing to a raid 5 is at a disadvantanged compared to the other formats.
     
  18. eva2000

    eva2000 DDR1/DDR2/DDR3 Addict

    Joined:
    Jun 27, 2001
    Messages:
    21,959
    Location:
    Brisbane, Australia
    vbulletin has more writing to database than reading.. most forum apps do.. so raid 5 in a theorectical sense isn't the best but it works when you have a fast enough raid 5 array - example again vbulletin.com's own server 3x 36GB SCSI 10k in raid 5
     
  19. Falls

    Falls Member

    Joined:
    Jun 28, 2001
    Messages:
    3,294
    Location:
    Gun Cat @ Bikini Atoll
    I would agree a good quality 128meg battery backed dedicated raid controller card is the way to go IF you can afford it.

    I dont know if you can get them to work in CLONE PC/SERVER.

    Falls
     
  20. Shado

    Shado Member

    Joined:
    Jun 27, 2001
    Messages:
    3,580
    Location:
    Brisbane, QLD, Australia
    If you're looking at 4 drive Raid 5, as you describe, you would have your 3 drives + hotspare, which would give you the same capacity as the same drives in Raid 10, which is faster, and has almost the same redundancy, (you need 2 specific drives to fail or any 3)..

    IMO unless you start looking at >4 drives in an array, there's not much point in Raid 5....
     

Share This Page

Advertisement: