1. OCAU Merchandise is available! Check out our 20th Anniversary Mugs, Classic Logo Shirts and much more! Discussion in this thread.
    Dismiss Notice

Test of Solaris x86 RAIDZ/ZFS performance with 6x500GB drives & cheap SATA cards.

Discussion in 'Storage & Backup' started by trza2k, Apr 19, 2007.

  1. trza2k

    trza2k Member

    Joined:
    Sep 17, 2002
    Messages:
    557
    Location:
    Sydney
    Well i was originally set on purchasing an Areca 1220 for use with raid5 but in the end couldn't justify the price for a home fileserver. I then had at look at using a software raid with Solaris 10 x86 and there built in RAIDZ/ZFS software RAID solution.

    I purchased 3x Skymaster 2port PCI-E card($32 each) to connect with 6x500GB Samsung drives. The Skymaster card uses the si3132 chipset which will work with Solaris 10 with a bit of playing around. I was really surprised at some of the features this RAIDZ/ZFS combo offered with block level checksumming/snapshots/write hole of raid5 removed etc etc and thought it wouldn't really offer much performance with all the overhead, boy was i surprised!!!

    With the OS booted and all disks connected i was able to build the 2.1TB RAIDZ pool in one command and it was ready to be used in under 30seconds :O

    I then used IOZONE to test the performance, using it to perform a 16gig READ and WRITE test i achieved the following performance using a 128k block size.

    WRITE: 230MB/s
    READ: 310MB/sec

    4GIG test yielded:
    WRITE: 262MB/s
    READ: 333MB/s

    The system had 1gig of ram so i was very surprised by the results and this will becoming my new fileserver :)
     
  2. oli

    oli Member

    Joined:
    Jun 29, 2002
    Messages:
    7,263
    Location:
    The Internet
    Interesting information. Thanks for posting it.

    What motherboard are you using, out of interest?
     
  3. OP
    OP
    trza2k

    trza2k Member

    Joined:
    Sep 17, 2002
    Messages:
    557
    Location:
    Sydney
    Asus a8n SLI premium with Athlon 64 3200+.

    Solaris runs in 64bit mode and supports the Nvidia onboard SATA ports but in emulated PATA mode, which mind you is still very fast from some quick tests. Also Nvidia Gigabit net works without a hitch.
     
  4. oli

    oli Member

    Joined:
    Jun 29, 2002
    Messages:
    7,263
    Location:
    The Internet
    Well obviously it's fast enough to fill your (presumably) gigE network, since you said it's for a fileserver. Wonder if performance is similar with Linux and software RAID5 on ext3... :confused:
     
  5. OP
    OP
    trza2k

    trza2k Member

    Joined:
    Sep 17, 2002
    Messages:
    557
    Location:
    Sydney
    I was able to get around 70MB/s read/write over the network(yes gige), though my 15k scsi drive on my desktop seemed to be the bottleneck.

    Not sure how linux would go with the PCI-E cards but RAIDZ is supposed to be faster then RAID5, without testing i would be speculating though. I may be able to do a quick install and test as i was going to rebuild the box on the weekend anyway.
     
  6. oh_noes

    oh_noes Member

    Joined:
    Oct 29, 2006
    Messages:
    991
    Location:
    Australia
    So your running a PCI Video card, and two PCI-E SATA Cards in both the PCI-E's?

    Can you post some more details on getting the card to work with Solaris 10? I'm thinking of doing something similar, but with the 3124 version of the Skymaster PCI SATA (4 port).
     
    Last edited: Apr 19, 2007
  7. OP
    OP
    trza2k

    trza2k Member

    Joined:
    Sep 17, 2002
    Messages:
    557
    Location:
    Sydney
    I have 4 PCI-E slots, though i was considering a PCI video card so i could have an additional 3132 card.

    The 3124 cards work fine with the default install(no hacking required), i have 2 skymaster 4port 3124 cards. I was able to get 120MB/s read and 60MB/s write with the PCI cards, PCI bus obviously being the limit there.
     
  8. fad

    fad Member

    Joined:
    Jun 26, 2001
    Messages:
    2,752
    Location:
    City, Canberra, Australia
    Did you use those dvds Sun was sending out?

    I downloaded a few iso's and the software was really buggy.
     
  9. OP
    OP
    trza2k

    trza2k Member

    Joined:
    Sep 17, 2002
    Messages:
    557
    Location:
    Sydney
    I've had no issues, i am using the Solaris 10 x86 11/06 build.

    There are other versions such as Open Solaris and what not which are community driven and may be buggy.

    What exactly was buggy though?
     
  10. OP
    OP
    trza2k

    trza2k Member

    Joined:
    Sep 17, 2002
    Messages:
    557
    Location:
    Sydney
    Well i managed to install linux today and give it a quick test, already i'm not liking the performance at all!! I'm currently have to init the raid5 mdadm array in linux and its going to take about 1.5hours. The init process is almost instant in solaris. Also the init speed is currently sitting at speed=70932K/sec.

    Anyway i'll give it a chance and benchmark once init'd.
     
  11. oli

    oli Member

    Joined:
    Jun 29, 2002
    Messages:
    7,263
    Location:
    The Internet
    Weren't your drives all brand new when you did it in Solaris?

    Now the second time it actually has to reformat the drives because they've been formatted differently before?

    Anyway 70932K(B I am guessing) a second is pretty fast. :)
     
  12. OP
    OP
    trza2k

    trza2k Member

    Joined:
    Sep 17, 2002
    Messages:
    557
    Location:
    Sydney
    I'm pretty sure an init doesn't need to occur because of the way ZFS/RAIDZ works and integrates with each other.

    Anyway i ran the same test on the linux box as i did on solaris with iozone:

    iozone -f /silvia/testfile -r 128k -s 4g -i0 -i1

    write rewrite read reread in Kb/sec.
    121014 112033 261309 262729

    Seems the writes are quite a bit slower and reads are a tad slower then the ZFS/RAIDZ combo.
     
  13. oli

    oli Member

    Joined:
    Jun 29, 2002
    Messages:
    7,263
    Location:
    The Internet
    Interesting.. Thanks for that. I will keep it in mind down the track!
     
  14. tephra

    tephra Member

    Joined:
    Mar 15, 2005
    Messages:
    519
    Location:
    Melbourne, Australia
    can you do a zpool status for us?
     
  15. OP
    OP
    trza2k

    trza2k Member

    Joined:
    Sep 17, 2002
    Messages:
    557
    Location:
    Sydney
    sure.

    -bash-3.00# zpool status
    pool: silvia
    state: ONLINE
    scrub: none requested
    config:

    NAME STATE READ WRITE CKSUM
    silvia ONLINE 0 0 0
    raidz1 ONLINE 0 0 0
    c2t0d0 ONLINE 0 0 0
    c3t0d0 ONLINE 0 0 0
    c4t0d0 ONLINE 0 0 0
    c2t1d0 ONLINE 0 0 0
    c3t1d0 ONLINE 0 0 0
    c4t1d0 ONLINE 0 0 0

    errors: No known data errors
     
  16. stmok

    stmok Member

    Joined:
    Jul 24, 2001
    Messages:
    8,882
    Location:
    Sydney
    So...It kicks ass? ;)
     
  17. OP
    OP
    trza2k

    trza2k Member

    Joined:
    Sep 17, 2002
    Messages:
    557
    Location:
    Sydney
    Well considering my $90 worth of PCI-E cards is beating my buddies $950 areca 1220 i'm pretty damn happy! Not to mention i now have checksumming and snapshots to give that extra level of protection.

    If you looking for a cheap nas/fileserver and you have the compatible hardware i would certainly at least give Solaris a try! (version 10 11/06+)
     
  18. stmok

    stmok Member

    Joined:
    Jul 24, 2001
    Messages:
    8,882
    Location:
    Sydney
    Definitely valued for money. :D

    Yeah, I've recently got a free copy of ver 10 11/06 through their mail initiative.

    I've tried it, but its a bit slow. Maybe I'm doing something wrong. I'll try again.

    Their Solaris Express seems pretty good. I'm trying it out as a desktop on a spare box.
     
  19. OP
    OP
    trza2k

    trza2k Member

    Joined:
    Sep 17, 2002
    Messages:
    557
    Location:
    Sydney
    When exactly do you mean by slow? Are you using the GUI? machine specs?
     
  20. grs1961

    grs1961 Member

    Joined:
    Jan 21, 2005
    Messages:
    549
    Location:
    Melbourne
    My S10u3 system, ASUS P5P800-MX, 3GHz P4-EMT64, 2Gb, RAIDZ, using an Addonics ADSA4R5 PCI and 4 x Seagate 320Gb SATAII 16Mb cache.

    (Okay, it's not a cheap card, but it's SATA II and PCI)

    $ zpool status
    pool: uu
    state: ONLINE
    scrub: none requested
    config:

    NAME STATE READ WRITE CKSUM
    uu ONLINE 0 0 0
    raidz1 ONLINE 0 0 0
    c3t0d0 ONLINE 0 0 0
    c3t1d0 ONLINE 0 0 0
    c3t2d0 ONLINE 0 0 0
    c3t3d0 ONLINE 0 0 0

    errors: No known data errors

    Build iozone using the default Solaris10 settings, using the Studio 11 compiler:
    $ iozone -f /uu/test/testfile -r 128k -s 4g -i0 -i1
    KB reclen write rewrite read reread
    4194304 128 25485 44854 121272 121686

    Now, lets see if I can get it to build a 64-bit binary...
    64-bit Studio 11 build:
    $ iozone -f /uu/test/testfile -r 128k -s 4g -i0 -i1
    KB reclen write rewrite read reread
    4194304 128 45878 43377 120806 121136

    Hmmm, that's interesting, the write doubled, but all else stayed abiut the same...
     
    Last edited: Apr 24, 2007

Share This Page

Advertisement: