[Solved] Issues with Storage System

Discussion in 'Storage & Backup' started by damox, Apr 8, 2013.

  1. damox

    damox Member

    Joined:
    Aug 22, 2011
    Messages:
    304
    The System:

    MOBO: Supermicro X9SCM-F
    CPU: Xeon E3-1230
    RAM: 16GB ECC
    HBA: 1x IBM ServeRAID M1015 (Cross flashed IT Firmware)
    EXP: 1x Chenbro CK23601
    CASE: Norco RPC-4224
    HDD: 24 WD Greens (WD20EARX + TLER: 300)
    PSU: HX850 (Gold efficiency, Single +12v rail)
    OS: Solaris 11.1
    Primary Use: Mass/Bulk Storage (Backups, Media, etc.)

    The Hardware Config:

    HBA --> EXP --> RPC-4224 Backplanes --> HDDs.

    The Drive Config (Desired):

    4x 6Disk raidz2 (Separate pools)

    The Problem:

    Having just one ZFS pool attached to the system works fine i.e. 6 drives attached. They spin up, scrub, etc work for days/weeks on end with no issues.

    Adding a second pool i.e. 12 drives attached. pools/drives will work fine as long as they are not both used at the same time. Using BOTH pools at the same time (perhaps transfer data between the two) and drives will start dropping. Usually the same ones, sometimes seemingly random.

    Adding a third pool is out of the question at this stage.

    Having just a single pool of 6 drives, and a second pool of a single drive - I can transfer back and forwards between them no problem. (Seems to be an issue with spinning up so many drives at the same time)

    HBA can get ridiculously hot. Stock/Passive cooling, have a screamer aimed at it though.

    Looking for opinions/ideas on what to do.

    ** I'm not going to pretend I'm on expert on this - even if you think there is something glaring obvious, post it; I'd prefer to hear it than have the idea disappear.
     
    Last edited: Apr 9, 2013
  2. fad

    fad Member

    Joined:
    Jun 26, 2001
    Messages:
    2,443
    Location:
    City, Canberra, Australia

    Have a look at your PSU. Mine was doing the same thing. Turns out I missed plugging in a few of the backplane power leads. In your case it might be the rails can not take the load when the drives are active.
     
  3. OP
    OP
    damox

    damox Member

    Joined:
    Aug 22, 2011
    Messages:
    304
    Power supply is a HX850, which afaik is a single +12v rail.

    There are 12 molex plugs that go to the backplanes. 1 set is for redundancy.

    I currently only have 1 set of 6 plugged in.

    Would you recommend plugging in both sets of 6 (all 12)? Even from the same PSU?
     
  4. terrastrife

    terrastrife Member

    Joined:
    Jun 2, 2006
    Messages:
    18,766
    Location:
    ADL/SA The Monopoly State
    Corsair use really thin gauge wiring on their psus, you might have to run cables to all the backplane power. if you can't do that with the noroc back;lanes, you may have to separate use a new modular cable for each backplane.

    i used to use a single cable from my psu for 3 backplanes/15 drives but over time the psu is getting old and now i need 2.
     
  5. OP
    OP
    damox

    damox Member

    Joined:
    Aug 22, 2011
    Messages:
    304
    I'm currently only running one(!?) modular cable - that connects to a Norco 1-7 molex splitter.

    Sounds like I've made an electrical turd.

    I have a 1200W silverstone to test with - Tomorrow I'll give that a go and try spreading the connections more evenly.
     
  6. terrastrife

    terrastrife Member

    Joined:
    Jun 2, 2006
    Messages:
    18,766
    Location:
    ADL/SA The Monopoly State
    I would run a single modular cable for 2 molex? or just use all your modular molex cables to spread teh load as much as you can

    when i was modding sata cables i found the most i could pull off a 16awg cable was about 10 drives safely.
     
  7. Onthax

    Onthax Member

    Joined:
    Nov 5, 2003
    Messages:
    462
    Not sure if it's the issue here, but WD Greens arent supported in RAID configuration. I've had about 10 die in my ZFS RAIDZ configuration. I've since switched to WD Reds
     
    Last edited: Apr 9, 2013
  8. OP
    OP
    damox

    damox Member

    Joined:
    Aug 22, 2011
    Messages:
    304
    Did this today. One cable to power 2 backplanes.

    System is currently happy to scrub all 4 pools (24 drives) in parallel. :D

    So it really does seem like I was not getting enough power (to the end of a cable?) when too many drives were spun up and being used.

    In hindsight I feel a bit silly :tongue:, but I am extremely grateful for those who had input!

    Marking thread as [Solved] - thanks guys!
     

Share This Page

Advertisement: