ZFS - upgraded all disks but vdev didnt grow

As the thread title says - i upgraded all my disks in my vdev. From 10x 1.8TB (real 2TB disks) to 10x 2TB (3TB disks but only showing as 2TB due to controller issues).

Thing is - ZFS sees these disks as 200GB larger than my old ones (via the sizes listed in the format command). However - my vdev has not grown and the size is still the same as before. I *should* have at least another 1.5TB (8 disks x 200GB + 2x parity disks). Ive ordered a new HBA so the drives get recognized at 3TB - but now i'm worried that once they become 3TB - ZFS wont acknowledge it.

I'm running solaris express 11

 

How'd you do that, exactly? How are you determining the size?

 

I replaced 10x 2TB disks with 10x 3TB disks. When i type "format" the drives now show as "2TB". Instead of "1.8TB".

When i type zpool iostat -v monkey, the free space is the same as always (800GB).

If there are other commands i should use i'm all for it

 

http://opensolaris.org/jive/thread.jspa?threadID=117805

Check out 'autoexpand'.

 

Try exporting your pool and then importing it. The import should detect the new size of your drives and allocate the free space accordingly.

... and as always, make sure you have good backups in place before playing around with arrays

 

Did you scrub after each disk replacement including the final disk replacement which will then detect the new pool size?

 

Scrubbing right now, so will see what happens when its done . 6 Hours to go.

I didn't scrub after each one because it takes 20 hours to replace each disk, then to scrub would take even more (for 10 disks). I scrubbed half way and there were no errors so i figured what i was doing was all good . So i hope a scrub at the end is enough.

Ahh this might be it. I'll give it a try once my scrub is done.

 

It's my understanding that without autoexpand on, the pool size will stay the same. I'm guessing in case you replace a drive with a larger one, only to return back to a smaller one (like I had to waiting for a warranty drive). If autoexpand been on, I wouldn't have been able to get back to my smaller drives.

 

I believe it goes

Offline a drive
Replace drive

all the way thru to the last drive

Then do a scrub and the pool automagically grows.

 

run a

Code:

zpool get all poolname 

and if auto expand is not set to on then it wont grow by itself.

if its off enable it with a

Code:

zpool set autoexpand=on poolname

Paul

 

Thanks guys, it was the auto expand value that did it. All good now

now i just need someone with as much storage space as me so i can redistibute my data better

(by copying everything off, then back on)

Code:

flain@monkey:~# zpool iostat -v monkey
                capacity     operations    bandwidth
pool         alloc   free   read  write   read  write
-----------  -----  -----  -----  -----  -----  -----
monkey       27.1T  10.9T  3.04K     11   378M   371K
  raidz2     17.5T  2.36T  1.97K      9   244M   365K
    c11t0d0      -      -    822      6  31.0M  71.4K
    c11t1d0      -      -    856      6  31.0M  71.4K
    c11t2d0      -      -    854      6  31.0M  71.4K
    c11t3d0      -      -    855      6  31.0M  71.4K
    c12t4d0      -      -    853      6  31.0M  71.4K
    c12t5d0      -      -    880      6  31.0M  71.4K
    c12t6d0      -      -    876      6  31.0M  71.4K
    c12t7d0      -      -    888      6  31.0M  71.4K
    c11t7d0      -      -    860      6  31.0M  71.4K
    c11t6d0      -      -    883      6  31.0M  71.4K
  raidz2     9.62T  8.51T  1.07K      1   134M  6.24K
    c11t5d0      -      -    626      1  17.0M  1.82K
    c11t4d0      -      -    616      1  17.1M  1.83K
    c12t0d0      -      -    634      1  16.9M  1.82K
    c12t1d0      -      -    703      1  16.9M  1.83K
    c12t2d0      -      -    490      1  16.9M  1.82K
    c12t3d0      -      -    633      1  16.9M  1.83K
    c1t4d0       -      -    625      1  16.9M  1.82K
    c1t5d0       -      -    631      1  16.9M  1.83K
    c1t6d0       -      -    635      1  16.9M  1.83K
    c1t7d0       -      -    713      1  16.9M  1.83K
-----------  -----  -----  -----  -----  -----  -----

 

I think you could probably just make some new shares, and copy the data from the original shares to the new ones, I assume that would redistribute the files across all the disks properly without you having to completely copy data off your pool somewhere else. It would be nice if scrub had an option to rewrite data evenly across disks.

 

I tried this, but it's very painful to do. I started copying folders with over 4TB worth of data, then deleting the original. Ive kind of made it worse somehow :/

I think the feature we are all waiting for is the infamous BP rewrite which would allow all kinda of cool magic . (vdev resizing, defraging, disk evacuating etc)

 

And if Oracle ever release it I expect them to do so just after they do a solaris source code release, so everyone else has to wait a full cycle before they get their hands on it.

 

suggest you turn autoexpand back off so you can't get caught inadvertantly

 

Or you could just do like me and build a second box which lives (in the server room) at work.

A quick rysnc onto a portable drive every week or so keeps everything nicely up to date.

Hehehe ...