The earlier mentioned Solaris 11.4 Beta build refresh is here.
Which means: ZFS DEVICE REMOVAL is available to all of you. :-)
Since I am short of time right now here is the very quick and trivial proof:
root@wacken:~# zpool list NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT rpool 99.5G 55.5G 44.0G 55% 1.00x ONLINE - root@wacken:~# for i in 1 2 3;do mkfile 1g diskfile$i;done root@wacken:~# zpool create rempool /root/diskfile1 /root/diskfile2 /root/diskfile3 root@wacken:~# zpool list NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT rempool 2.98G 152K 2.98G 0% 1.00x ONLINE - rpool 99.5G 55.5G 44.0G 55% 1.00x ONLINE - root@wacken:~# zpool status rempool pool: rempool state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM rempool ONLINE 0 0 0 /root/diskfile1 ONLINE 0 0 0 /root/diskfile2 ONLINE 0 0 0 /root/diskfile3 ONLINE 0 0 0 errors: No known data errors root@wacken:~# zpool remove rempool /root/diskfile2 root@wacken:~# zpool status rempool pool: rempool state: ONLINE scan: resilvered 1K in 1s with 0 errors on Fri Mar 9 13:08:34 2018 config: NAME STATE READ WRITE CKSUM rempool ONLINE 0 0 0 /root/diskfile1 ONLINE 0 0 0 /root/diskfile3 ONLINE 0 0 0 errors: No known data errors
Works!
This is what it looks like when there is not enough space left for removing a device from a pool.
root@wacken:~# zpool status rempool pool: rempool state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM rempool ONLINE 0 0 0 /root/diskfile1 ONLINE 0 0 0 /root/diskfile2 ONLINE 0 0 0 /root/diskfile3 ONLINE 0 0 0 errors: No known data errors root@wacken:~# zpool list rempool NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT rempool 2.98G 2.54G 443M 85% 1.00x ONLINE - root@wacken:~# zpool remove rempool /root/diskfile2 cannot remove device(s): not enough space to migrate data
It actually doesn’t matter what your top-level vdev is. Her is an example of a multiple mirrored vdevs:
root@wacken:~# zpool destroy rempool root@wacken:~# zpool create rempool2 mirror /root/diskfile1 /root/diskfile2 mirror /root/diskfile3 /root/diskfile4 mirror /root/diskfile5 /root/diskfile6 root@wacken:~# zpool status pool: rempool2 state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM rempool2 ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 /root/diskfile1 ONLINE 0 0 0 /root/diskfile2 ONLINE 0 0 0 mirror-1 ONLINE 0 0 0 /root/diskfile3 ONLINE 0 0 0 /root/diskfile4 ONLINE 0 0 0 mirror-2 ONLINE 0 0 0 /root/diskfile5 ONLINE 0 0 0 /root/diskfile6 ONLINE 0 0 0 errors: No known data errors pool: rpool state: ONLINE scan: resilvered 0 in 5s with 0 errors on Thu Mar 8 23:27:12 2018 config: NAME STATE READ WRITE CKSUM rpool ONLINE 0 0 0 c1d0 ONLINE 0 0 0 errors: No known data errors root@wacken:~# zpool remove rempool2 mirror-1 root@wacken:~# zpool status pool: rempool2 state: ONLINE scan: resilvered 1.50K in 1s with 0 errors on Tue Apr 10 05:25:51 2018 config: NAME STATE READ WRITE CKSUM rempool2 ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 /root/diskfile1 ONLINE 0 0 0 /root/diskfile2 ONLINE 0 0 0 mirror-2 ONLINE 0 0 0 /root/diskfile5 ONLINE 0 0 0 /root/diskfile6 ONLINE 0 0 0 errors: No known data errors pool: rpool state: ONLINE scan: resilvered 0 in 5s with 0 errors on Thu Mar 8 23:27:12 2018 config: NAME STATE READ WRITE CKSUM rpool ONLINE 0 0 0 c1d0 ONLINE 0 0 0 errors: No known data errors
Very nice additional improvement to an already feature-rich and fantastic release!