If you haven’t read Cindy Swearingen’s latest blog post yet, it is time for you to know that Solaris 11.3 beta is shipped with a zpool version 37 which brings lz4 compression to ZFS.
My last post is about generating report files in a html format and store them on a apache webserver.
Today I was wondering how much disk space the reporting will take and looked at the dataset.
# zfs get compression,compressratio,recordsize,referenced,used lofs/AP6A103/cve NAME PROPERTY VALUE SOURCE lofs/AP6A103/cve compression on inherited from lofs lofs/AP6A103/cve compressratio 2.45x - lofs/AP6A103/cve recordsize 128K default lofs/AP6A103/cve referenced 163M - lofs/AP6A103/cve used 163M - # mv 2015 /var/tmp/ # ptime mv /var/tmp/2015 . real 1.897081940 user 0.068310370 sys 1.828314030
Since this is not a Solaris 11.2 installation I wanted to change compression to lz4 and see what difference this would make. So I moved the data to a different zfs dataset, set the compression value to lz4 and moved the data back again. Just as a note, I set the compression value for the dataset of the whole zpool in this case. Could have also done it only for the child dataset.
# mv 2015 /var/tmp/ # zfs set compression=lz4 lofs # ptime mv /var/tmp/2015 . real 2.094843840 user 0.072113780 sys 2.022243500
The data is moved in only a few more milliseconds so let’s see if lz4 is worth using it.
# zfs get compression,compressratio,recordsize,referenced,used lofs/AP6A103/cve NAME PROPERTY VALUE SOURCE lofs/AP6A103/cve compression lz4 inherited from lofs lofs/AP6A103/cve compressratio 16.26x - lofs/AP6A103/cve recordsize 128K default lofs/AP6A103/cve referenced 16.9M - lofs/AP6A103/cve used 16.9M -
As the above output shows the compressratio is awesome. It might be a wink of an eye slower on compressible data like this but that for general purpose this environments that are not 100% I/O critical this shouldn’t matter.
This use case here is dealing with compressible data only. But on a companies storage you happen to not always being able to separate compressible and incompressible data. Unlike other algorithms if data is recognized as incompressible lz4 won’t try hard to compress it anyway and instead move on with the next data. lz4 has more to offer than just this but this feature by itself qualifies it as the new default value of compression for every zpool (besides the rpool, yet).