I am trying to test a project that needs compressed storage with use of the ext4 file system since the application I use relies on ext4 features.
Are there any production/stable solutions out there for transparent compression on ext4?
What I have tried:
Ext4 over ZFS volume with compression enabled. This actually had an adverse affect. I tried creating a ZFS volume with lz4 compression enabled and making an ext4 filesystem on /dev/zvol/… but the zfs volume showed double the actual usage and the compression did not seem to have any effect.
# du -hs /mnt/test
**1.1T** /mnt/test
# zfs list
NAME USED AVAIL REFER MOUNTPOINT
pool 15.2T 2.70G 290K /pool
pool/test 15.2T 13.1T **2.14T** -
ZFS Creation Commands
zpool create pool raidz2 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde2 /dev/sdf1 /dev/sdg1 /dev/sdh2 /dev/sdi1
zfs set recordsize=128k pool
zfs create -p -V15100GB pool/test
zfs set compression=lz4 pool/test
mkfs.ext4 -m1 -O 64bit,has_journal,extents,huge_file,flex_bg,uninit_bg,dir_nlink /dev/zvol/pool/test
Fusecompress: Seemed to work but not 100% stable. Looking for alternatives.
LessFS: Is it possible to use Lessfs in conjunction with ext4? I have not yet tried but would be interested in user insight.
One major problem: not true transparency
An issue I saw with fusecompress was quotas. For example, if I enabled compression on the filesystem, I would want my system to benefit from the compression, not necessarily the end user. If I enabled a quota of 1GB for a user, with a compression ratio of 1.5, they would be able to upload 1.5GB of data, rather than 1GB of data and the system benefiting from the compression. This also appeared to show on df -h. Is there a solution to have compression transparent to quotas?
Best Answer
I use ZFS on Linux as a volume manager and a means to provide additional protections and functionality to traditional filesystems. This includes bringing block-level snapshots, replication, deduplication, compression and advanced caching to the XFS or ext4 filesystems.
See: https://pthree.org/2012/12/21/zfs-administration-part-xiv-zvols/ for another explanation.
In my most common use case, I leverage the ZFS zvol feature to create a sparse volume on an existing zpool. That zvol's properties can be set just like a normal ZFS filesystem's. At this juncture, you can set properties like compression type, volume size, caching method, etc.
Creating this zvol presents a block device to Linux that can be formatted with the filesystem of your choice. Use
fdisk
orparted
to create your partition andmkfs
the finished volume.Mount this and you essentially have a filesystem backed by a zvol and with all of its properties.
Here's my workflow...
Create a zpool comprised of four disks:
You'll want the
ashift=12
directive for the type of disks you're using. The zpool name is "vol0" in this case.Set initial zpool settings:
I set
autoexpand=on
at the zpool level in case I ever replace the disks with larger drives or expand the pool in a ZFS mirrors setup. I typically don't use ZFS raidz1/2/3 because of poor performance and the inability to expand the zpool.Set initial zfs filesystem properties:
Please use the
lz4
compression algorithm for new ZFS installations. It's okay to leave it on all the time.Create ZFS zvol:
For ZFS on Linux, it's very important that you use a large block size.
-o volblocksize=128k
is absolutely essential here. The-s
option creates a sparse zvol and doesn't consume pool space until it's needed. You can overcommit here, if you know your data well. In this case, I have about 444GB of usable disk space in the pool, but I'm presenting an 800GB volume to XFS.Partition zvol device:
(should be /dev/zd0 for the first zvol; /dev/zd16, /dev/zd32, etc. for subsequent zvols)
Create and mount the filesystem:
mkfs.xfs or ext4 on the newly created partition, /dev/zd0p1.
Grab the UUID with
blkid
and modify/etc/fstab
.Mount the new filesystem.
Results...
ZFS filesystem listing.
ZFS zpool list.
ZFS zvol properties (take note of
referenced
,compressratio
andvolsize
).