The fastest way to remove thousands of files on a btrfs filesystem

btrfsfilesystemsremove

Currently I have a btrfs mount point with a structure like this:#

/mountpoint/month/day/hourAs24/

and each leaf directory contains between 5,000 to 20,000 small files. There I keep the files for two month. Each day I remove the directory older than 60 days with the command

rm -R /mountpoint/month/day/

This command takes ages to run and the load on the server is extremely high while the command runs.

Would it be better to create btrfs subvolumes under /mountpoint/month/ for every day and then purge the subvolumes in one single command?

Are there any other fast and lightweight solutions to get rid of the files under one btrfs directory in a single command?

Edit:
To clarify the situation. In the structure the folders month, day and hourAs24 are variables that are substituted by the correspondent values of the current date-time.

Edit after solution:
It works smoothly on my test machine. And all of the following works live with the mountpoint mounted! First I create normal directories for each month with

mkdir /mountpoint/month

Then I create btrfs subvolumes for each day of month with

btrfs subvolume create /mountpoint/month/day

then I create the normal directories for each hour of day with

mkdir /mountpoint/month/day/hourAs24

And after 60 days I can easily purge the directory of the day

btrfs subvolume delete /mountpoint/month/day

(Now I have to wait 60 days to see the performance on the production server)

Best Answer

I would go the subvolumes route, myself. You just can't beat it for speed, and if you tilt your head and squint a bit, you can even say that it looks like the "right" way to store the files anyway...

Related Topic