Linux – Storing files in a directory… are there limits

directoryfilesystemslimitationslinux

I am using CentOS 5 with Plesk 9 (64-bit), I am running a site where users will be uploading pictures. With a 64 bit os, are there any limits to how many files I can store? All I care about is performance, and serving up the files. I'd prefer not to have 4 directories deep of scattered files. However, I am hoping, that at some point I could have 200-300 thousand images.

Best Answer

If you are using ext3, I found this quote (warning: spanish speaking site)

"There is a limit of 32k (32768) subdirectories in a single directory, a limitation likely of only academic interest, as many people don't even have that many files (though huge mail servers may need to keep that in mind). The ext2 inode specification allows for over 100 trillion files to reside in a single directory"

Further reading showed that ext3 doesn't have a 32K limitation, which can be empirically proven with

a=0; i=1; while [ $a == 0 ]; do touch $i; a=$?; let i++; done

but it does have a 32K folder limit for folder, which can be tested with

a=0; i=1; while [ $a == 0 ]; do mkdir $i; a=$?; let i++; done

This (unfounded) claim says that

ReiserFS has no trouble at all with hundreds of thousands of files in a single directory. flabdablet - February 1, 2007

This question from sister site stackoverflow.com could help too.

In general:

  • There is a limit to the amount of directories,
  • You should keep your files/directories under 32K, but can go a lot further,
  • The file system you are using does matter.
Related Topic