I have an application writing to an ext3 directory which over time has grown to roughly three million files. Needless to say, reading the file listing of this directory is unbearably slow.
I don't blame ext3. The proper solution would have been to let the application code write to sub-directories such as ./a/b/c/abc.ext rather than using only ./abc.ext.
I'm changing to such a sub-directory structure and my question is simply: roughly how many files should I expect to store in one ext3 directory while still getting acceptable performance? What's your experience?
Or in other words; assuming that I need to store three million files in the structure, how many levels deep should the ./a/b/c/abc.ext structure be?
Obviously this is a question that cannot be answered exactly, but I'm looking for a ball park estimate.
Best Answer
Provided you have a distro that supports the
dir_indexcapability then you can easily have 200,000 files in a single directory. I'd keep it at about 25,000 though, just to be safe. Withoutdir_index, try to keep it at 5,000.