Linux – Filesystem large number of files in a single directory

ext3linux

OK, not so large but I need to use something where around 60,000 files with average size of 30kb are stored in a single directory (this is a requirement so can't simply break into sub-directories with smaller number of files).

The files will be accessed randomly, but once created there will be no writes to the same filesystem. I'm currently using Ext3 but finding it very slow. Any suggestions?

Best Answer

You should consider XFS. It supports a very large number of files both at the filesystem and at the directory level, and the performance remains relatively consistent even with a large number of entries due to the B+ tree data structures.

There's a page on their wiki to a large number of papers and publications that detail the design. I recommend you give it a try and benchmark it against your current solution.