Iterate through folders, then subfolders and print filenames with path to text file

directorypython-2.6subdirectory

I am trying to use python to create the files needed to run some other software in batch.
For part of this I need to produce a text file that loads the needed data files into the software.
My problem is that the files I need to enter into this text file are stored in a set of structured folders.

I need to loop over a set of folders (up to 20), which each could contain up to 3 more folders which contain the files I need. The bottom level of the folders contain a set of files needed for each run of the software. The text file should have the path+name of these files printed line by line, add an instruction line and then move to the next set of files from a folder and so on until all of sub level folders have been checked.

Best Answer

Charles' answer is good, but can be improved upon to increase speed and efficiency. Each item produced by os.walk() (See docs) is a tuple of three items. Those items are:

The working directory
A list of strings naming any sub-directories present in the working directory
A list of files present in the working directory

Knowing this, much of Charles' code can be condensed with the modification of a forloop:

import os

def list_files(dir):
    r = []
    for root, dirs, files in os.walk(dir):
        for name in files:
            r.append(os.path.join(root, name))
    return r

Related Solutions

C# – How to remove illegal characters from path and filenames

Try something like this instead;

string illegal = "\"M\"\\a/ry/ h**ad:>> a\\/:*?\"| li*tt|le|| la\"mb.?";
string invalid = new string(Path.GetInvalidFileNameChars()) + new string(Path.GetInvalidPathChars());

foreach (char c in invalid)
{
    illegal = illegal.Replace(c.ToString(), ""); 
}

But I have to agree with the comments, I'd probably try to deal with the source of the illegal paths, rather than try to mangle an illegal path into a legitimate but probably unintended one.

Edit: Or a potentially 'better' solution, using Regex's.

string illegal = "\"M\"\\a/ry/ h**ad:>> a\\/:*?\"| li*tt|le|| la\"mb.?";
string regexSearch = new string(Path.GetInvalidFileNameChars()) + new string(Path.GetInvalidPathChars());
Regex r = new Regex(string.Format("[{0}]", Regex.Escape(regexSearch)));
illegal = r.Replace(illegal, "");

Still, the question begs to be asked, why you're doing this in the first place.

Ruby – Iterate through every file in one directory

As others have said, Dir::foreach is a good option here. However, note that Dir::foreach and Dir::entries will always include . and .. (the current and parent directories). You will generally not want to work on them, so you can use Dir::each_child or Dir::children (as suggested by ma11hew28) or do something like this:

Dir.foreach('/path/to/dir') do |filename|
  next if filename == '.' or filename == '..'
  # Do work on the remaining files & directories
end

Dir::foreach and Dir::entries (as well as Dir::each_child and Dir::children) also include hidden files & directories. Often this is what you want, but if it isn't, you need to do something to skip over them.

Alternatively, you might want to look into Dir::glob which provides simple wildcard matching:

Dir.glob('/path/to/dir/*.rb') do |rb_filename|
  # Do work on files & directories ending in .rb
end

Best Answer

Related Solutions

C# – How to remove illegal characters from path and filenames

Ruby – Iterate through every file in one directory

Related Topic