R – How to use Perl’s readdir multiple times efficiently on the same directory

directoryperl

I had a question using Perl's readdir(). I want to gather all the files in a directory that have the same prefix file name I specified. So, for each prefix, I need to use Perl's readdir() to grep all related files.

Suppose the prefix is "abc", there are several files with the names "abc_1", "abc_2", etc.

However, I noticed that if I put opendir, closedir outside of a loop (loop through a list of file name prefixes), I can only grep the very first prefix from the dir — all the following grepping failed. If I chose to call opendir and closedir each time in the loop, it worked fine but I'm afraid it is not efficient at all.

My question is how can I make it more efficient? It is weird that I can't call readdir multiple times in a loop.

Thanks a lot in advance!

-Jin

Best Answer

Directory (and file) handles are iterators. Reading from one consumes data, you need to either store that data or reset the position of the iterator. Closing and reopening is the hard way; use rewinddir instead.

Alternately, use glob to do the reading and filtering in one step.

Related Solutions

Bash – How to get the source directory of a Bash script from within the script itself

#!/usr/bin/env bash

SCRIPT_DIR="$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )"

is a useful one-liner which will give you the full directory name of the script no matter where it is being called from.

It will work as long as the last component of the path used to find the script is not a symlink (directory links are OK). If you also want to resolve any links to the script itself, you need a multi-line solution:

#!/usr/bin/env bash

SOURCE="${BASH_SOURCE[0]}"
while [ -h "$SOURCE" ]; do # resolve $SOURCE until the file is no longer a symlink
  DIR="$( cd -P "$( dirname "$SOURCE" )" >/dev/null 2>&1 && pwd )"
  SOURCE="$(readlink "$SOURCE")"
  [[ $SOURCE != /* ]] && SOURCE="$DIR/$SOURCE" # if $SOURCE was a relative symlink, we need to resolve it relative to the path where the symlink file was located
done
DIR="$( cd -P "$( dirname "$SOURCE" )" >/dev/null 2>&1 && pwd )"

This last one will work with any combination of aliases, source, bash -c, symlinks, etc.

Beware: if you cd to a different directory before running this snippet, the result may be incorrect!

Also, watch out for $CDPATH gotchas, and stderr output side effects if the user has smartly overridden cd to redirect output to stderr instead (including escape sequences, such as when calling update_terminal_cwd >&2 on Mac). Adding >/dev/null 2>&1 at the end of your cd command will take care of both possibilities.

To understand how it works, try running this more verbose form:

#!/usr/bin/env bash

SOURCE="${BASH_SOURCE[0]}"
while [ -h "$SOURCE" ]; do # resolve $SOURCE until the file is no longer a symlink
  TARGET="$(readlink "$SOURCE")"
  if [[ $TARGET == /* ]]; then
    echo "SOURCE '$SOURCE' is an absolute symlink to '$TARGET'"
    SOURCE="$TARGET"
  else
    DIR="$( dirname "$SOURCE" )"
    echo "SOURCE '$SOURCE' is a relative symlink to '$TARGET' (relative to '$DIR')"
    SOURCE="$DIR/$TARGET" # if $SOURCE was a relative symlink, we need to resolve it relative to the path where the symlink file was located
  fi
done
echo "SOURCE is '$SOURCE'"
RDIR="$( dirname "$SOURCE" )"
DIR="$( cd -P "$( dirname "$SOURCE" )" >/dev/null 2>&1 && pwd )"
if [ "$DIR" != "$RDIR" ]; then
  echo "DIR '$RDIR' resolves to '$DIR'"
fi
echo "DIR is '$DIR'"

And it will print something like:

SOURCE './scriptdir.sh' is a relative symlink to 'sym2/scriptdir.sh' (relative to '.')
SOURCE is './sym2/scriptdir.sh'
DIR './sym2' resolves to '/home/ubuntu/dotfiles/fo fo/real/real1/real2'
DIR is '/home/ubuntu/dotfiles/fo fo/real/real1/real2'

Git – How to add a blank directory to a Git repository

Another way to make a directory stay (almost) empty (in the repository) is to create a .gitignore file inside that directory that contains these four lines:

# Ignore everything in this directory
*
# Except this file
!.gitignore

Then you don't have to get the order right the way that you have to do in m104's solution.

This also gives the benefit that files in that directory won't show up as "untracked" when you do a git status.

Making @GreenAsJade's comment persistent:

I think it's worth noting that this solution does precisely what the question asked for, but is not perhaps what many people looking at this question will have been looking for. This solution guarantees that the directory remains empty. It says "I truly never want files checked in here". As opposed to "I don't have any files to check in here, yet, but I need the directory here, files may be coming later".

Best Answer

Related Solutions

Bash – How to get the source directory of a Bash script from within the script itself

Git – How to add a blank directory to a Git repository

Related Topic