Hadoop – Create directory in hadoop filesystem

To check if a directory exists in a shell script, you can use the following:

if [ -d "$DIRECTORY" ]; then
  # Control will enter here if $DIRECTORY exists.
fi

Or to check if a directory doesn't exist:

if [ ! -d "$DIRECTORY" ]; then
  # Control will enter here if $DIRECTORY doesn't exist.
fi

However, as Jon Ericson points out, subsequent commands may not work as intended if you do not take into account that a symbolic link to a directory will also pass this check. E.g. running this:

ln -s "$ACTUAL_DIR" "$SYMLINK"
if [ -d "$SYMLINK" ]; then 
  rmdir "$SYMLINK" 
fi

Will produce the error message:

rmdir: failed to remove `symlink': Not a directory

So symbolic links may have to be treated differently, if subsequent commands expect directories:

if [ -d "$LINK_OR_DIR" ]; then 
  if [ -L "$LINK_OR_DIR" ]; then
    # It is a symlink!
    # Symbolic link specific commands go here.
    rm "$LINK_OR_DIR"
  else
    # It's a directory!
    # Directory command goes here.
    rmdir "$LINK_OR_DIR"
  fi
fi

Take particular note of the double-quotes used to wrap the variables. The reason for this is explained by 8jean in another answer.

If the variables contain spaces or other unusual characters it will probably cause the script to fail.

Try mkdir -p:

mkdir -p foo

Note that this will also create any intermediate directories that don't exist; for instance,

mkdir -p foo/bar/baz

will create directories foo, foo/bar, and foo/bar/baz if they don't exist.

Some implementation like GNU mkdir include mkdir --parents as a more readable alias, but this is not specified in POSIX/Single Unix Specification and not available on many common platforms like macOS, various BSDs, and various commercial Unixes, so it should be avoided.

If you want an error when parent directories don't exist, and want to create the directory if it doesn't exist, then you can test for the existence of the directory first:

[ -d foo ] || mkdir foo