Bash – How to escape an arbitrary string for use as a command line argument in Bash

bashperl

I have a list of strings and I want to pass those strings as arguments in a single Bash command line call. For simple alphanumeric strings it suffices to just pass them verbatim:

> script.pl foo bar baz yes no
foo
bar
baz
yes
no

I understand that if an argument contains spaces or backslashes or double-quotes, I need to backslash-escape the double-quotes and backslashes, and then double-quote the argument.

> script.pl foo bar baz "\"yes\"\\\"no\""
foo
bar
baz
"yes"\"no"

But when an argument contains an exclamation mark, this happens:

> script.pl !foo
-bash: !foo: event not found

Double quoting doesn't work:

> script.pl "!foo"
-bash: !foo: event not found

Nor does backslash-escaping (notice how the literal backslash is present in the output):

> script.pl "\!foo"
\!foo

I don't know much about Bash yet but I know that there are other special characters which do similar things. What is the general procedure for safely escaping an arbitrary string for use as a command line argument in Bash? Let's assume the string can be of arbitrary length and contain arbitrary combinations of special characters. I would like an escape() subroutine that I can use as below (Perl example):

$cmd = join " ", map { escape($_); } @args;

Here are some more example strings which should be safely escaped by this function (I know some of these look Windows-like, that's deliberate):

yes
no
Hello, world      [string with a comma and space in it]
C:\Program Files\ [path with backslashes and a space in it]
"                 [i.e. a double-quote]
\                 [backslash]
\\                [two backslashes]
\\\               [three backslashes]
\\\\              [four backslashes]
\\\\\             [five backslashes]
"\                [double-quote, backslash]
"\T               [double-quote, backslash, T]
"\\T              [double-quote, backslash, backslash, T]
!1                
!A                
"!\/'"            [double-quote, exclamation, backslash, forward slash, apostrophe, double quote]
"Jeff's!"         [double-quote, J, e, f, f, apostrophe, s, exclamation, double quote]
$PATH             
%PATH%            
&                 
<>|&^             
*@$$A$@#?-_

EDIT:

Would this do the trick? Escape every unusual character with a backslash, and omit single or double quotes. (Example is in Perl but any language can do this)

sub escape {
    $_[0] =~ s/([^a-zA-Z0-9_])/\\$1/g;
    return $_[0];
}

Best Answer

If you want to securely quote anything for Bash, you can use its built-in printf %q formatting:

cat strings.txt:

yes
no
Hello, world
C:\Program Files\
"
\
\\
\\\
\\\\
\\\\\
"\
"\T
"\\T
!1
!A
"!\/'"
"Jeff's!"
$PATH
%PATH%
&
<>|&^
*@$$A$@#?-_

cat quote.sh:

#!/bin/bash
while IFS= read -r string
do
    printf '%q\n' "$string"
done < strings.txt

./quote.sh:

yes
no
Hello\,\ world
C:\\Program\ Files\\
\"
\\
\\\\
\\\\\\
\\\\\\\\
\\\\\\\\\\
\"\\
\"\\T
\"\\\\T
\!1
\!A
\"\!\\/\'\"
\"Jeff\'s\!\"
\$PATH
%PATH%
\&
\<\>\|\&\^
\*@\$\$A\$@#\?-_

These strings can be copied verbatim to for example echo to output the original strings in strings.txt.

Related Solutions

Bash – How to check if a directory exists in a Bash shell script

To check if a directory exists in a shell script, you can use the following:

if [ -d "$DIRECTORY" ]; then
  # Control will enter here if $DIRECTORY exists.
fi

Or to check if a directory doesn't exist:

if [ ! -d "$DIRECTORY" ]; then
  # Control will enter here if $DIRECTORY doesn't exist.
fi

However, as Jon Ericson points out, subsequent commands may not work as intended if you do not take into account that a symbolic link to a directory will also pass this check. E.g. running this:

ln -s "$ACTUAL_DIR" "$SYMLINK"
if [ -d "$SYMLINK" ]; then 
  rmdir "$SYMLINK" 
fi

Will produce the error message:

rmdir: failed to remove `symlink': Not a directory

So symbolic links may have to be treated differently, if subsequent commands expect directories:

if [ -d "$LINK_OR_DIR" ]; then 
  if [ -L "$LINK_OR_DIR" ]; then
    # It is a symlink!
    # Symbolic link specific commands go here.
    rm "$LINK_OR_DIR"
  else
    # It's a directory!
    # Directory command goes here.
    rmdir "$LINK_OR_DIR"
  fi
fi

Take particular note of the double-quotes used to wrap the variables. The reason for this is explained by 8jean in another answer.

If the variables contain spaces or other unusual characters it will probably cause the script to fail.

Bash – How to get the source directory of a Bash script from within the script itself

#!/usr/bin/env bash

SCRIPT_DIR="$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )"

is a useful one-liner which will give you the full directory name of the script no matter where it is being called from.

It will work as long as the last component of the path used to find the script is not a symlink (directory links are OK). If you also want to resolve any links to the script itself, you need a multi-line solution:

#!/usr/bin/env bash

SOURCE="${BASH_SOURCE[0]}"
while [ -h "$SOURCE" ]; do # resolve $SOURCE until the file is no longer a symlink
  DIR="$( cd -P "$( dirname "$SOURCE" )" >/dev/null 2>&1 && pwd )"
  SOURCE="$(readlink "$SOURCE")"
  [[ $SOURCE != /* ]] && SOURCE="$DIR/$SOURCE" # if $SOURCE was a relative symlink, we need to resolve it relative to the path where the symlink file was located
done
DIR="$( cd -P "$( dirname "$SOURCE" )" >/dev/null 2>&1 && pwd )"

This last one will work with any combination of aliases, source, bash -c, symlinks, etc.

Beware: if you cd to a different directory before running this snippet, the result may be incorrect!

Also, watch out for $CDPATH gotchas, and stderr output side effects if the user has smartly overridden cd to redirect output to stderr instead (including escape sequences, such as when calling update_terminal_cwd >&2 on Mac). Adding >/dev/null 2>&1 at the end of your cd command will take care of both possibilities.

To understand how it works, try running this more verbose form:

#!/usr/bin/env bash

SOURCE="${BASH_SOURCE[0]}"
while [ -h "$SOURCE" ]; do # resolve $SOURCE until the file is no longer a symlink
  TARGET="$(readlink "$SOURCE")"
  if [[ $TARGET == /* ]]; then
    echo "SOURCE '$SOURCE' is an absolute symlink to '$TARGET'"
    SOURCE="$TARGET"
  else
    DIR="$( dirname "$SOURCE" )"
    echo "SOURCE '$SOURCE' is a relative symlink to '$TARGET' (relative to '$DIR')"
    SOURCE="$DIR/$TARGET" # if $SOURCE was a relative symlink, we need to resolve it relative to the path where the symlink file was located
  fi
done
echo "SOURCE is '$SOURCE'"
RDIR="$( dirname "$SOURCE" )"
DIR="$( cd -P "$( dirname "$SOURCE" )" >/dev/null 2>&1 && pwd )"
if [ "$DIR" != "$RDIR" ]; then
  echo "DIR '$RDIR' resolves to '$DIR'"
fi
echo "DIR is '$DIR'"

And it will print something like:

SOURCE './scriptdir.sh' is a relative symlink to 'sym2/scriptdir.sh' (relative to '.')
SOURCE is './sym2/scriptdir.sh'
DIR './sym2' resolves to '/home/ubuntu/dotfiles/fo fo/real/real1/real2'
DIR is '/home/ubuntu/dotfiles/fo fo/real/real1/real2'

Best Answer

Related Solutions

Bash – How to check if a directory exists in a Bash shell script

Bash – How to get the source directory of a Bash script from within the script itself

Related Topic