Bash – Escaping special characters while sourcing file in bash

bashshell-scripting

I've written a script that sources a bash shell fragment. The shell fragment is supposed to be a kind of configuration file, basically it's a bunch of bash variable.

The issue here is when some of those string variable contain characters that should be escaped. As they aren't escape, the script might faill later on or have undesired behavior.

At the moment, my script is validating the shell fragment (the configuration file) bash syntax (bash -n) but if any charachter is to be escaped, that won't be detected.

So, I would like either to auto-escape sensitive character or, at least, detect their are some so I can display an error and exit.

Any idea how could I achieve that?

Best Answer

The $? variable is your friend. It contains the return status of the most recently executed command, including bash built-in functions.

$ /bin/false
$ echo $?
1
$ /bin/true
$ echo $?
0
$ echo 'FOO="bar' > atest
$ . atest
-bash: atest: line 1: unexpected EOF while looking for matching `"'
-bash: atest: line 2: syntax error: unexpected end of file
$ echo $?
1

Keep in mind that it only returns the status of the most recent command. If you need to run an additional command before performing your error test, you need to immediately assign it to a variable like so:

$ /bin/false
$ MYRETURN=$?
$ /bin/true
$ echo $?
0
$ echo $MYRETURN
1

Related Solutions

Bash – Clean way to write complex multi-line string to a variable

This will put your text into your variable without needing to escape the quotes. It will also handle unbalanced quotes (apostrophes, i.e. '). Putting quotes around the sentinel (EOF) prevents the text from undergoing parameter expansion. The -d'' causes it to read multiple lines (ignore newlines). read is a Bash built-in so it doesn't require calling an external command such as cat.

IFS='' read -r -d '' String <<"EOF"
<?xml version="1.0" encoding='UTF-8'?>
 <painting>
   <img src="madonna.jpg" alt='Foligno Madonna, by Raphael'/>
   <caption>This is Raphael's "Foligno" Madonna, painted in
   <date>1511</date>-<date>1512</date>.</caption>
 </painting>
EOF

Bash avoide intrpretate special characters

I don't understand what particular problem you are stuck on. Your description is really unclear. So I'll give some general advice on simplifying the script; if it's not enough to solve your problem, try to come up with a clearer explanation.

I'm pretty sure this script is much more complicated than it needs to be. Spending a few minutes browsing the documentation of a command to see if one of its options could help you can save hours of debugging. Spending a few minutes thinking about the general structure of the script can save hours of debugging.

Here are several ways in which you could have made your script simpler.

All variable substitutions should be inside double quotes, i.e., always write "$foo" and not just ~~$foo~~. You've done it sometimes, but not systematically. Always use double quotes unless you know why you do not want them in a particular case.

Here's a simpler way of writing your help function; it's called a “here document”.

function help()
{
    cat <<EOF
This script is responsible for simply recursively searching patterns
given in [INPUT FILE] in [PATH]. If pattern is not found then it is
written to [OPTIONAL OUTPUT FILE]. If [OPTIONAL OUTPUT FILE] is not
given the default [OUTPUT FILE] name is: out
Usage: $0 [INPUT FILE] [PATH TO DIRECTORY] [OPTIONAL OUTPUT FILE]
e.g. : $0 input_file /var/www/html/ output_file
or   : $0 help -> this help
EOF
}

Give your script a non-zero exit code to indicate failure:

if [ $# -lt 2 ]; then help; exit 2; fi
if [ ! -e "$in" ]; then echo "Input file: $1 does not exist"; exit 2; fi
if [ ! -d "$path" ]; then  echo "Path: $path does not exist"; exit 2; fi

Modifying the input file in is surprising, and you can combine the whitespace-only line removal with the multiple sed expressions that add a backslash before some characters.
```
<"$in" sed -e '/^ *$/d' -e 's,[-\\".],\\&,g' > "$tmp"
```
However the quoting you perform here is strange. Why are you quoting - and ", which are not special to grep, but not * and [ which are special? What is the syntax of the patterns supposed to be?

If you meant the patterns to be literal strings to look for, all this work is unnecessary (except for removing whitespace-only lines): call grep -F.
In the part that reads the lines from $tmp, you don't need the counter variable, you can just append to the array. You also need to pass the -r argument to the read built-in, so that it doesn't strip some backslashes.
```
while read -r line; do
    linesTable+=("$line")
done <"$tmp"
```
In the loop over the patterns, you store the output of grep in a variable, but all you're doing is testing if grep found a match. It would be a lot easier (and faster) to use the return code of grep for that. (I've also removed what is presumably debugging output from the loop; you don't need tee to append to a file, just use the redirection operator >>.)
```
for line in "${linesTable[@]}"; do
    grep -q -r -- "$line" "$path"
    if [ $? -ne 0 ]; then echo "$line" >>"$tmp"; fi
done
```
You don't need to free memory at the end of the script. If this was really a function that was part of a bigger script, you should declare them with the local builtin.

Here's a restructured version of the part of your script after the command line parsing. I've incorporated the local changes outlined above and used functions to make the structure clearer. Note that the clearer structure means I don't need to use a temporary file. I don't know if the resulting script does what you want, since you don't explain precisely what you want.

function extract_patterns () {
    <"$in" sed -e '/^ *$/d' -e 's,[-\\".],\\&,g'
}

function report_missing_patterns () {
  local pattern
  for pattern in "$@"; do
    grep -q -r -- "$pattern" "$path"
    if [ $? -ne 0 ]; then printf "%s\n" "$pattern"; fi
  done
}

process_patterns () {
  local patterns line
  patterns=()
  while read -r line; do
      patterns+=("$line")
  done
  report_missing_patterns "${patterns[@]}" | sort -u >"$out"
}

extract_patterns | process_patterns

Best Answer

Related Solutions

Bash – Clean way to write complex multi-line string to a variable

Bash avoide intrpretate special characters

Related Topic