Mysql – bash: use outside variables when reading multiple lines into ssh

To understand the behaviour of the bash history first you have to know the following:

1. There is the history in the history file.
2. There is the history in the memory of a bash process.
3. The history in the memory of one bash process is not synced with the history in the memory of any other bash process.
4. The history in the memory of a bash process is not synced with the history in the file, unless explicitly asked to or during some specific event (see below).

Using the default settings, the lifecycle of a bash session with regard to the history is as follows:

1. During startup bash will read the history file. The content of the history file is now in the memory of the bash process.
2. During normal use only the history in memory is manipulated.
3. During shutdown the history in memory is written to the history file, overwriting any previous content of the history file.

The seemingly nondeterministic behaviour you have observed is mostly because the content of the history file is always the history of the last closed bash session, and bash only reads the history file during startup.

Read the bash manual for a more detailed explanation of the startup and shutdown process.

Note that with default settings I mean the default settings from bash. Your distribution might have provided a .bashrc (or /etc/bash.bashrc) which change this behaviour.

By enabling the shell option histappend you can tell bash to append instead of overwriting the history file. You can enable histappend using the command shopt -s histappend. To have this option always enabled you have to put the command in your .bashrc (or other initialization file). Read more about the shopt command in the bash manual

Note that enabling histappend will not reduce the seemingly nondeterministic behaviour by much. This is because every bash session still has it's own history in memory. It is possible to have a mostly synchronized bash history. There is a guide how to get every bash process to have a mostly synced history in a thread on stack overflow.

using the builtin command history you can explicitly tell bash to read the history from file to memory, or the write from memory to file. For example: history -r will read the content of the file and append it to the history in memory. history -w will write the current history from memory to file, overwriting the previous content. This is basically what happens during shutdown. Read more about the history command in the bash manual

For completeness here is a list of the internal variables which modify the history behaviour:

• HISTFILE: the file to read from and write the history to.
• HISTFILESIZE: the maximum line count for the history file.
• HISTSIZE: the maximum line count for the history in memory.
• HISTCONTROL, HISTIGNORE, HISTTIMEFORMAT: not relevant for this discussion. Read the bash manual for details.

Gawk accepts regular expressions as field delimiters. The following eliminates spaces around the equal sign, but preserves them in the rest of the line. Quotes are added around the value so those spaces, if any, are preserved when the Bash assignment is performed. I'm assuming that the section names will be numeric variables, but if you're using Bash 4, it would be easy to adapt this to use associative arrays with the section names themselves as the indices.

awk -F ' *= *' '{ if ($1 ~ /^\[/) section=$1; else if ($1 !~ /^$/) print $1 section "=" "\"" $2 "\"" }'

Note that you may want to also do the space removal that Khaled shows (on only $1 and section) since Bash variable names can't contain spaces.

Also, this method won't work if the values contain equal signs.

Another technique would be to use a Bash while read loop and perform the assignments as the file is read, using declare which is safe from most malicious content.

foobar=1
barfoo=2  # or you could increment an index variable each time a section is found
while IFS='= ' read var val
do
    if [[ $var == \[*] ]]
    then
        section=$var
    elif [[ $val ]]
    then
        declare "$var$section=$val"
    fi
done < filename

Again, associative arrays could fairly easily be supported.