Bash Scripts Not Working Through Supervisord – Troubleshooting

bashshell-scriptingsupervisordubuntu-18.04

I'm using the following systems:

  • Ubuntu 18.04.2 LTS

  • bash 4.4.19(1)-release

  • supervisord 3.3.1-1.1

I have written two scripts that work pretty well from the CLI, but when I try to use supervisord with them, they don't work.

When I execute them by hand, I cd into the directory folder where they both reside and I issue the commands. Generally, I have two different ssh sessions going and issue one command in one ssh window and the other command in the other ssh window.

root@LPRcloud:~/lpr-scripts# ./lpr-loader.sh

and

root@LPRcloud:~/lpr-scripts# ./file-minder.sh -s /home/ -d /home/rslsync/lpr-sync/ -e /home/rslsync -T

I have supervisor set up in /etc/supervisor/supervisord.conf with this directive at the bottom:

files = /etc/supervisor/conf.d/*.conf

I have placed the following file in the directory:

/etc/supervisor/conf.d/lpr-loader.conf

Here are the relevant lines from that file:

[program:lpr-loader]
command=bash -c "/root/lpr-scripts/lpr-loader.sh"   ; the program (relative uses PATH, can take args)
process_name=%(program_name)s  ; process_name expr (default %(program_name)s)
numprocs=1                    ; number of processes copies to start (def 1)

[program:file-minder]
command=bash -c "/root/lpr-scripts/file-minder.sh -s /home/ -d /home/rslsync/lpr-sync/ -e /home/rslsync"  ; the program (relative uses PATH, can take args)
process_name=%(program_name)s  ; process_name expr (default %(program_name)s)
numprocs=1                    ; number of processes copies to start (def 1)

After making any changes to the lpr-loader.conf file, I run these commands, in this order:

root@LPRcloud:~/lpr-scripts# supervisorctl reread
file-minder: available
root@LPRcloud:~/lpr-scripts# supervisorctl status all
lpr-loader                       RUNNING   pid 6173, uptime 5:49:11
root@LPRcloud:~/lpr-scripts# service supervisor stop
root@LPRcloud:~/lpr-scripts# service supervisor start
root@LPRcloud:~/lpr-scripts# supervisorctl status all
file-minder                      RUNNING   pid 91580, uptime 0:00:02
lpr-loader                       RUNNING   pid 91581, uptime 0:00:02
root@LPRcloud:~/lpr-scripts# supervisorctl status all
file-minder                      RUNNING   pid 91580, uptime 0:00:06
lpr-loader                       RUNNING   pid 91581, uptime 0:00:06

So I know both lpr-loader and file-minder are supposedly running. I even pull one of these:

root@LPRcloud:~/lpr-scripts# ps aux | egrep 'lpr|minder'
root      15492  0.0  0.0  16948  1008 pts/0    S+   23:40   0:00 grep -E --color=auto lpr|minder
root      91580  0.0  0.0  23860  3648 ?        S    23:38   0:00 /bin/bash /root/lpr-scripts/file-minder.sh -s /home/ -d /home/rslsync/lpr-sync/ -e /home/rslsync
root      91581 14.0  0.0  23988  3704 ?        S    23:38   0:20 /bin/bash /root/lpr-scripts/lpr-loader.sh

But yet the scripts are not working in the background. And if I manually run these scripts, they work correctly.

What am I doing wrong here?

Is there a PATH directive that I could/should use in lpr-loader.conf that would tell supervisor the working directory needed?

Thanks in advance!

After implementing suggestions from the comments below, I have gotten file-minder to work correctly, but lpr-loader is not working. supervisorctl status all says it is RUNNING but there is no change in the files or database occuring. Is there any logging that supervisor is performing that I could use to tell me what is going on?

By the way, I investigated the environment directive in supervisor. I added it in the [program:lpr-loader] section of my conf.d and it did not seem to help. (Or hurt!)

Best Answer

Looks like you're trying to maintain sync between files in different directories. Do your scripts have the necessary infinity loops in place?

I'd suggest you debug with a basic script to determine if the issue is with your scripts of with supervisord eg

while true; do echo "test" >> /tmp/test.log; sleep 5; done