Linux – Unable to run a python3 script in CentOs7 [with python2.7] via cronjob

bashcentoscronlinuxpython

I have been trying to deal with this strage problem all morning.

I have a simple crawler.py file, which contans a code that can only run under python3 to be exact under python3.3 evenrionment

Now, my default CentOs7 machine comes with python2.7 so, I am forced to switch using python3 to run my script, example: 

scl enable python33 bash

example. 

[root@bt py]# python -V
Python 2.7.5
[root@bt py]# scl enable python33 bash
[root@bt py]# python -V
Python 3.3.2

as you can see, it changed the version after which, I can run simple run crawler.py without a problem as easy as python crawler.py.

But, I need this python file to run via cron, at specific time of the week.

The problem is that once I log out of my VPS, python2.7 becomes the default and if cron triggers it, it won't run.
For this I thought I should use the python3 binary instead of just python .. so, I tried with the following 

/opt/rh/python33/root/usr/bin/python3.3 crawler.py

But, it seems the binary does not appear unles you run scl enable python33 bash

you can see this for yourself, 

[root@bt ~]# whereis python
python: /usr/bin/python /usr/bin/python2.7 /usr/lib/python2.7 /usr/lib64/python2.7 /etc/python /usr/include/python2.7 /usr/share/man/man1/python.1.gz
[root@bt ~]# scl enable python33 bash
[root@bt ~]# whereis python
python: /usr/bin/python /usr/bin/python2.7 /usr/lib/python2.7 /usr/lib64/python2.7 /etc/python /usr/include/python2.7 /opt/rh/python33/root/usr/bin/python3.3m /opt/rh/python33/root/usr/bin/python3.3-config /opt/rh/python33/root/usr/bin/python3.3m-config /opt/rh/python33/root/usr/bin/python /opt/rh/python33/root/usr/bin/python3.3 /usr/share/man/man1/python.1.gz

The binary, only appears after python33 is called via scl. So, all the time I have to tell cron to somehow execute scl enable python bash command so my python3 scripts runs without issues.

So, I created a bash file as 

#!/bin/bash 
$(scl enable python33 bash)
python crawler.py 
# alternatively I tried the below solution also. 
# /opt/rh/python33/root/usr/bin/python3.3 crawler.py

But, once bash executes the first command it exits the process or spawns other processes. even tried putting this

scl enable python33 bash && python crawl.py

also 

scl enable python33 bash && /opt/rh/python33/root/usr/bin/python3.3 crawl.py

So, in a nutshell, it seem there is no way to run a python3 script via cronjob, if you have anything but python3 as default in your machine.

Best Answer

If I understand it correctly, then you want to use cron to run a bash script, in which a call to python results in the scl'ed python33 version.

I just implemented and tested that here, on a Scientific Linux box with python33 from scl, just as in your case.

Create a bash script:

$ cat /home/username/mybashfile.sh 
#!/bin/bash
echo `date` >> /home/username/pythonversionfile
python --version >> /home/username/pythonversionfile 2>&1

and create a cron entry like this:

* * * * * /usr/bin/scl enable python33 'bash -c /home/username/mybashfile.sh'

Which means you call scl to use the python33 environment, which calls bash, directly with the call to the script you created before.

This works nicely and gives me the following output:

$ tail pythonversionfile 
Wed Mar 30 17:36:01 CEST 2016
Python 3.3.2
Wed Mar 30 17:37:02 CEST 2016
Python 3.3.2
Wed Mar 30 17:38:01 CEST 2016
Python 3.3.2
Wed Mar 30 17:39:01 CEST 2016
Python 3.3.2
Wed Mar 30 17:40:01 CEST 2016
Python 3.3.2

So you can see that the bash script is called by cron every minute, and inside the bash script the environment is correctly set such that python results in python 3.3.2 being called.

Related Topic