Environment variables using subprocess.check_output Python

python-3.3subprocess

I'm trying to do some basic module setups on my server using Python. Its a bit difficult as I have no access to the internet.

This is my code

import sys
import os
from subprocess import CalledProcessError, STDOUT, check_output

def run_in_path(command, dir_path, env_var=''):
    env_var = os.environ["PATH"] = os.environ["PATH"] + env_var
    print(env_var)
    try:
        p = check_output(command, cwd=dir_path, stderr=STDOUT)
    except CalledProcessError as e:
        sys.stderr.write(e.output.decode("utf-8"))
        sys.stderr.flush()
        return e.returncode
    else:
        return 0

def main():
    requests_install = run_in_path('python setup.py build', 
                'D:\installed_software\python modules\kennethreitz-requests-e95e173')
    SQL_install = run_in_path('python setup.py install', # install SQL module pypyodbc
                   'D:\installed_software\python modules\pypyodbc-1.3.3\pypyodbc-1.3.3')
    setup_tools = run_in_path('python setup.py install', # install setup tools
                   'D:\installed_software\python modules\setuptools-17.1.1')
    psycopg2 = run_in_path('easy_install psycopg2-2.6.1.win-amd64-py3.3-pg9.4.4-release', # install setup tools
                   'D:\installed_software\python modules', ';C:\srv_apps\Python33\Scripts\easy_install.exe')
    print('setup complete')

if __name__ == "__main__":
    sys.exit(main())

now it gets tricky when i start trying to use easy install. It appears my env variables are not being used by my subprocess.check_output call

File "C:\srv_apps\Python33\lib\subprocess.py", line 1110, in _execute_child
    raise WindowsError(*e.args)
FileNotFoundError: [WinError 2] The system cannot find the file specified

I don't want to have to upgrade to 3.4 where easy install is installed by default because my other modules are not supported on 3.4. My main challenge is the subprocess.check_call method does not take environment variables as an input and im wary of trying to use Popen() as I have never really got it to work successfully in the past. Any help would be greatly appreciated.

Best Answer

PATH should contain directories e.g., r'C:\Python33\Scripts', not files such as: r'C:\Python33\Scripts\easy_install.exe'

Don't hardcode utf-8 for an arbitrary command, you could enable text mode using universal_newlines parameter (not tested):

#!/usr/bin/env python3
import locale
import sys
from subprocess import CalledProcessError, STDOUT, check_output

def run(command, *, cwd=None, env=None):
    try:
        ignored = check_output(command, cwd=cwd, env=env, 
                               stderr=STDOUT,
                               universal_newlines=True)
    except CalledProcessError as e:
        sys.stderr.write(e.output)
        sys.stderr.flush()
        return e.returncode
    else:
        return 0

Example:

import os

path_var = os.pathsep.join(os.environ.get('PATH', os.defpath), some_dir)
env = dict(os.environ, PATH=path_var)
run("some_command", cwd=some_path, env=env)
run("another_command", cwd=another_path, env=env)
Related Topic