Python KeyError – Exporting Environment Variables on Ubuntu

docker-machineenvironment-variablespythonUbuntu

I have a weird case where i have a secret.env file where i set all my environment variables as such:

secret.env

export TWITTER_CONSUMER_KEY="something"
export TWITTER_CONSUMER_SECRET="something"

Then i built a docker file to export all the variables and run the app as such:

FROM python:3.8-slim-buster

# Set the working directory to /app
WORKDIR /app

# Copy the current directory contents into the container at /app
ADD . /app

# Install the dependencies
RUN pip install -r requirements.txt

RUN find . -name \*.pyc -delete

# Export all variables
RUN /bin/bash -c "source secret.env";

# tell the port number the container should expose
EXPOSE 8083

# run the command
ENTRYPOINT ["python", "run.py"]

However, this is throwing a key error:

$ docker run --name fortweet --rm -i -t fortweet:latest bash
Traceback (most recent call last):
  File "run.py", line 1, in <module>
    from app import socketio, app
  File "/app/app/__init__.py", line 65, in <module>
    app = create_app()
  File "/app/app/__init__.py", line 38, in create_app
    my_settings = settings.TwitterSettings.get_instance()
  File "/app/app/setup/settings.py", line 47, in get_instance
    TwitterSettings()
  File "/app/app/setup/settings.py", line 14, in __init__
    self.consumer_key = os.environ["TWITTER_CONSUMER_KEY"]
  File "/usr/local/lib/python3.8/os.py", line 675, in __getitem__
    raise KeyError(key) from None
KeyError: 'TWITTER_CONSUMER_KEY'

When i run this on my windows, it works fine!

Can someone please help me on this ?

Best Answer

Change the final line to:

ENTRYPOINT ["/bin/bash", "-c", "source secret.env ; python run.py"]

instead, and remove the RUN where you are performing the sourcing. See also https://goinbigdata.com/docker-run-vs-cmd-vs-entrypoint/ for an explanation of the difference between RUN, CMD and ENTRYPOINT.

In a nutshell:

  • RUN executes command(s) in a new layer and creates a new image. E.g., it is often used for installing software packages.
  • CMD sets default command and/or parameters, which can be overwritten from command line when docker container runs.
  • ENTRYPOINT configures a container that will run as an executable.

I'm not a docker expert, but have used it at my work a handful of times so have some basic understanding of it. I think the reason this works is because of the layering, and more importantly, because the action of sourcing environment variables is purely in the realm of memory only, not storage on disk. So sourcing under a RUN doesn't actually achieving anything. You need to source them at the point of execution of your actual application, which is why the ENTRYPOINT fix above works, because we are invoking BASH, sourcing the variables into the environment, and then forking your python application, all under the same shell, at execution time.

However, this still doesn't explain why it works in your windows environment - my suspicion is that you have the environment variables set somewhere in your windows environment, so that has been working for you, but not for the reason you think it has.

Related Topic