Python – Installing scipy in virtualenv on Elastic Beanstalk

elastic-beanstalkflaskpippythonvirtualenv

I'm using Amazon's Elastic Beanstalk to deploy an example Flask app. I can get a simple "Hello World" app deployed perfectly, but now I'm trying to deploy the app with scipy as a requirement.

I've included the necessary packages in my .ebextensions/:

packages:
    yum:
        gcc-c++: []
        gcc-gfortran: []
        python27-devel: []
        atlas-sse3-devel: []
        lapack-devel: []
        libpng-devel: []
        zlib-devel: []
        postgresql93-devel: []

If I leave scipy and numpy in the requirements.txt file, the deploy fails because numpy has to be installed before scipy.

I can fix this by commenting out scipy from my requirements.txt, and adding a container_commands section to my .ebextensions:

container_commands:
    01_install_scipy:
        command: "pip install scipy"

I don't like this approach because I want all of my requirements to live in my requirements.txt file for development purposes. Selectively commenting out pip requirements from the requirements.txt file feels wrong and can get complicated if I have a bunch of other libraries that depend on scipy.

Additionally, building scipy from source takes a very long time, especially on relatively small EC2 instances. I have tried installing using yum, but this leads to using old versions of scipy and not having scipy in the virtual environment.

So, I have two problems:

  1. requirements.txt: Is there any way to install scipy to my virtual environment that doesn't require me to comment out selective requirements from my requirements.txt file?
  2. Speed: Is there any way to pre-compile scipy and still make it available in the virtual environment?

Best Answer

You should package your application (zip) before to deploy it. This package should include all deps your application need so you don't have to pre-install module when deploying.