Python – Remove dependency on matplotlib and qt in pyinstaller (slow startup)

matplotlibpyinstallerpythonqt

I am using pyinstaller to make a one folder executable of a python script. It starts up rather slow. The first thing in my code is print "start". It takes over 10 seconds before this appears on screen when I run the executable. I would like to bring down this delay time. I'm guessing that it would be helpfull to bring down the number of dlls needed for startup.

One thing I noticed was that matplotlib and qt5 dlls are being copied to the output folder, even though I do not intend to use these. In my own code I do not have dependencies on these libraries. My imports are:

import os
import sys
import argparse
import numpy as np
import logging
import json
from element import Square, Slanted
from skimage import measure
from skimage.segmentation import clear_border
try:
    from skimage import filters
except:
    from skimage import filter as filters

from scipy import ndimage 
from scipy.ndimage.filters import gaussian_filter, median_filter
from scipy.optimize import curve_fit
from enum import Enum
import time

First part of the pyinstall output, until it starts mentioning matplotlib, is this:

212 INFO: PyInstaller: 3.2.1
213 INFO: Python: 3.5.2
214 INFO: Platform: Windows-7-6.1.7601-SP1
249 INFO: wrote x:\script.spec
255 INFO: UPX is not available.
308 INFO: Extending PYTHONPATH with paths
['x:\\', 'x:\\script']
310 INFO: checking Analysis
311 INFO: Building Analysis because out00-Analysis.toc is non existent
312 INFO: Initializing module dependency graph...
346 INFO: Initializing module graph hooks...
352 INFO: Analyzing base_library.zip ...
6794 INFO: running Analysis out00-Analysis.toc
7585 INFO: Caching module hooks...
7596 INFO: Analyzing x:\script.py
10819 INFO: Processing pre-find module path hook   distutils
13057 INFO: Processing pre-find module path hook   site
13061 INFO: site: retargeting to fake-dir 'c:\\anaconda3\\envs\\myeenv\\lib\\site-packages\\PyInstaller\\fake-modules'
20369 INFO: Processing pre-safe import module hook   win32com
27275 INFO: Processing pre-safe import module hook   six.moves
38245 INFO: Loading module hooks...
38247 INFO: Loading module hook "hook-pydoc.py"...
38250 INFO: Loading module hook "hook-xml.dom.domreg.py"...
38253 INFO: Loading module hook "hook-xml.py"...
38256 INFO: Loading module hook "hook-distutils.py"...
38265 INFO: Loading module hook "hook-pytz.py"...
38434 INFO: Loading module hook "hook-numpy.core.py"...
38832 INFO: MKL libraries found when importing numpy. Adding MKL to binaries
38837 INFO: Loading module hook "hook-_tkinter.py"...
40127 INFO: checking Tree
40129 INFO: Building Tree because out00-Tree.toc is non existent
40131 INFO: Building Tree out00-Tree.toc
40264 INFO: checking Tree
40266 INFO: Building Tree because out01-Tree.toc is non existent
40266 INFO: Building Tree out01-Tree.toc
40301 INFO: Loading module hook "hook-PyQt5.QtCore.py"...
40918 INFO: Loading module hook "hook-matplotlib.py"...
43519 INFO: Loading module hook "hook-win32com.py"...

How do I find out which library is giving me the matplotlib/qt5 dependency? Are there other things I can do to improve startup time?

I've tried using:

pyinstaller script.py --exclude-module matplotlib,qt5

But it still starts including matplotlib.backends etc…

Edit

It appears that skimage.segmentation is including matplotlib…

Traceback (most recent call last):
  File "myenv\script.py", line 11, in <module>
  File "c:\anaconda3\envs\myenv\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 389, in load_module
    exec(bytecode, module.__dict__)
  File "site-packages\skimage\segmentation\__init__.py", line 6, in <module>
  File "c:\anaconda3\envs\myenv\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 389, in load_module
    exec(bytecode, module.__dict__)
  File "site-packages\skimage\segmentation\boundaries.py", line 5, in <module>
  File "c:\anaconda3\envs\myenv\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 389, in load_module
    exec(bytecode, module.__dict__)
  File "site-packages\skimage\morphology\__init__.py", line 1, in <module>
  File "c:\anaconda3\envs\myenv\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 389, in load_module
    exec(bytecode, module.__dict__)
  File "site-packages\skimage\morphology\binary.py", line 6, in <module>
  File "c:\anaconda3\envs\myenv\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 389, in load_module
    exec(bytecode, module.__dict__)
  File "site-packages\skimage\morphology\misc.py", line 5, in <module>
  File "c:\anaconda3\envs\myenv\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 389, in load_module
    exec(bytecode, module.__dict__)
  File "site-packages\skimage\morphology\selem.py", line 3, in <module>
  File "c:\anaconda3\envs\myenv\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 389, in load_module
    exec(bytecode, module.__dict__)
  File "site-packages\skimage\draw\__init__.py", line 1, in <module>
  File "c:\anaconda3\envs\myenv\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 389, in load_module
    exec(bytecode, module.__dict__)
  File "site-packages\skimage\draw\draw.py", line 6, in <module>
  File "c:\anaconda3\envs\myenv\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 389, in load_module
    exec(bytecode, module.__dict__)
  File "site-packages\skimage\_shared\_geometry.py", line 4, in <module>
ImportError: No module named 'matplotlib'
Failed to execute script myenv

Edit 2

Small test. Used pyinstaller to make a one folder executable of the following script:

import numpy as np
print(np.array([0]))

Took about 6 seconds to run.
This morning I realized I was running from a samba mount and that this may influence startup time considerably.
The same executable took 0.7 seconds when started locally! I am still interested in bringing down the size of the distribution though (> 500MB).

Best Answer

The issue with excluding modules is, that pyinstaller interprets the command line argument to --exclude-module as one package, hence

pyinstaller script.py --exclude-module matplotlib,qt5

results in excludes=['matplotlib,qt5'] in the .spec file.

To exclude both packages, they need to be separated:

pyinstaller script.py --exclude-module matplotlib --exclude-module qt5
Related Topic