Python – Multiprocessing debug techniques

deadlockdebuggingmultiprocessingpython

I'm having trouble debugging a multi-process application (specifically using a process pool in python's multiprocessing module). I have an apparent deadlock and I do not know what is causing it. The stack trace is not sufficient to describe the issue, as it only displays code in the multiprocessing module.

Are there any python tools, or otherwise general techniques used to debug deadlocks?

Best Answer

Yah, debugging deadlocks is fun. You can set the logging level to be higher -- see the Python documentation for a description of it, but really quickly:

import multiprocessing, logging
logger = multiprocessing.log_to_stderr()
logger.setLevel(multiprocessing.SUBDEBUG)

Also, add logging for anything in your code that deals with a resource or whatnot that might be in contention. Finally, shot in the dark: spawning off child processes during an import might cause a problem.

Related Topic