I have my Python source structured as follows:
+-branchname/
+-dst/
+-src/
| +-library/
| | +-cleese/
| | | +-test/
| | | | +-__init__.py
| | | | +-test_cleese.py
| | | +-__init__.py
| | | +-cleese.py
| | +-palin/
| | +-test/
| | | +-__init__.py
| | | +-test_palin.py
| | +-__init__.py
| | +-palin.py
| +-productline/
| +-circus/
| | +-test/
| | | +-__init__.py
| | | +-test_circus.py
| | +-__init__.py
| | +-circus.py
| +-grail/
| +-test/
| | +-__init__.py
| | +-test_grail.py
| +-__init__.py
| +-grail.py
+-branch_root_marker
Entry points are in circus/ and grail/, as well as (possibly) each of the test/ directories, depending on how testing is implemented.
I have multiple copies of this source tree present on local storage at any one point in time (corresponding to various maintenance and feature branches), so I cannot set PYTHONPATH in my shell without some pain. (I would need to remember to change it each time I switched to work on a different branch, and I am very forgetful)
Instead, I have some logic that walks up the file tree, starting at the "current" file location, moving from leaf towards root, looking for branch_root_marker. Once the root directory of the current working copy is found, it adds library/ and productline/ to sys.path. I call this function from each entry-point in the system.
"""Add working copy (branch) to sys.path"""
import os
import sys
def _setpath():
"""Add working copy (branch) to sys.path"""
markerfile = "branch_root_marker"
path = ""
if ("__file__" in locals() or globals()) and __file__ != "__main__":
path = __file__
elif sys.argv and sys.argv[0] and not sys.argv[0] == "-c":
path = sys.argv[0]
path = os.path.realpath(os.path.expandvars(path))
while os.path.exists(path):
if os.path.exists(os.path.join(path, markerfile)):
break
else:
path = os.path.dirname(path)
errormsg = " ".join(("Could not find", markerfile))
assert os.path.exists(path) and path != os.path.dirname(path), errormsg
path = os.path.join(path, "src")
(_, subdir_list, _) = os.walk(path).next()
for subdir in subdir_list:
if subdir.startswith("."):
continue
subdir_path = os.path.join(path, subdir)
if subdir_path in sys.path:
continue
sys.path.append(subdir_path)
_setpath()
Currently, I need to keep a separate but identical copy of this function in each entry point. Even though it is quite a short function, I am quite distressed by how fragrantly the DRY principle is being violated by this approach, and would love to find a way to keep the sys.path modification logic in one place. Any ideas?
Note:- One thing that springs to mind is to install the sys.path modifying logic into a common location that is always on PYTHONPATH. Whilst this is not a terrible idea, it means introducing an installation step that needs to be carried out each time I move to a fresh environment; another thing to remember (or, more likely, forget), so I would like to avoid this if at all possible.
Best Answer
OK, I started to craft a solution that involved PEP 302 import hooks. Way too complicated.
I think you answered your own question with:
if you are already using virtualenv, put the logic you have in the "~/venv/or-whatever-path/bin/activate" script that is already there. I'd suggest listing of possible entry points and prompting for a response (list with numbers and a raw_input() statement will do) if you are not specifying one.
I'd also put the name of your branch in to $PS1 so that is shows up in the tab name or window title of your terminal. I so something similar and change the color of the prompt so I don't do the right thing in the wrong place.