Python profiling using line_profiler – clever way to remove @profile statements on-the-fly

decoratorprofilingpythonpython-decorators

I want to use the excellent line_profiler, but only some of the time. To make it work I add

@profile

before every function call, e.g.

@profile
def myFunc(args):
    blah
    return

and execute

kernprof.py -l -v mycode.py args

But I don't want to have to put the @profile decorators in by hand each time, because most of the time I want to execute the code without them, and I get an exception if I try to include them, e.g.

mycode.py args

Is there a happy medium where I can dynamically have the decorators removed based on some condition switch/argument, without having to do things manually and/or modify each function too much?

Best Answer

Instead of removing the @profile decorator lines, provide your own pass-through no-op version.

You can add the following code to your project somewhere:

try:
    # Python 2
    import __builtin__ as builtins
except ImportError:
    # Python 3
    import builtins

try:
    builtins.profile
except AttributeError:
    # No line profiler, provide a pass-through version
    def profile(func): return func
    builtins.profile = profile

Import this before any code using the @profile decorator and you can use the code with or without the line profiler being active.

Because the dummy decorator is a pass-through function, execution performance is not impacted (only import performance is every so lightly affected).

If you don't like messing with built-ins, you can make this a separate module; say profile_support.py:

try:
    # Python 2
    import __builtin__ as builtins
except ImportError:
    # Python 3
    import builtins

try:
    profile = builtins.profile
except AttributeError:
    # No line profiler, provide a pass-through version
    def profile(func): return func

(no assignment to builtins.profile) and use from profile_support import profile in any module that uses the @profile decorator.

Related Solutions

Python – What’s the canonical way to check for type in Python

To check if o is an instance of str or any subclass of str, use isinstance (this would be the "canonical" way):

if isinstance(o, str):

To check if the type of o is exactly str (exclude subclasses):

if type(o) is str:

The following also works, and can be useful in some cases:

if issubclass(type(o), str):

See Built-in Functions in the Python Library Reference for relevant information.

One more note: in this case, if you're using Python 2, you may actually want to use:

if isinstance(o, basestring):

because this will also catch Unicode strings (unicode is not a subclass of str; both str and unicode are subclasses of basestring). Note that basestring no longer exists in Python 3, where there's a strict separation of strings (str) and binary data (bytes).

Alternatively, isinstance accepts a tuple of classes. This will return True if o is an instance of any subclass of any of (str, unicode):

if isinstance(o, (str, unicode)):

Python – How to import a module given its name as string

With Python older than 2.7/3.1, that's pretty much how you do it.

For newer versions, see importlib.import_module for Python 2 and Python 3.

You can use exec if you want to as well.

Or using __import__ you can import a list of modules by doing this:

>>> moduleNames = ['sys', 'os', 're', 'unittest'] 
>>> moduleNames
['sys', 'os', 're', 'unittest']
>>> modules = map(__import__, moduleNames)

Ripped straight from Dive Into Python.

Best Answer

Related Solutions

Python – What’s the canonical way to check for type in Python

Python – How to import a module given its name as string

Related Topic