Python and Lisp – Python Decorators and Lisp Macros

clojurelispmacrospython

When looking Python decorators someone made the statement, that they are as powerful as Lisp macros (particularly Clojure).

Looking at the examples given in PEP 318 it looks to me as if they are just a fancy way of using plain old higher-order functions in Lisp:

def attrs(**kwds):
    def decorate(f):
        for k in kwds:
            setattr(f, k, kwds[k])
        return f
    return decorate

@attrs(versionadded="2.2",
       author="Guido van Rossum")
def mymethod(f):
    ...

I haven’t seen any code transforming in any of the examples, like described in Anatomy of a Clojure Macro. Plus, Python’s missing homoiconicity could make code transformations impossible.

So, how do these two compare and can you say they are about equal in what you can do? Evidence seems to point against it.

Edit: Based on a comment, I’m looking for two things: comparison on “as powerful as” and on “as easy to do awesome things with”.

Best Answer

A decorator is basically just a function.

Example in Common Lisp:

(defun attributes (keywords function)
  (loop for (key value) in keywords
        do (setf (get function key) value))
  function)

In above the function is a symbol (which would be returned by DEFUN) and we put the attributes on the symbol's property list.

Now we can write it around a function definition:

(attributes
  '((version-added "2.2")
    (author "Rainer Joswig"))

  (defun foo (a b)
    (+ a b))

)

If we want to add a fancy syntax like in Python, we write a reader macro. A reader macro allows us to program on the level of s-expression syntax:

(set-macro-character
 #\@
 (lambda (stream char)
   (let ((decorator (read stream))
         (arg       (read stream))
         (form      (read stream)))
     `(,decorator ,arg ,form))))

We then can write:

@attributes'((version-added "2.2")
             (author "Rainer Joswig"))
(defun foo (a b)
  (+ a b))

The Lisp reader reads above to:

(ATTRIBUTES (QUOTE ((VERSION-ADDED "2.2")
                    (AUTHOR "Rainer Joswig")))
            (DEFUN FOO (A B) (+ A B)))

Now we have a form of decorators in Common Lisp.

Combining macros and reader macros.

Actually I would do above translation in real code using a macro, not a function. 

(defmacro defdecorator (decorator arg form)
  `(progn
     ,form
     (,decorator ,arg ',(second form))))

(set-macro-character
 #\@
 (lambda (stream char)
   (declare (ignore char))
   (let* ((decorator (read stream))
          (arg       (read stream))
          (form      (read stream)))
     `(defdecorator ,decorator ,arg ,form))))

The use is as above with the same reader macro. The advantage is that the Lisp compiler still sees it as a so-called top-level form - the *file compiler treats top-level forms specially, for example it adds information about them into the compile-time environment. In the example above we can see that the macro looks into the source code and extracts the name.

The Lisp reader reads the above example into:

(DEFDECORATOR ATTRIBUTES
  (QUOTE ((VERSION-ADDED "2.2")
           (AUTHOR "Rainer Joswig")))
  (DEFUN FOO (A B) (+ A B)))

Which then gets macro expanded into:

(PROGN (DEFUN FOO (A B) (+ A B))
       (ATTRIBUTES (QUOTE ((VERSION-ADDED "2.2")
                           (AUTHOR "Rainer Joswig")))
                   (QUOTE FOO)))

Macros are very different from reader macros.

Macros get the source code passed, can do whatever they want and then return source code. The input source does not need to be valid Lisp code. It can be anything and it could be written totally different. The result has to be valid Lisp code then. But if the generated code is using a macro too, then the syntax of the code embedded in the macro call could again be a different syntax. A simple example: one could write a math macro which would accept some kind of mathematical syntax:

(math y = 3 x ^ 2 - 4 x + 3)

The expression y = 3 x ^ 2 - 4 x + 3 is not valid Lisp code, but the macro could for example parse it and return valid Lisp code like this:

(setq y (+ (* 3 (expt x 2))
           (- (* 4 x))
           3))

There are many other use cases of macros in Lisp.

Related Topic