
Python decorators they don't tell you about - emidln
https://github.com/hchasestevens/hchasestevens.github.io/blob/master/notebooks/the-decorators-they-wont-tell-you-about.ipynb
======
ramnes
This notebook should be named "how to abuse decorators". Honestly, there is
very little here that you want to use in a real project.

I completely agree in that saying "a decorator is a function that takes a
function and returns a function" isn't technically correct since you can do
much more than just that; but for readability, maintainability and other real
world purposes, you might want to keep to that definition.

~~~
rhaps0dy
I disagree. From the end section of "real-world usage examples", there is this
library
[https://github.com/mrocklin/multipledispatch/](https://github.com/mrocklin/multipledispatch/)
that implements "Decorators for registration. It's used in at least one
(probably many more) serious open-source project that I know of: GPflow
[https://github.com/GPflow/GPflow/blob/master/gpflow/expectat...](https://github.com/GPflow/GPflow/blob/master/gpflow/expectations_quadrature.py#L63)
.

"Decorators for preconditions" look pretty neat as well. It's a little sad
that the preconditions themselves have to be written inside strings though.

~~~
kuschku
> It's a little sad that the preconditions themselves have to be written
> inside strings though.

Isn’t it possible to pass a lambda instead?

~~~
fireflies_
Yes. Lambdas in Python are awkward, but you can definitely use either a lambda
or a named function.

In fact, you could define a number of useful precondition functions in a
single module and use them throughout a project. A couple of higher-order
functions could make the post's examples safer and nicer-looking too. For
example:

    
    
        @precondition(starts_with('The year is '))
    

where the starts_with precondition is:

    
    
        def starts_with(param):
            def test(s):
                return s.startswith(param)
            return test

~~~
ivanjr0
> Lambdas in Python are awkward

Can you elaborate on that, please?

Considering the examples given by the author, I think they would be a better
option.

~~~
abhirag
Not the OP but I think he was referring to the restrictions python puts on
lambda expressions:

\- they can only contain a single expression

\- they cannot contain statements

~~~
Sohcahtoa82
Python has lambdas that have multiple expressions and statements.

They're called functions.

~~~
yen223
Python functions aren't anonymous

------
tedmiston
If you're on mobile, view the page as a desktop site in your browser to see
the IPython notebook rendered properly. This works on iOS at least.

~~~
dmurray
Same in Chrome on Android. Thanks!

------
osteele
Consider functools.wraps[1] in your own decorators. This uses
functools.update_wrapper to copy the name, docstring, and some other other
attributes[2] from the wrapped function to the wrapper.

[1]:
[https://docs.python.org/3/library/functools.html#functools.w...](https://docs.python.org/3/library/functools.html#functools.wraps)

[2]:
[https://docs.python.org/3/library/functools.html#functools.u...](https://docs.python.org/3/library/functools.html#functools.update_wrapper)

EDIT: s/wrapped/wraps/; tip of the hat to @ramnes

~~~
ramnes
Yes! It's functools.wraps*, though. :)

------
dveeden2
URL for mobile:
[https://nbviewer.jupyter.org/github/hchasestevens/hchasestev...](https://nbviewer.jupyter.org/github/hchasestevens/hchasestevens.github.io/blob/master/notebooks/the-
decorators-they-wont-tell-you-about.ipynb)

------
srathi
I've used decorators mainly to retry a function if it throws an exception,
like retrying DNS queries multiple times, etc. I've never got a chance to do
more with them, but this article gives me an incentive to try more.

~~~
sheraz
Yep. I do the exact thing for a lot of network related things (dns lookups,
http requests). I use this nifty library:
[https://github.com/jd/tenacity](https://github.com/jd/tenacity)

~~~
creeble
Wow, both terrific tips, thanks!

------
alexhayes
My favourite library for writing decorators that "behave nicely" is wrapt –
[https://wrapt.readthedocs.io/en/latest/](https://wrapt.readthedocs.io/en/latest/)

There's also a wealth of resource in the docs!

------
chairmanwow
Medium to large Django projects were the first times that I began to see the
power and usefulness of decorators. The most useful one I've used was a
decorator to require certain JSON parameters in a POST and optionally specify
a list of possible parameters. Makes the API code logic infinitely simpler,
especially if you write some custom middleware to handle / report API errors.
The code is clear and declarative, and errors are handled gracefully all in
one place.

~~~
dozzie
Which is very funny, because you'd get the same without decorators by simply
using properly defined RPC protocol instead of ad hoc developed entangled mess
that passes as REST.

~~~
icebraining
You still have to define the procedures to use with the protocol using
language features - like decorators. For example:
[http://spyne.io/docs/2.10/reference/decorator.html](http://spyne.io/docs/2.10/reference/decorator.html)

------
osteele
functools.lru_cache[1] from the standard library is a decorator that
memoizes[2] a function.

[https://github.com/osteele/callgraph/blob/master/examples/ca...](https://github.com/osteele/callgraph/blob/master/examples/callgraph-
decorator-examples.ipynb) has some examples of using @lru_cache, together with
a decorator (@callgraph) defined in that repo, that helps visualize recursion
and memoization.

[1]:
[https://docs.python.org/3/library/functools.html#functools.l...](https://docs.python.org/3/library/functools.html#functools.lru_cache)

[2]:
[https://en.wikipedia.org/wiki/Memoization](https://en.wikipedia.org/wiki/Memoization)

------
aib
WTH is that call to os.system("rm -rf /") doing there?! Issued a PR.

~~~
fiblye
Reading the text around it makes it pretty clear: it's code that shouldn't
ever be called.

~~~
kipari
It’s still widely inappropriate.

~~~
Walkman
It is showing confidence that the author know what he is talking about,
because as you can see he ran the code and that part did not run.

~~~
kzrdude
jupyter notebooks are easy to share and run locally, so it's still
inappropriate

~~~
zbentley
curl | bash is easy to run locally too, and very few people actually read the
code that runs to check for “rm -rf /“-type things. When executing untrusted
code from the internet, either you read it or your fate is in the author’s
hands. No exceptions, not even for beginners getting started in programming
via jupyter notebooks.

~~~
aib
Well, then let's go with the fact that a function called "never_called" is
trying to delete all the files on the computer. At best it's a misnomer.

And it is right that the fate is in the author's hands. It's those hands that
we are trying to tame.

------
qualitytime
I've been exposed to decorators (starting with Java) and have 'gotten' them
and I will proudly admit that I despise decorators and don't think they are
great at all.

~~~
ben174
Just hate them? Did they steal yoUr lunch money? Or is there something more
specific you hate about them?

~~~
qualitytime
Mate, you've got some broken images on your homepage.

~~~
ben174
Thank you for the heads up! Will fix when I get home

------
praulv
It's all very well writing fancy decorators, but unittesting them,
specifically, mocking decorators is trickier.

------
bbayles
I had fun a little while back abusing decorators to alter the return value of
generator functions:
[https://sites.google.com/site/bbayles/index/decorator_factor...](https://sites.google.com/site/bbayles/index/decorator_factory)

------
smortaz
Uploaded to Azure notebooks in case anyone wants to run/play with the code:

[https://notebooks.azure.com/smortaz/libraries/PythonDecorato...](https://notebooks.azure.com/smortaz/libraries/PythonDecorators/tree/notebooks)

Click Clone & Run.

------
agumonkey
one guy made a "pure" decorator that used the ast module to assess the purity
(to a certain extent) of a function and allow optimisation on the way

fun

crazy and fun

~~~
roberto
I once wrote a decorator that translated the function to Javascript and
spawned a web server, allowing the function to be computed in parallel by
multiple browsers:
[https://bitbucket.org/robertodealmeida/parallejax](https://bitbucket.org/robertodealmeida/parallejax)

~~~
agumonkey
cute one

------
mythrwy
Decorators are great. One or two or three. 4 is pushing it. Some people get
all carried away though. I saw code recently that had a stack of like 8-10
decorators on top of each function. More decorators than function in every
case. Thinking it might be time to re-evaluate basic structure in this case
but maybe I just don't get it.

------
lmm
> Nothing in x = d(x) necessitates that d is a function - d just has to be
> callable!

Python being duck-typed, what's the distinction?

~~~
Sohcahtoa82
"callable" essentially just means you can put a "()" after the name in the
code and it works.

You can "call" a class. Doing so allocates an instance of that class (Or calls
the __new__ function if one is defined) and returns it, first calling the
__init__ function of that class if necessary.

You can also "call" an object if its class has a __call__ function defined.

~~~
lmm
Indeed, but isn't that exactly what "quacking like a function" is?

------
orblivion
The funny thing is that all of the "useful examples" were functions that took
functions and returned functions.

------
brian_herman
Can you make a decorator that references a class instance and has parameters?

~~~
JonathonW
Yes on the former (works just like you'd expect); to make a decorator take
parameters, to do something like:

    
    
        @decorator_with_params("Some string")
        def my_function():
            pass
    

you have to make 'decorator_with_params' a function that returns a _decorator_
; i.e.

    
    
        def decorator_with_params(param):
            def real_decorator(function):
                def inner(*args, **kwargs):
                    print(param)
                    function(*args, **kwargs)
                return inner
            return real_decorator
    

(You can also do something similar with callable objects, but that's ickier
IMO.)

Demo of both (decorator as instance method and decorator with parameters):
[https://gist.github.com/jonathonw/dac88b715aca4e4f1b2f513e1f...](https://gist.github.com/jonathonw/dac88b715aca4e4f1b2f513e1feaa102)

~~~
myaso
So just a function that returns a closure? It can be a real pain to read code
like this if it's overdone -- I really would need to trust somebody to let
them go _meta_ at their discretion.

~~~
mst
I can't see any reason you can't do something like:

    
    
        class MetaDec(args, here):
            ...
            def __call__(self, func):
                # Decorate function here
    
        def meta_dec(args, here):
            return MetaDec(args, here)
    
        @meta_dec
        def decorated:
            ...
    

(apologies for any syntax fails, python is not my best language)

At which point one would imagine you can abstract out the __call__ part and
just have a 'decorate' routine on any class designed to work what way or
similar.

Whether this is an improvement or not over closures almost certainly depends
on the use case, of course.

~~~
myaso
Does the code actually work? Wouldn't you need to create a instance within
meta_dec and return the result of the __call__ method. See at first glance I
won't think it would work if I didn't know __call__ is a magic method in
python. I don't think I'm being overly picky but I found the previous example
much easier to read -- since it's a pattern I defaulted to myself before
across several different languages. Mixing classes and FP like that doesn't
resonate well with me tbh, but yeah you could do it ... but should you?

~~~
mst
> Does the code actually work?

Given its reliance on '...' and the fact I'm not great at python and typed it
straight into the comment ... no. No it doesn't.

> Wouldn't you need to create a instance within meta_dec and return the result
> of the __call__ method.

No, like with ES6 parameterised decorators return a function that then gets
called with the thing being decorated.

> See at first glance I won't think it would work if I didn't know __call__ is
> a magic method in python.

(1) my reply was assuming the context of us both having just read the article,
which has an example with __call__

(2) as I said in my original comment, you would abstract this away (and
document it), I was sketching out an alternative implementation path to allow
you to have an object instead of a closure since you didn't seem to be happy
with closures, not offering a complete solution

> Mixing classes and FP like that doesn't resonate well with me tbh, but yeah
> you could do it ... but should you?

(3) if you don't like closures _and_ you don't like objects, I'm not sure what
other combined function+data abstraction is available in python that you
wouldn't also dislike

(4) I'd probably consider doing it if I had a bunch of similar decorators and
found the functional approach was starting to groan at the seams - I use
objects for 'parameterised bag of functions, possibly with inheritance' quite
a lot for that purpose

~~~
myaso
Sorry, must of given off the wrong impression. I'm quite happy with FP in
general, I'm just aware it is possible to write some pretty horrendous things.
But yeah you are completely right about (4), at some point it doesn't seem
like overkill anymore.

~~~
mst
I was very much trying to find an answer to your

> It can be a real pain to read code like this if it's overdone

wherein (4) is, when things start getting complicated, a way to end up
avoiding things turning into closure soup. Also being able to selectively
override things with inheritance can really help factor things out and clean
things up.

> at some point it doesn't seem like overkill anymore.

Though I suspect "at that point, ask yourself if you're overengineering it and
only if you're sure you aren't consider moving to a class" definitely applies
:)

------
nabla9
Python decorators are like poor man's method combination.

~~~
osteele
They're a kind of advice [1]. Advice was indeed subsumed by CLOS method
combination, but existed prior to that and has a life of its own, for example
in Aspect Oriented Programming.

[1]:
[https://en.wikipedia.org/wiki/Advice_(programming)](https://en.wikipedia.org/wiki/Advice_\(programming\))

[2]: [https://en.wikipedia.org/wiki/Aspect-
oriented_programming](https://en.wikipedia.org/wiki/Aspect-
oriented_programming)

------
adamnemecek
To be honest, decorators were always, idk, I felt like there was something off
about them.

~~~
UncleEntity
> To be honest, decorators were always, idk, I felt like there was something
> off about them.

I think they're one of those things you don't need to use unless you _need_ to
use them...though, honestly, you probably don't ever _need_ to use them unlike
metaclasses where sometimes they are the only way to do something in a
reasonable manner.

~~~
anowlcalledjosh
> _you probably don 't ever need to use them_

By their definition, you don't ever need to use them. They're just a
convenient syntactic sugar.

    
    
      @foo
      def bar():
          ...
    

is, as far as I know, exactly equivalent to

    
    
      def bar():
          ...
      bar = foo(bar)

