
Going beyond the idiomatic Python - okaleniuk
https://hackernoon.com/going-beyond-the-idiomatic-python-a321b6c6a5e6
======
tzs
> People don’t speak entirely in idioms unless they are totally off their
> rockers.

My first thought on reading that was to think of Darmok and Jalad at Tanagra.
Then I realized that was allegory, not idiom. Then I realized I had no idea
what the actual definition of "idiom" is, and looked it up: "a group of words
established by usage as having a meaning not deducible from those of the
individual words (e.g., rain cats and dogs, see the light)."

...and now I'm confused because nearly everything we call an idiom in
programming isn't. What would be the correct term for whatever it is we are
currently calling programming idioms?

For example, one from the idiomatic Python book that the article criticizes:

    
    
      def contains_zero(iterable):
        # 0 is "Falsy," so this works
        return not all(iterable)
    

That one was confusing to me, because I'm not a heavy Python user. I've never
encountered all() before. However, after looking up the all() function in the
Python documentation that code becomes immediately obvious.

It seems that most of what we call idioms in programming are just using more
advanced or more obscure language features or library functions to do
something in a more compact or efficient way at the cost of immediate clarity
to people who are not as advanced in the language. We've decided what these do
is needed often enough that we agree that these are the way people should do
these things, even if they don't understand what is actually going on.

I'm not sure programming languages can have actual idioms, except perhaps in
the case of language bugs or bad/missing documentation.

The combination of language plus hardware could have idioms, like nested loops
iterating a multidimensional array should have the outer loop on the first
dimension, the second loop on the second dimension, and so on to minimize
jumping around in memory and so hopefully interact better with the cache.

~~~
empthought
It's even worse; the book is wrong. That implementation of contains_zero is
not idiomatic to anyone who writes Python.

~~~
hxtk
Not only is it not idiomatic; it's functionally incorrect.

An iterable containing `[]`, `None`, or any other value that is false in a
boolean context would return true. They've assumed that `(0 -> false) ->
(false -> 0)`, which is simply not true.

Even if you're dealing with a context that cannot pass in an iterable with
non-zero "falsy" values then it needs to be explicitly documented. At that
point, why not "document" it by being explicit with the code. This is how I'd
do it, but I feel like it could be made cleaner:

    
    
        def contains_zero(iterable):
            return any([i is 0 for i in iterable])
    

Edit: I forgot how to python. It's `return 0 in iterable`.

------
brudgers
The worst idiom of Python is the idea of Pythonic. It is usually prefixed with
"un" and used pejoratively. The rest of the cases it is used to rationalize
arbitrary biases such as Python's lack of an identity function and the removal
of map and filter [1] from the core language.

There's no such thing as "very Pythonic." It's a binary and the worst aspect
of Pythonic is that people waste so much energy trying _not_ to be unPythonic
in order to meet the mores of the community.

[1]: Edit, Ok, I was confusing two experiences. It was reduce's removal that
broke my code using the map->filter->reduce idiom. Map broke by returning a
Map object rather than a list (but that was between Python and Python3). And
so far as I know, filter has not been changed so as to break code.

~~~
kevinmgranger
> the removal of map and filter from the core language

This never happened-- perhaps you're thinking of reduce? And reduce is still
in it, it just got moved to `functools`.

~~~
brudgers
Thanks, I stand corrected. My high level memory is that the
map->filter->reduce idiom was broken because <arbitrary>. Adding to my
confusion was my conflating that <arbitary> with my memory of map breaking
code by returning a map object rather than a list.

~~~
kevinmgranger
You're right in that map returned a list in python 2, but in python 3, it
returns an iterator. Anyone who relied on indexing or non-lazy behavior would
have their code break.

GVR doesn't like functional-style programming, so he much prefers the
iterative but compact comprehension alternatives.

It may just be the circles I hang out in, but most folks aren't map/filter
versus comprehension purists. I'll do comprehensions usually, but map/filter
when it helps make the line shorter or more readable.

~~~
brudgers
I don't have anything against list comprehensions. Sometimes they are a very
useful abstraction. Other times map->filter->reduce better expresses the
business logic of the program (e.g. when the analog is signal processing [1]
or Unix pipes).

My issue with 'Pythonic' is that it is used to express the idea that someone's
thoughts are or are not properly structured.

[1]: [https://mitpress.mit.edu/sicp/full-
text/sicp/book/node35.htm...](https://mitpress.mit.edu/sicp/full-
text/sicp/book/node35.html)

------
k_sze
Some of the examples in the book violate the Zen of Python. So I wouldn't even
call them pythonic or idiomatic in the first place.

E.g.:

`contains_zero` violates "There should be one-- and preferably only one
--obvious way to do it."

`should_raise_shields` violate "Explicit is better than implicit."

`linsolve` and `contains_duplicate` violate "Readability counts."

~~~
shangxiao
I was thinking exactly this as I was going through those examples.

------
llimllib

        def inverse_of(matrix):
            return adjugate_of(matrix) / determinant_of(matrix)
    

aiiieeee no, don't do that

~~~
jacob019
Why? Don't like self-documenting code?

~~~
knappa
It's numerically unstable and O(n^3) (at least, probably worse) when computing
via row reduction is O(n^2).

------
pwaivers
I like the rules that lays out in this article. However, I don't see his
argument against Writing Idiomatic Python. He cherry picked two methods that
(correctly) could have been written better. However, this does not mean that
all idioms are bad, and he doesn't provide any examples where using an idiom
is sometimes bad and sometimes good.

Additionally, it seems like the word idiom is just re-skinned as a "rule" in
this article.

~~~
dcdanko
He provides three examples where an idiom is sometimes bad and sometimes
good...

The point of the article is just the classic 'foolish consistency is the
hobgoblin...'

------
fhood
This is a little bit off topic, but one thing that I wish python used is
ruby's indicators for the purpose of a function. i.e. "?" for boolean returns,
and "!" for modifying functions in place.

In a language without explicit typing and return values having an instant
indicator of the functionality of a function can be a huge help in writing
readable code.

~~~
cup-of-tea
Is this enforced in Ruby or just a convention? It's a great convention and is
used in Clojure and older Lisps at least with the -p suffix (for predicate).
There's no reason you can't use the same convention in Python. I've often
wanted to write predicates with the -p suffix but stopped myself becasue I
don't think other Python people would understand it. Maybe I should just do it
anyway.

~~~
brudgers
{being pedantic} In Python, you can't use the '-p' suffix because Python will
treat it as subtraction.

{being unPythonic} If it makes you happy to use '_p', do it. Explain it in the
documentation if it might cause confusion. Be flexible enough that you'll
change before letting yourself be voted off the island because it's not that
important.

{rabble rousing} I think there is room in the world for an alt-python
movement...[https://github.com/altpython](https://github.com/altpython)

~~~
cup-of-tea
Yes, you'd have to use just p. The convention in Lisp is it's just p unless
the function name is multiple words anyway in which case the words are
separated by hyphens and the suffix is -p.

I suppose it is more Pythonic to name predicates like "is_full" or something
rather than "fullp". But whatever it is I've found I can't stand not using
some kind of naming convention for predicates.

~~~
brudgers
I think the problem is more that Python doesn't really have the idea of
predicates as an abstraction. It's not that it doesn't have predicates, it's
just that they are not called that and not seen as thing in their own right
and not something that programmers are encouraged to write.

~~~
cup-of-tea
Yeah, that's true. Maybe I'm being narrow minded and not "Pythonic" enough,
but I always end up writing predicates. I can see that non-trivial predicate
functions could be changed to non-trivial functions that return something
which can later be compared to get the boolean. But what about having a list
of of predicates and wanting to test them all at once. all(p(x) for p in
predicates) seems perfect for this...

------
crdoconnor
If you're aiming for idiomatic on 6 you don't do:

    
    
        def contains_duplicate(array):
            return sum([sum([1 if a_i-rot_ij else 0 
                        for a_i, rot_ij in zip(array, rot_i)]) 
                        for rot_i in [array[i:] + array[:i] 
                        for i in range(1, len(array))]]
                       ) != len(array) * (len(array) - 1)
    

OR:

    
    
        def contains_duplicate(array):
            for i in range(len(array)):
                for j in range(i):
                    if array[i] == array[j]:
                        return True
            return False
    

You do:

    
    
        def contains_duplicate(array)
            return len(set(array)) < len(array)
    

Or, if you can't hash the items:

    
    
        def contains_duplicate(array):
            any(len([y for y in items if x == y]) > 1 for x in items)
    
    

FWIW, I don't think I've ever seen an example where the cleanest (list |
generator) comprehension is less clear than the cleanest equivalent for loop.
I don't believe one exists.

So I think the examples are contrived and the 'rule' is bullshit.

~~~
LyndsySimon
> FWIW, I don't think I've ever seen an example where the cleanest (list |
> generator) comprehension is less clear than the cleanest equivalent for
> loop. I don't believe one exists.

Interesting. I'm not sure if I agree or disagree.

I think the primary benefit of a comprehension is that it can be considered as
a "unit", and you don't have to think about the iteration as fully to
understand what's happening. As soon as "what's happening" gets more than
minimally complex, that advantage is lost. Generator comprehensions should be
much more clear and concise than their equivalent implementations using
generator functions, though.

If I remember this evening, I'm going to go through my code library and see if
I can find something that will satisfy your challenge :)

~~~
crdoconnor
I consider it an instance of
[https://en.wikipedia.org/wiki/Rule_of_least_power](https://en.wikipedia.org/wiki/Rule_of_least_power)

List comprehensions are less expressive than for loops hence there is less to
reason about.

Hence, if what you want is achievable with a list comprehension, its intent
will naturally be clearer than the for loop equivalent.

...provided you don't make it intentionally obtuse like the OP did.

------
freyr
Obviously, any falsey value triggers the "idiomatic" contains_zero function:

    
    
        [1, False, 3]
        [1, '', 3]
        [1, None, 3]
        [1, [], 3]
    

This seems like it could only make bugs harder to track down, yet this is
often touted as the Pythonic way. What's the benefit of this approach?

~~~
empthought
The original implementation of `contains_zero` in the blog post is not
idiomatic. (It's also incorrect, as you point out.)

The idiomatic implementation of `contains_zero` is the second one given, `0 in
container`.

Just because someone claims something is idiomatic in a book doesn't mean they
are correct about it being idiomatic.

~~~
freyr
You're right. PEP8 says:

> _beware of writing if x when you really mean if x is not None -- e.g. when
> testing whether a variable or argument that defaults to None was set to some
> other value. The other value might have a type (such as a container) that
> could be false in a boolean context!_

This should apply to the contains_zero function as well. It seems a little
murky when PEP8 recommends checking whether a sequence type is empty,

    
    
        Yes: if not seq:
             if seq:
    
        No: if len(seq):
            if not len(seq):
    

if seq were accidentally set to False or 0. But I guess that would be revealed
as soon as you start treating seq like a sequence.

------
falcor84
I'd argue that actually all of the suggestions in this post are more
idiomatically pythonic than the original ones. So the trick is not to avoid
idioms but just to write clearer code :)

~~~
agumonkey
it's also a quality that takes times, you have to weight a lot of things
together. shorter names sometimes, longer names sometimes... Something you
appreciate with times, no matter the language.

------
divs1210
Let me unleash a monster that I created:

[https://stackoverflow.com/a/44729275](https://stackoverflow.com/a/44729275)

~~~
masklinn
It seems unnecessary:

    
    
        LET(('a', 1,
               'b', 2),
              lambda o: [o.a, o.b])
    

alternative:

    
    
        lambda a=1, b=2: [a, b]
    

Thing _do_ get worse when you need dependencies:

    
    
        LET(('a', 1,
               'b', lambda o: o.a + 1),
              lambda o: o.b)
    

alternative:

    
    
        (lambda a=1: lambda b=a+1: b)()

~~~
divs1210
I would argue my version scales more gracefully.

How would you write this, for example?

    
    
        LET(('a', 2,
             'b', lambda o: o.a + 1),
             'c', lambda o: [i * i for i in range(o.b)]),
            lambda o: o.c)
    
        => [0 1 4]

------
ralmidani
While I adore Python generally and use it all the time (including for two
startups), here are a few things that I find unintuitive:

1\. List comprehensions are hard to read and understand, especially for
beginners. I tutor online in Python, and when I show a student a list
comprehension, it is one of the most difficult parts of the language to
explain. Do you read left to right, or inside to outside? Postfix if at the
end makes things even more confusing.

There are better alternatives in languages like Ruby and JavaScript. Which
brings us to the next point:

2\. No native map, filter, reduce, etc. methods. It is far easier to explain
that you are just calling another method on a list/array than it is to get a
beginner to trust the hocus-pocus apparent in Python's list comprehensions.

Edit: Python has built-in functions for map and filter. See correction by
ubercore below, and my response.

3\. The way a class is accessed from an instance in Python is not nearly as
elegant as Ruby's. Do I use "type(obj)" or make a direct call with
"obj.__class__"? Neither is as clean or intuitive as "obj.class".

4\. Speaking of classes, why are built-in classes like "str", "int", "float",
etc. lowercase while in everything else they are conventionally
CapitalizedClasses?

5\. Remembering that "sorted(my-list)" returns a new sorted list while "my-
list.sort()" does the sort in-place requires experience, which someone taking
an Intro to Programming course will probably not accumulate over 8 or even 14
weeks.

Note: I flirted with Ruby and Rails, but never built anything real with them.
I vastly prefer Python to Ruby (and Django to Rails), but think Ruby got a lot
of things right.

~~~
ubercore
No native map or filter? Unless I don't understand what you mean by native,
don't
[https://docs.python.org/3/library/functions.html#map](https://docs.python.org/3/library/functions.html#map)
and
[https://docs.python.org/3/library/functions.html#filter](https://docs.python.org/3/library/functions.html#filter)
and
[https://docs.python.org/3/library/functools.html#functools.r...](https://docs.python.org/3/library/functools.html#functools.reduce)
count?

~~~
ralmidani
Thank you for correcting me. I edited my post above to reflect that.

But having to provide a lambda for the simple case makes them less useful.
Also, nested function calls are less readable than chaining method calls on
the same object.

------
dec0dedab0de
_Rule 6. Just because you can do anything with list comprehensions, doesn’t
mean you should_

I have been very guilty of this one, it's a running joke at work gatherings.
In the flow I can quickly spit out comprehensions that are obtuse to everyone
else. My general rule of thumb is that if I ever have to troubleshoot it I
break it out into a for loop

------
Veedrac
The Zing of Python

(Or the well intentioned but ultimately misguided attempt to enlighten the
masses.)

Pretty is better than boring.

Comprehensions are better than loops.

Short is better than long.

Clever is better than tangled.

Nested is better with line breaks.

Compact is better than meandering.

Appearances matter.

The point of a Python program is to teach people to write better Python
programs.

Though if they're Java programmers just try to make them jealous.

Exceptions should never be explicitly checked.

Except StopIteration.

If you don't know how to make it Pythonic, refuse to take the obvious
approach.

There should be one-- and preferably only one --Pythonic way to do it.

Although that way may not be obvious at first unless you're Dutch.

Now is better than never.

But it's fine to take ramp it up over time as long as you get there.

If it doesn't make a good blog post, perhaps it doesn't make good code.

If it doesn't make good code, it definitely doesn't make a good blog post.

List comprehensions are one honking great idea -- let's do more of those!

------
extr
To use a math analogy, I think of idioms as solutions to generalized coding
problems. If you recognize your situation as a special case of the general
form you can save yourself some time thinking through the implementation and
make it easier to understand. Great. But also like math, you can get yourself
into trouble trying to frame something in terms of a general case that doesn't
actually fit, suddenly the plumbing is more complex than the original problem.
The same thing plays itself out at a higher level with design patterns. The
(obvious) point is not to force a solution into an abstract pattern at the
expense of clarity, because that's the reason we like abstract patterns in the
first place.

------
graton
On a tangent. I'm still kind of pissed off at Jeff Knupp. I bought the e-book
which claims it is updated frequently and you get free updates for life. I
have emailed him over three times asking about updates and so far have
received zero replies :(

------
Walkman

        To learn to write good code you have to write a shit-metric-ton of bad code.
    

The problem with this is how do you know if your code is good or bad? A lot of
programmers never learn. They only way is to learn from others and write. You
won't necessarily will be a better programmer if you just write programs by
yourself.

------
danso
I teach Python to beginners, some of whom have had a CS class (e.g. in Java).
I try to avoid idioms early on, such as list comprehensions and context
managers, and slowly introduce them after students are confident with loops
and such. It does feel painful and backwards, though, which is ironic for me
to feel when I consider how confused/annoyed I was with Python initally when
coming from Ruby.

------
zwieback
Agree with the critique but it's not quite fair. The original book is about
Python, maybe sometimes misguided. The critique could be leveled at any
language, it basically says: "be an experienced programmer with good taste."

------
Walkman

        def are_all_numbers(everything):
            return all(is_float(e) for e in everything)

