
PEP 572: Python assignment expressions has been accepted - est
https://groups.google.com/d/msg/dev-python/egS7A22oJgE/Ar6TfdrQCAAJ
======
sametmax
I will be happy to be able to do:

    
    
        while (bytes := io.get(x)): 
    

and:

    
    
        [bar(x) for z in stuff if (x := foo(z))] 
    

Every time Python adds an expression counterpart to an existing statement
(lambdas, intensions, ternary...) there is a (legit) fear it will be abused.

But experience tells that the slow and gradual pace of the language evolution
combined with the readability culture of the community don't lead that way.

While we will see code review breaking materials in the wild, I believe that
the syntax will mostly be used sparingly, as other features, when the specific
needs arise for it.

After all, it's been, as usual, designed with this in mind: "=" and ":=" are
mutually exclusive. You don't use them in the same context.

The grammar makes sure of it most of the time, and for the rare ambiguities
like:

    
    
        a = b
    

vs

    
    
        (a := b)
    

The parenthesis will discourage pointless usage.

My bet is that we will see essentially rare but useful and expressive use
cases in productions, which is exactly the goal.

Given the month of debates around this, I think it's a fine compromise.

Like many, I would have preferred the use of the "as" keyword instead of a new
operator, since it's already used to bind things to names in imports, context
managers and try/except.

However, the new syntax has 2 advantages: it reads the same way than the
original operator, and it supports type hints out of the box.

~~~
gshulegaard
I agree that I would have preferred "as"...but that said I am struggling to
think of a reason this is needed.

    
    
        while (bytes := io.get(x)):
    

Would currently be written:

    
    
        bytes = io.get(x)
        while bytes:
    

And likewise:

    
    
        [bar(x) for z in stuff if (x := foo(z))]
    

is equivalently:

    
    
        [bar(foo(z)) for z in stuff if foo(z)]
    

Perhaps this is just my personal opinion but I don't really think the ":=" (or
"as" for that matter) adds much in the way of clarity or functionality. I
guess at the end of the day I am neutral about this addition...but if there
isn't a clear upside I usually think it's better to have less rather than add
more.

~~~
Dunnorandom
The first example would actually be equivalent to something like

    
    
        while True:
            bytes = io.get(x)
            if not bytes:
                break
            ...
    

which I think is objectively less readable.

In the second example, you have an extra call to foo for every element of
stuff. If foo(z) is expensive, you'd probably want to write this as

    
    
        [bar(x) for x in map(foo, stuff) if x]
    

instead - which I personally don't mind, but it's arguably not as clear as
having the in-line assignment expression.

~~~
amorousf00p
Quibbles and bits. Python is the only language where I write logic and then
massage data structures and outputs + design 'cooler' ways to create these for
an extra hour -- a week after it is in production.

------
_Codemonkeyism
The PEP is here:
[https://www.python.org/dev/peps/pep-0572/](https://www.python.org/dev/peps/pep-0572/)

"This is a proposal for creating a way to assign to variables within an
expression using the notation NAME := expr."

~~~
mci
> x = y = z = 0 # Equivalent: (x := (y := (z := 0)))

This comment is false. It should say

    
    
      x = y = z = 0  # Equivalent: (z := (y := (x := 0)))
    

"...assigns the single resulting object to each of the target lists, from left
to right."
[https://docs.python.org/3/reference/simple_stmts.html#assign...](https://docs.python.org/3/reference/simple_stmts.html#assignment-
statements)

Here is a demonstration of the difference:

    
    
      >>> class Node: pass
      ...
      >>> node = blue_node = Node()
      >>> red_node = Node()
      >>> node = node.next = red_node
      >>> blue_node.next is red_node
      Traceback (most recent call last):
        File "<stdin>", line 1, in <module>
      AttributeError: Node instance has no attribute 'next'

~~~
jessaustin
I don't see how that demonstrates anything about the order of the assignment.
It seems to show that node and blue_node are no longer the same object? (I'm
not sure I understand how that came to be either, but perhaps it's due to the
PEP...) I'm not saying your general point is wrong, just that the REPL listing
doesn't demonstrate your general point.

~~~
jwilk
"node" is initially blue, and then you do:

    
    
      node = node.next = red_node
    

If assignments were right-to-left, the .next attribute would be set on the
blue node.

If they were left-to-right, it would be added to the red node.

The AttributeError exception shows it is the latter order.

~~~
fractalb
It doesn't say anything about the order of assignment. It just shows that
`node.next` wasn't evaluated to anything. `node.next` needs to be evaluated
before doing any assignments

~~~
jjnoakes
Sure it does. Write out the example with both orders of assignments (convert
it to single assignments only) and run it both ways.

------
KirinDave
I hope I'm not the only person who reads this and really, really dislikes
this.

If Python's gonna have breaking syntax, why not work on bringing it more in
line with other modern languages that don't require special breakout syntax
for expressions and rely more on functional features?

Are we still maintaining that lambdas are hard but suggesting expression-
scoped variables are easy?

~~~
viraptor
The difference here is that this pattern of get-value-check-it is in pretty
much every program longer than a few lines. And possibly in every one using
regexes. Missing lambda opportunities are not nearly as easy to point out.

~~~
ehsankia
Making your whole language and code more complicated just to save one line of
code? No thanks. Python to me has always been about clarity and simplicity. If
i wanted to write cipher unreadable code, I'd just use C++

------
amelius
If they add anything to Python, it should be the ability to do functional-
style programming without hassle. Right now it's almost impossible to compose
functions in an inline style such as used in functional programming languages.
Yes, there's lambda, but it doesn't extend into multiple lines, and applying a
function to a bunch of lambdas directly leads to line-width overflow. Even
JavaScript has better support for functional-style programming. Perhaps Guido
should spend a year writing Haskell :)

~~~
sametmax
Haskell has always officially inspired Python tooling. But Python does things
in it's own way, and they are not random, they are the result of an opinion.

First class citizen functions, short lambdas, comprehension lists, generators,
map(), filter(), itertools, operator and functools are quite a rich toolbox
already. But you won't have more. It's a choice.

The idea is to have enough to be productive, and not enough to be dogmatic.
The experience of Guido, and it's one that I share, is that too much
functional tooling drives a style that favors expressive writing at the
expense of ease of reading.

It's not by chance that LISP and Haskell are considered hard languages to get
into, while Python is considered easy to start with.

It has a cost, since no language is perfect, but that's the path this language
follows and requesting a snake to fly will only bring you disappointments.

Python tries to strike the balance between the importance of a rich
expressiveness and the non negotiable necessity of keeping the code readable:
you read a line much more often that you write it, after all. It's a key
philosophy of the language. It shaped and will shape numerous decisions around
it.

This PEP is a perfect example : it tooks years for the concept to be
integrated in Python, and the last debate about this concrete implementation
took months. The result is a carefully crafted feature with a lot of details
to discourage abuse and remove the needs for pondering when to use it or not.

~~~
hellofunk
> The experience of Guido, and it's one that I share, is that too much
> functional tooling drives a style that favors expressive writing at the
> expense of ease of reading.

This is absolutely dead on accurate. As a Clojure developer, using one of the
most expressive -- dare I say, artistic -- programming languages _ever_
created, I can say that I am totally in the zone writing code which is elegant
and terse and really packs a punch, does clever things.... and then just a few
days later, it is very hard for my own brain to parse my own code and figure
out what it does.

For each line of code you write once, it will be read dozens of times by you
or others. Code is for reading. Languages that get this right make things a
lot easier for everyone.

~~~
roman_g
What other languages do you think "made it right" ?

------
carapace
I'm a huge Python fanboy and I've been so happy to be able to use it
professionally for the last decade or so, but I think the BDFL may have lost
his touch. Or he just doesn't care anymore. If I hadn't already decided to
stick to Python 2 (likely in the form of the Tauthon project, but there are
lots of Python 2 runtimes out there. Lots. (Everybody always forgets about
Stackless, for example)), I say, if I hadn't already decided to stick to
Python 2, this would be the camel-back-breaking straw. We're trucking in
footguns from C now?

These examples in the PEP, they all seem bad to me, written to be too clever
by someone who doesn't think well.

E.g.:

    
    
        filtered_data = [y for x in data if (y := f(x)) is not None]
    

How about:

    
    
        filtered_data = [y for y in (f(x) for x in data) if y is not None]
    

Or just?

    
    
        filtered_data = filter(None, map(f, data))
    

(If f() can return non-None "Falsey" values then this would require a
predicate function. But maybe f() is badly designed?)

Or this:

    
    
        if any(len(longline := line) >= 100 for line in lines):
            print("Extremely long line:", longline)
    

What's wrong with the "old-fashioned" way?

    
    
        for line in lines:
            if len(line) >= 100:
                print("Extremely long line:", line)
                break
    

Of course, in the bad old days when loop variables leaked I _think_ you could
just write:

    
    
        if any(len(line) >= 100 for line in lines):
            print("Extremely long line:", line)
    

But I'm not sure, and in any event leaking loop vars was fixed at some point.
(Good. It was useful but bug-prone. Just like this PEP!)

To me this is also a mess:

    
    
        results = [(x, y, x/y) for x in input_data if (y := f(x)) > 0]
    

It should be:

    
    
        results = [(x, y, x / y) for x, y in zip(input_data, map(f, input_data)) if y > 0]
    

Maybe all the examples are just poorly chosen, but I don't believe that. I
think this PEP is just a bad idea. Badly bad.

~~~
UncleEntity
> But I'm not sure, and in any event leaking loop vars was fixed at some
> point. (Good. It was useful but bug-prone. Just like this PEP!)

So, basically, they have a proposal for an expression that explicitly says "I
want to leak this variable into the enclosing scope" and it is less useful
than the implicit old way?

Python's variable scoping rules are a mess anyway -- some are function level,
some are block level and I usually can't tell which is which until the
compiler complains...

~~~
camel_Snake
Local -> Enclosing -> Global -> Builtin

I use Python the most so maybe I'm just used to it but I always found scoping
pretty straightforward.

Do you have an example that you found surprising?

~~~
UncleEntity
Mostly I can't remember if a variable escapes from an if-else block, context
manager, &etc until I start playing around with the code.

In C it's 100% clear, variables only live inside their respective blocks so if
you want to mess with the value in an if-else you have to declare it outside
of the block.

------
korijn
It feels like we are repeating a mistake the PHP community made years ago. PHP
supports assignment in expressions, and the community seems to avoid its use.
There are plenty of examples of stackoverflow questions and answers where the
problem is caused by coders overlooking an assignment in an expression. You
just don't expect assignment to occur in an expression, and its visually
similar enough to gloss over it.

~~~
sametmax
In PHP, the "if ($a = 1)" and "if ($a == 1)" look a lot like each others.

In the Python version, the new "(a := 1)" will stand out compare to the
canonical "a == 1" as the ':' and '()' are unexpected here, and scream that
it's not regular if test.

~~~
korijn
I'm curious to see how it will pan out in practise, but I'm not getting my
hopes up.

I made a flake8 plugin to forbid assignment expressions:

[https://github.com/Korijn/flake8-assignexp](https://github.com/Korijn/flake8-assignexp)

Will be released once python 3.8 is out and I can test it.

~~~
geoelectric
That seems unnecessarily reactionary, but ok. I agree with the post you're
responding to that having a completely separate token for inline assignment
fixes the majority of issues that have historically come up with it (which
mostly come down to forgetting an = in the ==). Might be worth feeling out the
trajectory of the feature before proactively banning it from codebases you
control.

~~~
korijn
You're right. Like I said, I'm curious to see how it will pan out, but I'm not
getting my hopes up. I'm just not a fan of this new syntax. :) IMHO null-safe
navigation operators like ?. and ?[] are much more important to have.

------
passive
This is nice.

I very frequently use something like the following:

    
    
        [node.find(SOME_XPATH).get("value") for node in tree if node.find(SOME_XPATH) is not None]
    

Which I can soon rewrite as:

    
    
        [found_node.get("value") for node in tree if (found_node := node.find(SOME_XPATH)) is not None]
    

There's a certain amount of complexity introduced, but I think removing the
duplication makes up for it. This is one of the few remaining cases in Python
where I feel like there's not a simple way to avoid repeating myself.

~~~
stared
Still fail to see why instead of that there isn't more functional list
comprehension, with maps and filters. That way we wouldn't get such problems
in the first place.

For more advanced list compherensions, even JavaScript (ES6+) is more
readable.

~~~
joeframbach
[found_node.get("value") for node in tree if (found_node :=
node.find(SOME_XPATH)) is not None]

tree.map(node => node.find(SOME_XPATH)).filter(Boolean).map(node =>
node.get("value"))

I can deal with either language at this level of complexity. Anything more
complicated needs more LoC in either language.

~~~
hellofunk
What is "node => ", is that valid Python? I'm new to the language and have
never seen the => syntax before. The docs for 'map' don't show it either.

~~~
vthriller
It's not Python, the second is actually a counterexample in JavaScript, where
`node =>` is similar to Python's `lambda node:`

------
bluecalm
This is badly needed equivalent of where keyword in Haskell very useful (among
other things) in list comprehensions to avoid duplicating function calls (now
you assign the result of the call and use it again in the beginning part of
the comprehension).

That being said it's another example of how after many years it turns out that
C got it right all along. Assignment being an expression (not a statement)
makes many useful idioms possible. Sure, there was a problem with typos
concerning '=' and '==' being similar but this was solved by convention of
requiring parenthesis around assignment if its value is used as boolean. If
you turn on warning in modern C compiler you will never make this error again.
Interestingly it's also what Python has chosen even though they went with :=
operator.

Personally I prefer having one assignment operator and assignment being an
expression. If it's := or = really doesn't matter in my view.

------
gbfowler
It is not syntactic sugar, "x := 10" is an assignment _expression_ in contrast
with "x = 10", which is a _statement_.

Hence the former can be used in contexts like "if x := 10: pass", which is the
whole point of the PEP.

~~~
Aardwolf
Why is it not syntactic sugar? It looks like convenience. You could argue
everything above machine language is syntactic sugar.

~~~
akvadrako
Not so. Many things change the semantics, otherwise high level languages would
just be glorified macros.

For example, _import mod_ is NOT defined as

    
    
      mod = eval(open("mod.py").read())
    

but involves abstract _load module_ operation, which is dependant on the
environment.

That's why _:=_ is just syntactic sugar; there are no new semantics.

~~~
junke
Honestly curious, how do you expand a "x := y" expression into an expression
that returns y and affects x as a side-effect?

~~~
akvadrako
I'm not sure exactly, but my point is that a program transformation can do it.
However, no transformation can change an _import_ to other operations while
perserving semantics.

Just for fun, this seems to work:

    
    
      (locals().pop('x', None), locals().setdefault('x', y))[1]

~~~
yorwba
That only works at top-level scope, where locals() == globals(). Inside a
function, changing the dict returned by locals() doesn't actually change the
variable.

------
Animats
Must...have...functional...features.

Python's indentation system was so nice for imperative programs. But it gets
in the way when functional features are retrofitted. The "everything must be
one big expression" style is a total mismatch to Python. The beauty is gone.

~~~
pgorczak
Yet using functional staples like map and reduce is discouraged by GvR
himself. I guess it's alright as long as the language can adapt to your needs
(even though this might contradict the "one obvious way" idea).

------
logicallee
This is unpythonic, breaking about half of these design rules:

[https://en.m.wikipedia.org/wiki/Zen_of_Python](https://en.m.wikipedia.org/wiki/Zen_of_Python)

For anyone who hasn't read the original PEP link, what do you suppose this
does? Guess its meaning:

1.

    
    
        if (row := cursor.fetchone()) is None:
              raise NotFound
          return row
    

2.

Next guess the meaning of this - what does it look like it does?

    
    
      row = cursor.fetchone()
      if row is None:
          raise NotFound
      return row
    
    
    

Answers:

The first does the second. The second doesn't need explanation. I literally
don't need to tell you what it does.

(The two examples are from the link.)

Now you might think that this means I'm against this PEP.

But actually [EDIT: because I know there is a VERY high bar to being accepted
into Python] to me it means this is going to save an _incredible_ amount of
time - it must be very good indeed to be accepted.

So if they decided to adopt it - sure, I'll use it. And I bet it's great in
practice.

It's also super explicit. If you don't know what := does you will go look it
up.

If Python is still driven by Python design philosophy and all that discussion,
then this will be very helpful. It surely had a very high threshold to meet.

~~~
PurpleRamen
The problem were always those people who will misuse something. The Zen was
never meant as a guide, but as a guardian. Python in the early days was good
because there simply was no way to write ugly code. But with all the abilitys
modern python has, this wall has fallen, and now ugly python has become
normal. And with this it will sometimes even become horrible.

~~~
logicallee
I thought the Zen was some of the design philosophy that went into the
language itself. (The Wikipedia entry calls it "a collection of 20 software
principles that influences the design of Python Programming Language".)

So at least according to Wikipedia it is a guide. Of course, whether it's
applied in practice is a different matter.

~~~
PurpleRamen
At the time it was written, python was already around a decade old. There was
some discussion about the "true" way of python and some demands to "fix"
python, yada yada. Someone demanded some short description of the python
philosophy to counter those discussion, and the Zen was made up for this.

It's a condensed definition of the philosophy and goals which went into python
at that point. A guide to define the borders of the road to good python, but
also a guard against those who wanna change the direction of this road.

~~~
logicallee
Interesting history, thanks. If you can find any sort of cite (maybe a short
interview or forum link) I bet you could add this context to the Wikipedia
article in a few minutes, it's missing it now.

------
Myrmornis
From the PEP

    
    
      # Share a subexpression between a comprehension filter clause and its output
      filtered_data = [y for x in data if (y := f(x)) is not None]
    

What about

    
    
      filtered_data = [(y := f(x)) for x in data if y is not None]
    

will that work also?

Attempted guess at answer: no, because the "gathering" part of the
comprehension is downstream of the "sequence construction" part of the
comprehension in the language implementation. But if that's so, I'm a bit
concerned that this will be confusing to users.

------
jkabrg
It would be nice if in Python you could define new operators instead of
overloading existing ones. It would make matrix multiplication look nicer.

I'm thinking it could look like this:

    
    
      import numpy as np
      
      def M1 %*% M2 with same precedence as *:
        return M1.matmul(M2)
    
      foo_matrix = np.matrix([[1,1],[1,1]])
      bar_matrix = np.matrix([[2,2],[2,2]])
      print(foo_matrix %*% bar_matrix)
    

Also, it would be nice to have a pipe operator `%>%` such that

    
    
      foo %>% f()
    

is equivalent to

    
    
      f(foo)
    

The alternative is to make f a method of foo, and then you can write

    
    
      foo.f()
    

But what happens if I don't want f to be a method? I just want the _style_ of
writing the f after the foo, but I don't want the baggage of OOP. Is that
anti-Pythonic?

~~~
ben509
I think the problem with custom operators is they have custom associativity
and precedence. Not saying this is a deal breaker, just the biggest issue
you'll run into.

In toy examples, you can see all that because they'll show the declarations.

Here's a case[1] trying to explain monads by showing a simpler example:

    
    
        parseP5_take2 s =
         matchHeader (L8.pack "P5") s       >>?
          \s -> skipSpace ((), s)           >>?
          (getNat . snd)                    >>?
          skipSpace                         >>?
          \(width, s) ->   getNat s         >>?
          skipSpace                         >>?
          \(height, s) ->  getNat s         >>?
          \(maxGrey, s) -> getBytes 1 s     >>?
          (getBytes (width * height) . snd) >>?
          \(bitmap, s) -> Just (Greymap width height maxGrey bitmap, s)
    

Is all that left or right associative?

Generally, to read an expression when you have custom operators, you have to
dig up the associativity and precedence. That's why I see many newer libraries
avoiding custom operators, [2] vs [3].

I think you can do custom operators in a language, you just need to require
that the associativity and precedence be declared in import. I'd also have
named levels of associativity, so maybe:

    
    
        import MyModule.Foo.Bar (op(!$!, additive, right), ...)
    

Now, that's some boilerplate in your imports, but for readability, it'd be
huge. And automated tools can tweak stuff like this.

[1]
[http://book.realworldhaskell.org/read/monads.html](http://book.realworldhaskell.org/read/monads.html)

[2]
[http://hackage.haskell.org/package/pretty-1.1.3.6/docs/Text-...](http://hackage.haskell.org/package/pretty-1.1.3.6/docs/Text-
PrettyPrint.html#g:6)

[3]
[https://hackage.haskell.org/package/prettyprinter-1.2.1/docs...](https://hackage.haskell.org/package/prettyprinter-1.2.1/docs/Data-
Text-Prettyprint-Doc.html#g:17)

~~~
hodgesrm
On my first C++ project we enthusiastically implemented customized versions of
operators like '+' on new types. It turned out to be really confusing for
exactly the reasons you mention.

Operator overloading in general only seems to be practical for mathematical
types like sets or sequences where the rules are well-defined thanks to
generations of people thinking about them. Yet, even the set case works poorly
for C++ because in addition to associativity new operators also inherit the
built-in operator precedence rules. For example should * take precedence over
+ in set operations? (Assuming you implement * for cartesian product and + for
union.)

Maybe C++ has changed since I used it but this sort of thing really gets in
the way of writing correct code.

~~~
ktpsns
From my feeling, C++ makes it verbose to overload operators in every possible
context. Thinking of Foo::operator()+, there are all these border cases where
custom overload functions have to be provided (such as "(int) + (Foo)", "(Foo)
+ (int)", "... += Foo", etc.). I assume that in other languages it is probably
simpler to fully implement own operators.

~~~
hodgesrm
Exactly. This part of C++ always felt experimental to me--something that was
plausible to try but ended up digging a hole with a lot more complexity than
the language designers perhaps intended.

------
Derbasti
Does this mean that I can write

    
    
        with file := open('filename'):
            data = file.read()
    

instead of

    
    
        with open('filename') as file:
            data = file.read()
    

I don't know how I feel about this.

~~~
glifchits
Interesting! I figured the `as` in a `with` statement was handled uniquely,
but I learned something new about Python today:

    
    
       x = open('filename')
       x.closed  # False
       with x:
         print(x.readline())
       x.closed  # True
    

I think you're right. I prefer the `as` variant for readability.

~~~
zb
It's the same for classes that define it like:

    
    
        def __enter__(self):
            return self
    

which is a common pattern (used by open()), but there's no requirement that
__enter__() return the same object.

In cases where __enter__() does something different, the assignment expression
and the 'as' variable would have different values (the object of the 'with'
statement, x, and the result of calling x.__enter__(), respectively).

------
blindseer
My biggest problem with this change (and dataclasses) is that it is not
backward compatible (I know that a backport of dataclass is available, but it
is only for Python 3.6). Can people tell me how they deal with this? Are you
using features that are only available in the version of Python all your
colleagues / users have access to? Are you using the latest version of Python
and hoping your colleagues and users can upgrade to that version of Python?

One main reason in my opinion as to why Python 2.7 stuck around for so long
was that everyone agreed that it was the last stable version of Python 2, and
the devs could make Python 3 changes without worrying significantly about
backward compatibility. I've been using Python3 since Python 3.3 but have had
to write code that was Python 2 compatible for large code bases for about 5
years because I knew colleagues / users of my packages would not upgrade. This
meant using a select subset of features in Python 3 that had been backported
to a version of Python 2.7 that I knew I could get my colleagues / users to
upgrade to. It has been great to watch the language evolve and Python 3 really
gets a lot of things right, but adding breaking syntax features every minor
release is extremely annoying. I have to have a mental checklist of all the
different possible subsets of features I have access to given a minimum
version of Python I want to support. I've spent the last couple of years in my
professional career converting large Python 2 only code bases to Python 3.3+.
But if someone wants to use async, I need to bump the minor version. If
someone wants to use type hinting I have to bump the minor version. If someone
wants to use f-strings I have to bump the minor version. If someone wants to
use data classes I have to bump the minor version. It's nuts (to me anyway)!

This sounds rant-y but I genuinely want to know what other people are doing to
mitigate this. I love Python! I have spent years advocating its merits. But
thinking about large codebases in Python just worries me and the future
doesn't look promising here. Are Python developers hoping that once 2020
arrives they'll have a stable Python release and work on Python 4. We'll have
the Python 2/3 split all over again!

My personal opinion is that Python needs an officially maintained translator
for the latest Python features back to older versions. My current favorite
package is py-backwards [0] but it is rather unmaintained at the moment.

[0] - [https://github.com/nvbn/py-backwards](https://github.com/nvbn/py-
backwards)

~~~
marcus_holmes
A bit off-topic, but I'm really curious. Why do people not upgrade Python and
stick with 2.7?

~~~
dwheeler
A lot of the problem is that originally the developers of Python made it
absurdly difficult to transition from Python 2 to Python 3, and thus the costs
far exceeded the benefits. It was historically difficult to write code that
ran in both Python 2 and Python 3. If you wanted to transition to Python 3,
you had to simultaneously transition every file in your program, every library
you depended on, and all libraries they depended on transitively, to Python 3.
The "2to3" program was supposed to automate, but this never worked reliably,
and _cannot_ work reliably - to do that reliably requires type information
that is typically unavailable in Python.

Things have gotten _much_ better, thankfully. Python 3 (and 2) have been
tweaked over the years to make it much easier to write code that will work on
both Python 2 and 3, and to make it easier to tweak existing code so that it
will work on both. As a result, it's possible to transition code a file at a
time, or even a portion at a time, instead of the impossible "all at once"
transition. Almost no one used Python 3 after it was released, or for a number
of years later. Now that the developers of Python have started to make
transition practical, people have started transitioning.

Still, it takes real effort to transition to Python 3, and many people have
"real work" to do instead of transitioning language versions without any real
benefit. "3 is larger than 2" is not a real benefit. A real benefit is
something like "this transition will radically increase performance" \- and no
one is claiming that Python 3 has a real-world advantage like that over Python
2. "Python 2 will eventually be unsupported" is a problem, but no one is
providing free money to do the transition, so for many people it's just a
fact.

Historically Python has been very good about supporting backwards
compatibility and smooth upgrades. I hope that the Python 2->3 transition was
an anomaly. You can make changes to languages, but you have to make it easy
for users to do the transition.

------
ckastner
Link to the email on the Python-Dev mailing list:
[https://mail.python.org/pipermail/python-
dev/2018-July/15423...](https://mail.python.org/pipermail/python-
dev/2018-July/154231.html)

------
esaym
With every PEP it seems Python gets closer to Perl...

~~~
vgy7ujm
It was done right the first time. Why not just use Perl.

~~~
esaym
I'm a full time "perl developer" actually :)

------
julienfr112
I miss that feature in multiple occasion list comprehension. I hacked through
that with for x in [expr] but it never fell right.

------
meken
Don't anaphoric macros [1] provide an alternative solution to this problem?
Though they seem cool I can see how introducing a magic variable can result in
unreadable code...

\- [1]
[https://en.wikipedia.org/wiki/Anaphoric_macro](https://en.wikipedia.org/wiki/Anaphoric_macro)

------
BerislavLopac
I would like to see adding a comprehension-like filtering clause to for-
statements:

    
    
        for n in range(100) if n%2:
            print(f'{n} is odd number')
    

Does anyone know if there is a PEP covering that?

~~~
eesmith
Historically there has been much resistance to proposals like this which only
save a line. The existing code is, after all:

    
    
        for n in range(100):
            if n%2:
                print(f'{n} is odd number')
    

You proposal also leads to a more ambiguous grammar because the following is
currently allowed:

    
    
        for n in range(100) if n%2 else range(n):
    

The ambiguity can be extended with multiple if's, compare:

    
    
        for x in range(10) if n%2 if n else range(n):
        for x in range(10) if n%2 if n else range(n) else n**2:
    

A work-around would be to raise something akin to the "SyntaxError: Generator
expression must be parenthesized if not sole argument" that occurs with
expressions like "f(b, a for a in range(3))", but that's a lot of work just to
save a newline, two indents, and ":", isn't it?

~~~
BerislavLopac
How is that ambiguity currently handled in comprehensions?

My point is that it would be nice to have a consistent syntax for all for-
loops, either being a part of a comprehension or standing on their own.

EDIT:

> You proposal also leads to a more ambiguous grammar because the following is
> currently allowed:
    
    
        for n in range(100) if n%2 else range(n):
    

Not really, I gives me "NameError: name 'n' is not defined". Unless it is an
'n' defined in the outer scope, of course.

~~~
eesmith
"How is that ambiguity currently handled in comprehensions?"

A bit poorly. Compare:

    
    
      >>> f(1, 2 for x in )
        File "<stdin>", line 1
          f(1, 2 for x in )
                          ^
      SyntaxError: invalid syntax
      >>> f(1, 2 for x in r)
        File "<stdin>", line 1
      SyntaxError: Generator expression must be parenthesized if not sole argument
    

See how the first one gives the location of the error while the second does
not? As I recall, this is because the first can be generated during parsing,
while the second is done after the AST is generated, when the position
information is no longer present.

That's why the following:

    
    
      >>> f(2 for x in X) + g(1, 2 for y in Y) + h(z**2 for z in Z)
        File "<stdin>", line 1
      SyntaxError: Generator expression must be parenthesized if not sole argument
    

doesn't tell you which generation expression has the problem.

Yes, I meant that if 'n' is defined in an outer scope. The expression I gave
is not a syntax error but a run-time error.

~~~
BerislavLopac
This does not answer my question, so I checked -- comprehensions simply do not
accept the `else` clause:

    
    
        >>> [a for a in range(10) if True else range(2)]
          File "<stdin>", line 1
            [a for a in range(10) if True else range(2)]
                                             ^
        SyntaxError: invalid syntax
    

And this is the argument why I can't have my wish, because the standard `for`
loops have always accepted `if else`, so it would be a backward incompatible
change.

That said, I have another idea: an update to the comprehension syntax which
would omitting duplication of variables, using a new "for in" construct. For
example, this line:

    
    
        (x for x in range(100) if x%2)
    

...could be written as:

    
    
        (x for in range(100) if x%2)
    

Just an idea... :D

------
p3llin0r3
That's nice.

Give me the `|>` operator plz

------
oooooof
What is it? The link points to a discussion more deep than I’m willing to
read.

~~~
est
It's a controversial PEP
[https://www.python.org/dev/peps/pep-0572/](https://www.python.org/dev/peps/pep-0572/)
which allows you to write Python like this:

    
    
        def foo():
            if n := randint(0, 3):
                return n ** 2
            return 1337
    
    
        [(x, y, x/y) for x in input_data if (y := f(x)) > 0]

~~~
stfwn
This immediately looks useful for things like:

    
    
        if foo := bar[baz]:
            bar[baz] += 1
            return foo
        else:
            bar[baz] = 1
            return 0
    

Where foo is a dict keeping track of multiple things, and a non-existing key
(baz) is never an error but rather the start of a new count. Faster and more
readable than

    
    
        if baz in list(bar.keys()):
        ....
    

Similar to Swift’s ‘if let’, it seems.

~~~
antoinealb
As pointed, you can use either a default dict or just simply, and [more
pythonic]([https://blogs.msdn.microsoft.com/pythonengineering/2016/06/2...](https://blogs.msdn.microsoft.com/pythonengineering/2016/06/29/idiomatic-
python-eafp-versus-lbyl/)):

    
    
        try:
          bar[baz] += 1
        except KeyError:
          bar[baz] = 1
    

Also you can check if a key is in a dict simply by doing "if baz in bar" no
need for "list(bar.keys())", which will be slow (temp object + linear scan) vs
O(1) hashmap lookup.

~~~
stfwn
The error-catching method seemed too drastic to me before, but the article
explains the LBYL vs. EAFP arugument quite well. Thanks!

I should find a way to get more code reviews, I really enjoy learning these
small nuggets of info.

------
roel_v
Having just spend the last few weeks writing Python, this comment will come
off as bitter, but - really? Out of all the shitty syntax things, this sort of
thing is what they're willing to fix?

~~~
roel_v
OK you're all right, I should've included some examples, here's just some from
the last few days (I guess some of these are arguably not purely 'syntax', but
to me they mostly come down to that, and I guess most can be explained away
with 'different philosophy', and I'm sure someone will come out and say 'oh
but if 'only' you had just done this and this', but still...) :

\- Instantiating an object from a name in a string. Like instantiate a 'Foo'
when you have a string variable that contains 'Foo'. I can't remember the
syntax even though I looked it up two days ago, and I never will because it's
such a shit show. Not to use PHP here as and example of a great language, but
there at least the intuitive '$obj = new $var' works as you expect it. Or, in
C++ you have to do it manually, which is also fine - at least be consistent.

\- The weird sort-of typing of variables. Variables have types, but you can
assign a different value of a different type to them, and the actual types
usually doesn't matter except when it does. So you do print "Hey " \+ var but
now you need to know what type var is because you might need to str() it.

\- The whitespace-is-important-except-when-it-isn't. OK so braces are the
devil's work, but when it's inconvenient, we're not that strict on white space
(when initializing lists, when having expressions that span several lines,
...) so now everything can still look out of whack.

\- .iteritems(). Really?

\- super(ClassName, self).__init__(argument). Wut? Yes when I parse it token
by token I understand, but why? Maybe the other magic methods are in this
category too, but probably to a lesser degree.

\- (I had some other things here about primitive OO capabilities, shitty
package system/versioning, and some more, but those were all so far away from
'syntactic sugar' that they didn't fit this list no matter how hard I twisted
the argument)

Look, I do understand _why_ they are this way. For each of them, there is a
reason that any reasonable person would say 'yeah that makes sense' to,
possibly after some explanation of the history or context or whatever. But
then at least be honest and stop promoting the language as so 'intuitive' or
'beginner-friendly' or 'much more clean than other languages'. Sure, it's not
as bad as R, but it's still just like any other 20+ year old language in wide
spread use - crufty, idiosyncratic in many respects, and in general requiring
a bunch of frustrating head butting before you can be productive in it.

And to tie it to the OP - it seems this new syntax is promoted as being for
'beginners' or to make it 'easier to teach'. Well good luck with that, I say.

~~~
eesmith
"Instantiating an object from a name in a string. Like instantiate a 'Foo'
when you have a string variable that contains 'Foo'. I can't remember the
syntax even though I looked it up two days ago, .."

Python doesn't have specific syntax for that. It can be as simple as:

    
    
      obj = globals()["Foo"]()
    

That assumes "Foo" is in your global namespace. If you don't care about
security then you can do:

    
    
      >>> import math
      >>> s = "math.cos"
      >>> eval(s)(3)
      -0.98999249660044542
    

If you care about security then you might not want to allow arbitrary objects,
like "os.unlink" to be referenced. There are third-party packages which
provide different models of how to get objects, like Django's "import_string"
at
[https://docs.djangoproject.com/en/2.0/ref/utils/#django.util...](https://docs.djangoproject.com/en/2.0/ref/utils/#django.utils.module_loading.import_string)
.

"The weird sort-of typing of variables. Variables have types,"

Variables have only one type, "reference to Python Object". An expression like
'var = "Hey " \+ var' may change the type of the value that var references,
just like how 'var = var * 234.567' may change the type of the value that var
references from an integer/long to a float, or "var = var * "Hey"', if var ==
2, causes var to be a reference to the string "HeyHey".

".iteritems(). Really"

This was for a transition phase. It no longer exists in Python 3, where
items() returns an iterator instead of a list.

"super(ClassName, self).__init__(argument). Wut?"

In Python 3 this is: "super().__init__(argument)", as in:

    
    
      >>> class A:
      ...   def __init__(self, s):
      ...     print("A says", s)
      ...
      >>> class B(A):
      ...   def __init__(self, t):
      ...     super().__init__(t*2)
      ...
      >>> B("hello?")
      A says hello?hello?
      <__main__.B object at 0x10b201630>
    

"but it's still just like any other 20+ year old language in wide spread use -
crufty, idiosyncratic in many respects"

A reason for the oft-bemoaned backwards-incompatible changes to Python 3 was
to remove some of the crufty, idiosyncratic language features that you rightly
pointed out. You are still using Python 2.7, so cannot take advantage of those
changes.

~~~
roel_v
Ok the 'global' syntax thing makes a lot more sense than the one I was told
about (which involved the 'inspect' module iirc). The typing - yes I
understand all of that, my point still stands that it's all 'you don't need to
know all of this! Oh, at least, until you do'. Wrt python3 - fair enough,
things probably do get better. But that just reinforces my point about cruft
building up in any practical language, and Python being just as susceptible to
it as other languages.

