
Python: Even a Feature That You Do Not Use Can Bite You - ingve
https://blog.petrzemek.net/2019/02/22/even-feature-that-you-do-not-use-can-bite-you/
======
goodside
Understanding the basic syntax for assignment isn’t a high bar to clear. While
it’s maybe not ideal to use colons for both the “assignment-like” separator
between keys and values inside dictionaries and the annotation of types,
introducing any new syntax features into a language that’s been around for
decades is never without compromise. Colons are also used at the end of block
statements and in index ranges and nobody complains.

In this case you hit the most benign sort of change: Code that was previously
syntactically invalid is now syntactically valid but has silly semantics.
There’s never going to be a way to avoid this with full generality, and your
code _never_ worked — type annotations just delayed your runtime error by one
line.

When annotations were introduced in 3.4, there was no requirement they be used
to annotate types instead of something else, so any valid Python expression is
allowed syntactically. In 3.6 the use of annotations for anything other than
types was provisionally deprecated, and in 3.7 is fully deprecated. In 3.8,
non-type usage will be disallowed as a TypeError, and the only allowable non-
type annotations will be strings, (which will be interpreted as bare
identifiers for backward compatibility with pre-3.7 code that uses them as
forward references). At this point your code will revert to raising an
exception on exactly the same line it used to.

~~~
masklinn
The problem is not understanding, it's fat-fingering (especially when dict
literals and dict comps use `:`).

> type annotations just delayed your runtime error by one line.

Or it introduces a subtle bug because this was intended as a reassignment or
somesuch.

~~~
goodside
The real problem, as others point out, is not using a linter to catch errors
before runtime. Mypy flags this problem through static analysis.

You’re right I’m being uncharitable by saying “understanding”. I’m annoyed he
bothered writing an accusatory blog post about this aspect of the language
without mentioning any of the alternative syntax proposals helpfully discussed
in the PEP, or exploring why annotations work this way. The designers have
been very transparent and conservative in considering different ways of doing
it, and the OP doesn’t offer any ideas for improving the interpreter, or note
that the behavior he’s complaining about will be fixed in 3.8.

~~~
lukeschlather
> Mypy flags this problem through static analysis.

The problem is that there's not really such a thing as static analysis in
python. For example, if I wrap it in a function like this, mypy doesn't
complain at all.

    
    
        def blah():
            ages = {}
            ages['John']: 42
            print(ages['John'])
            return ages

~~~
goodside
Mypy intentionally does not check annotations inside untyped `def` statements.
This is a feature meant to make it easier to gradually introduce types in code
that wasn't intended to be typechecked. If you call mypy with the flag
`--check-untyped-defs` or just annotate the function with anything at all, you
will see mypy checks the inside of the function:

    
    
        from typing import Any
    
        def blah() -> Any:
            ...
    

I'm not sure what you mean by "there's not really such a thing as static
analysis" in Python -- that's exactly what Mypy does.

~~~
lukeschlather
Cool, I knew I had to be missing something. You're right of course.

------
larkeith
This is a weird article to me. There seems to be an implicit condemnation of
Python for this new feature resulting in non-error, unexpected behavior under
certain typoes, despite the fact that that is true for literally any typo that
results in valid code in any language (such as, e.g., = instead of ==). I
don't understand how it would be possible to add a new syntactic option
without being "bit".

Regarding type hints' ability to be arbitrary code, I suspect this is to allow
you to use nested type hints from the `typing` library, like `Tuple[int,
int]`, as well as more complex or user-defined types. Plus it just gives more
flexibility in general.

I'd be interested to hear the author's suggestions on alternative syntax or
ways to catch when the programmer intended something other than a type hint,
or even arguments that type hints are extraneous and should not have been
added - personally, I've found them quite useful recently, in combination with
pylint.

~~~
mjw1007
There were some people who thought that trying to bolt a type system on to
Python was a bad idea.

A common response to them was "you don't have to use it; you can ignore it and
you're no worse off than you were before". I think it's reasonable for him to
point out that this isn't quite true.

~~~
oconnor663
I dunno. This sort of problem would apply to any new feature with syntax. It
doesn't seem fair to blame the type annotations specifically.

------
masklinn
I mean technically it's a feature you're using, just unintentionally. Kinda
like setting a mutable object as a default argument.

But I'll admit I got bit by exactly this recently, wrote up an "assignment"
using a colon instead of an equal sign and it didn't do anything, but didn't
fail either.

Somewhat frustratingly, pycharm didn't warn about it either. I should probably
check their bug tracker to see if somebody already reported it.

~~~
benatkin
I don't think so, because 42 is not a type in Python (it could be one day,
though - in TypeScript, 42 can be used as a type, basically an enum with only
one value). I think (naively) that it should throw an error.

~~~
masklinn
Annotations were specifically designed as general-purpose, not specifically
_type_ annotations. So the ability to put arbitrary expressions in annotations
was very much intended.

In fact, the type _hints_ of the typing module are not actually types.

~~~
benatkin
Ah, I see, good to know.

What I wanted when I found the PEP a few days ago were docstrings for instance
variables, which isn't one of the things that this provides, so I stopped
reading. I ended up using properties but it's a bit verbose.

It's great that typings are in a module so it can be extended or replaced!

------
cv1010
I wonder why most people defend this feature. The problem is not the
annotation, you can do that in ML, too:

    
    
      Poly/ML 5.6 Release
      > 5 : int;
      val it = 5: int
    

But you cannot just use 42 where a type is expected:

    
    
      > 5 : 42;
      poly: : error: <identifier> expected but 42 was found
      Static Errors
      
    

It's pretty silly that Python did not develop an actual type system if
annotations like that are supported.

~~~
zimablue
In the void of enforcing a speficic one, you allow people flexibility to do
custom things.

For instance linters not understanding type specificatins beyond a certain
complexity but still using them in your code and some custom decorator to
check them at run/dev time

------
orf
non-clickbait title: Python: Mistyping a character in your code can bite you.

Any IDE or linter worth its salt would catch this for you.

~~~
lukeschlather
Can you recommend a particular linter? I use flake8, which does not warn for
this. I just tested pylint which also does not appear to warn for this.

~~~
heavenlyblue
PyCharm would highlight the expression after : as a type that was not imported
correctly (or an expression that is not a type).

I would assume mypy would cause an error.

~~~
lukeschlather
If I wrap it in a function, mypy doesn't throw an error. I'm not sure if mypy
has a function to lint such code without executing it.

------
kazinator
In any language that undergoes syntactic extension, there is new syntax that
is a syntax error in an old version of the language. Erroneous programs
written for the old version which land on that previously bad syntax will
undergo a change in behavior.

This is true even in languages that don't undergo syntactic extension. If you
call a nonexistent function today, your program bombs. Tomorrow, that function
could be added, so now something unexpected happens, perhaps.

Lisps are susceptible. Today (foo bar) is diagnosed; tomorrow it's new syntax
(not simply a new function), because _foo_ exists as a macro operator.

However, functions and macros, being named, can be namespaced to curtail such
issues. Random, _ad hoc_ read syntax extensions, not so much.

One approach is that files, or sections of files, can support annotations
which indicate what version of the syntax is required.

Under GCC, if you want only C89, so that any syntactic extensions from later
dialects are invalid syntax, you use '-ansi' on the command line to select
that dialect. For oinstance, support for // comments disappears, and so x //
_blah_ / y is x / y, rather than x followed by a comment-to-end-of-line.

------
benatkin
This is true. I don't think Python is to blame for it, though. Any language
has potential for making a mistake that's syntactically valid but doesn't have
the intended meaning. It's like when a former manager of mine typed "call me
at your continence" when he meant to say "call me at your convenience".

~~~
kazinator
Languages can help manage change, though, providing features such that
programmers working against older dialects of the language, who make mistakes,
get diagnostics according to the old version, rather than accidentally
stepping on new syntax.

------
msluyter
It seems odder to me that 42 is a valid type hint than that the LHS can be an
expression. But I share with most of the others here the opinion that this
really isn’t a problem with Python. Every language makes trade offs, and
python gives you incredible flexibility and expressiveness in exchange for a
lack of certain guardrails.

Once a coworker came to me with a program that was unexpectedly failing, and
it turned out that somewhere in the code there was a function that takes a
callable, but instead of

    
    
        something(iterable, func)
    

he had written

    
    
        something(iterable, func())
    

There were no IDE errors/warning (in fact PyCharm tends to induce this error
by auto inserting the parens) and the behavior that resulted made it tricky to
catch.

Nonetheless, personally I’d take that over most other options for run of the
mill progamming tasks.

~~~
heavenlyblue
PyCharm would highlight that error as a yellow warning if you correctly define
the types of the arguments for "something".

This would also not be accepted by mypy.

~~~
msluyter
Iirc this was pre type hints, so the example might be invalid now, but I’m
sure I could come up with other examples that demonstrate the same point.

------
steven2012
This isn't a good article to post. It's just one person complaining about a
change he didn't know about.

It's really just a syntax change. I probably wouldn't have chosen that
particular syntax using colons, or I would have forced another keyword like
"set" in front of it, but it is what it is. He created a bug and he likely
won't do it again, so problem solved.

~~~
daveFNbuck
I also didn't know about this change but now I do because this article was
posted.

------
cantcomplain
Ok I don't really see the issue. You could just as easily write x is 42 as x:
42 and you have the same issue. it's not really a language problem. If you see
keyerror, you should know that that value didn't properly get added to the
dictionary. When investigating why, you should see your error of using the
wrong operator. Language is functioning properly

------
xfoo__
So how do people defend this one:

    
    
      >>> [][0]
      Traceback (most recent call last):
        File "<stdin>", line 1, in <module>
      IndexError: list index out of range
      >>> 
      >>> [][0] : "bug"
      >>>

------
zimablue
Instead of the weirdly alarmist nature of the article, it would be a more fair
point to make that colons are now very overloaded in python syntax.

------
Ultimatt
The level of assumed ignorance of the reader in this article is a bit high
given the subject matter.

------
agumonkey
also, annotations end up accumulated into global __annotations__

PS: wait until he reads about := assignement syntax

------
philipov
Local variable type hints are a major reason I refuse to use python versions
before 3.6. What is this guy's problem?

------
Sir_Cmpwn
Weird article title. This is more neat than it is alarming.

