
Python Type Hints - lelf
https://kunigami.blog/2019/12/26/python-type-hints/
======
allovernow
Type hinting feels like a bandaid for a fundamental limitation of dynamic
languages. I've just gotten back to a complex, experimental codebase after
only a couple months of absence, and am refactoring it to accommodate for the
implementation of a number of previously unplanned features. Even with type
hinting and heuristic linting it's such a huge pain! After making a large
number of changes I end up having to rerun the code repeatedly to find and
squash bugs, and that says nothing of the code paths I don't end up taking. Is
there a better way to utilize the convenience of python for experimental code
without running into the scalability issues of large python codebases?

Contrast this to my experience with C# in visual studio (not visual studio
code, which is inferior). The state of flow I can get into when performing
large scale refactoring, with immediate, accurate feedback in the form of a
clickable, line by line list of what's broken is unmatched. I would love such
a pleasant experience in a python IDE but I'm not sure it's possible because
of the nature of duck typing. It's like a real time, automatically generated
checklist of exactly where to propagate changes, takes a massive load off my
working memory and provides an uninterrupted flow of dopamine. A true state of
zen if I've ever experienced one.

~~~
ken
Counterpoint: I feel like static typing is a bandaid for the fundamental
problem that the language isn't powerful enough to allow one's code to truly
be OAOO.

Back when I wrote C#, static typing was indeed helpful for making large-scale
changes and providing an automated check that this wasn't destroying
everything. But then, it's only in languages like C# that I have to make these
kinds of large-scale changes. There's no macros or syntactic abstraction. When
you want to construct the same category of operation in different contexts,
and functions and classes don't operate on the right axis, you're SOL. You
just end up repeating yourself.

When I write Lisp, I never have to make changes that cover more than their one
area of responsibility. At worst, I'll rename a function, and a simple textual
find-and-replace is more than sufficient -- better than most refactoring
browsers, even, since it will hit my comments and documentation. Do fancy
refactoring tools update your README and docs/ folder yet?

(In one case, I knew an experienced C# programmer to build and compile an
expression at run-time using LambdaExpression [1]. It takes about 10 times as
many lines to achieve the same thing, and you have to write in a style that
looks nothing at all like a normal function, so in practice nobody ever does
this. In contrast, the way to accomplish this in Lisp takes _one character_ ,
and the code looks identical to a normal function, so it's not unheard of.)

I feel that Python, in many ways, combines the worst aspects of both worlds.
It's not strict enough to be a good static language, and not powerful enough
to be a good dynamic language. Sadly, the most popular dynamic languages today
are Python and JS and PHP, so a lot of failures and limitations of these
languages get blamed on dynamic languages in general.

[1]: [https://docs.microsoft.com/en-
us/dotnet/api/system.linq.expr...](https://docs.microsoft.com/en-
us/dotnet/api/system.linq.expressions.lambdaexpression?view=netframework-4.8)

~~~
whb07
There is a better approach by using statically typed functional languages like
F#, Ocaml, or Haskell.

They don’t need to have the type declaration as most things can be inferred.

So you get the feelings of writing Python with the full suite of benefits of a
strongly typed functional language when using say F#

~~~
weberc2
I actually find that excessive type inference is much harder to understand.
It’s almost like the worst of dynamic and static types. You have no idea what
the types are, but you know it won’t compile because of a cryptic error
message.

~~~
zozbot234
You can usually get the compiler to tell you the inferred type of an
expression, if only by writing in one that's obviously wrong, e.g. () and
looking at the resulting error message. Some languages, e.g. Haskell support a
"holes" mechanism that formalizes and expands on this 'trick' to enable a kind
of 'dynamic', exploratory programming even in a wholly static language.

~~~
weberc2
Fair point. Perhaps my issue was less about the visibility of the annotation
and more that the types in functional languages tend to be more abstract or
complex (e.g., monads, functors, etc) and harder (for me) to reason about
despite the code being terser.

------
emodendroket
There is part of me that wants to laugh at people independently discovering
over and over again that static typing is actually useful after dismissing it,
but I'm just glad it's available more places.

~~~
tomnipotent
Don't be silly, no one is claiming to be "independently discovering" anything.
Typing is a tool, just like anything else. When I'm working with Python, I
specifically want types during development (thanks PyCharm) but not at
runtime. I spend a lot of time in my code, and type hints provide the same
structure to Python code that they add to Java except when I don't want to
worry about them.

~~~
emodendroket
What I mean is, a while ago you were hearing a lot of ra-ra stuff about how
great dynamic languages were because static typing was just a waste of time
that made you less productive. And then sooner or later all those languages
evolve a documentation system with types in it, and then type hints, and so
on, as it becomes clear that those earlier claims weren't true for long-lived
projects.

~~~
dragonwriter
You see the same in reverse with many industrially popular statically typed
languages gaining dynamic escape hatches, such that optional static typing,
from one end or the other, is pretty much the uniform rule for industrially
popular languages.

~~~
emodendroket
That is true, but as I see it the use of the dynamic features in statically
typed languages tends to be much more limited, with most preferring it as an
absolute last resort.

~~~
joshuamorton
Really? I feel like stuff like DI frameworks are super common in Java, and
they're essentially dynamic magic.

~~~
emodendroket
I'm more familiar with C#, so there may be differences, but while the DI
containers do do a lot of dynamic work, it is carefully abstracted so that all
code the user writes to interact with the container is type-safe. It's very
rare for methods in these languages to return object or dynamic, and slightly
less rare but still far from common for these to be arguments.

Isn't enabling this style exactly why generic support was such a big deal?

~~~
joshuamorton
> I'm more familiar with C#, so there may be differences, but while the DI
> containers do do a lot of dynamic work, it is carefully abstracted so that
> all code the user writes to interact with the container is type-safe. It's
> very rare for methods in these languages to return object or dynamic, and
> slightly less rare but still far from common for these to be arguments.

Sure, but this is true with (most) weird dynamic code in dynamic languages
too. Just look at something like `attrs`. Its letting you dynamically
construct classes, but you can do so in a type safe way.

~~~
emodendroket
The static typing I care most about is function signatures, because it 1)
helps my IDE catch me invoking things in unintended ways 2) lets me figure out
how to use a function without reading it. Inside the body of a method, go
nuts, I don't care. But whenever I'm dealing with a long-lived project I find
signatures are a big help.

------
coleifer
I don't understand why anyone would want type hints. It makes the code so ugly
and complex. Pythons appeal for me was always that it is "runnable pseudocode"
with a strong object system and great libraries.

To me it feels like python is running as far away from simplicity as fast as
possible.

~~~
bakery2k
Spot on. Static typing has benefits, but it also has a cost. Python's optional
type hints have all the complexity but only some of the benefits - in my
experience, runtime type errors are still common.

> To me it feels like python is running as far away from simplicity as fast as
> possible.

Despite its reputation, Python hasn't been a simple language for a long time.

~~~
darkkindness
There really is a cost, in the form of having to rerun code a dozen more times
to catch unresolved type errors. But AFAIK, that's it. Can I ask what I'm
missing? What cost of this (optional) static typing are folks talking about?

IMO this is worth it, since the goal is less about eliminating runtime errors
or other errors, and more about encouraging documentation in code rather than
in comments/docstrings. This makes reading code simpler, not less so.

~~~
bakery2k
In a language like Python that encourages duck typing, type hints can become
extremely complex [1] and can get in the way of the ideal of "executable
pseudocode" [2].

[1] [https://mail.python.org/pipermail/python-
dev/2015-April/1392...](https://mail.python.org/pipermail/python-
dev/2015-April/139267.html)

[2]
[https://twitter.com/dabeaz/status/981832805540909056](https://twitter.com/dabeaz/status/981832805540909056)

~~~
darkkindness
Thanks for the links! Those fully specified frankentypes are indeed atrocious,
and no sane person should ever need to read them. That being said, isn't "Any"
meant to act as a wildcard, used precisely for ignoring the unimportant parts
of the type? Specifying huge and rigorously correct types just seem like an
eager misuse of type hints to me.

At the same time, adding the capabilility for complexity the language does
encourage complexity, doesn't it? (Rust comes to mind.) So I admit what I just
said isn't much of an argument -- frankentype hints _will_ be written, and
therefore we _will_ have to suffer them someday...

------
honkycat
I started using Typescript at work.

It is so joyful to install a library, import it, and then to explore the API
through my turbo-charged IDE thanks to the type-hints provided by the type
files.

~~~
mxz3000
Same here. It's brilliant. The type system is so good that it's painful when
you have to go back to writing Java (for the backends) and it's substantially
inferior/archaic type system. Going to back to legacy UIs that use old school
JavaScript is similarly painful...

------
hansdieter1337
I’m still waiting for a Python fork with a static type system :) One step
closer.

~~~
m_ke
You should check this out:

[https://www.youtube.com/watch?v=Z8vsTxzmorE](https://www.youtube.com/watch?v=Z8vsTxzmorE)

[https://openteams.com/initiatives/2](https://openteams.com/initiatives/2)

> "We will create an embedded domain specific language (DSL) using the Python
> language itself along with the typing module and specialized objects as
> necessary to allow nearly all the existing extensions to be written using
> this DSL. We will then port several key libraries such as NumPy, SciPy, and
> Pandas to use this DSL. We will also provide a runtime so these extensions
> can work on PyPy, C-Python, and RustPython."

I'm pretty sure we'll see a strict, typed subset of python that is jitted or
compiled at some point in the next few years.

Edit: Oh also mypyc
[https://github.com/python/mypy/tree/master/mypyc](https://github.com/python/mypy/tree/master/mypyc)

~~~
pjmlp
A "Crystal" version of Python would be quite sweet, I don't feel Nim is quite
it.

~~~
qmmmur
Depending on what you want Nim is definitely closer than Crystal. Support is
better for cross platform and the number of bindings available to C libraries,
or true native implementations are growing rapidly. For anything ML look into
arraymancer[1].

[1] -
[https://github.com/mratsim/Arraymancer](https://github.com/mratsim/Arraymancer)

------
TeeWEE
I'm working on a big python project, working with 30+ developers on it.

We've added type hints in most places. And it makes it a lot easier reading
other people code and seeying their intentions.

Code is read a lot more often than that its written. And just being more
explicit in what it does makes it better.

Also type hints allow for certain static verification, making the code quality
better.

The big thing i miss is: Python is not a pure typed language.

Programming in Kotlin or Golang feels so much more "engineering". I feel a lot
saver. When it compiles i'm 100% sure which types are coming into my
methods...

With python it can always break in production.

Choosing a type save language is the way togo now. Languages like python and
kotlin are just as expressive as python, but they are just engineered a lot
better imho.

------
rurban
> In general typing local variables is not necessary since their type can
> often be inferred from the assignment / initialization.

That's one of the problems with the idea of such an optional not-integrated
type checker, compared to a type system.

Untyped values can freely change its types during runtime, typed variables
not. Its fundamental safety mypy is violating here. types are giving a strong
guarantee here. Only untyped const variables may be inferred.

Furthermore 3 more fundamental type system features are missing:

1) Typed variables can be unboxed if it makes sense for the compiler. at
compile-time, not run-time.

2) some loop counters can be checked against array indices at compile-time,
run-time oob checks can be omitted then.

3) the ffi needs types to be integrated. you can even layout structs in native
classes as expected in C, completely unboxed. which is about 4x smaller and
faster to use. and much easier to use.

type checking only is only about a third of a real type system. I've
implemented a proper type system in cperl, and worked happily also with perl6
and other integrated gradual type systems, mypy and similar systems for other
languages seem like a joke to me. a hint is something different than a
guarantee.

~~~
jononor
Agreed that this is not ideal. Would be nice to be able to express that the
type does not change. Maybe even have the ability to make it strict, ie unless
opt-out, it is an error to change type of a variable.

------
jane_red
I feel like the primal joy of programming in Python is somehow lost between
those type hints. Type hints even though being very useful do look alien in
Python. I myself like type hinting but cannot get rid of the feeling. Funny,
but in reality, most people fail to find that sweet middle ground of
"optionality". We have a middle-sized project in Python where every single
object is type hinted, even if it is a simple one-liner function. And at some
point you start having a feeling that if that is what Python development looks
like, there must be something we are missing. But what are the actual reasons
of overdoing it? Could it be your prior long-term exposure to the typed
languages, magic IDE plugin or is it just an opinionated attempt to push the
language into unnatural domain? I can't help but keep asking myself these
whenever I see such heavily verbose Python code.

~~~
banannaise
I'm currently working in a Python codebase that I love specifically because it
uses type hints judiciously. They're particularly useful when you're calling a
library function that has an unexpected return type, and particularly useless
when you're passing around a string that's obviously a string.

------
dveeden2
Note that you can query the annotations during runtime:

In [1]: a: int

In [2]: b = 5

In [3]: c: int = 8

In [4]: __annotations__

Out[4]: {'a': int, 'c': int}

In [5]: c = 'foo'

In [6]: __annotations__

Out[6]: {'a': int, 'c': int}

Tinkering with __annotations__ is also needed for hy (python lisp(-like?)
language, [http://hylang.org/](http://hylang.org/))

Did anyone try this with things that interface with Fortran or C?

------
thundergolfer
Dropbox is using Bazel + MyPy to scale it’s massive Python codebase.

Some at my company are wanting the same so I’ve started developing
[https://github.com/thundergolfer/Bazel-mypy-
integration](https://github.com/thundergolfer/Bazel-mypy-integration).

I hadn’t used MyPy much before this work but I’m already quite happy with it.

------
lokimedes
Are we seeing the convergence towards a more generic high-level language. As
an old timer more and more of what I saw as distinct language development are
moving to similar syntax, for instance C++ becoming more Pythonic, Swift,
TypeScript/ES6 and now Python adapting a similar type notation and so on.

------
yegle
One serious problem not addressed in Python type checking is that there's no
Char type: you cannot type hint a function to only accept an iterable of
strings without accidentally accept str itself. The best thing you can do is a
Union of all possible types.

That's why I think the type hinting thing is always a bandaid.

~~~
hannofcart
This really is nitpicking. Not having a Char type is something that has
seriously affected you?

Honestly, how often have you had to write a function that strictly takes a
char and not string? In a decade of programming professionally, I can probably
count 3.

------
Animats
This is a such a strange concept - optional static typing as an external add-
on. All the headaches of needing another tool, none of the guarantees of the
compiler doing the checking, and no optimization benefit from knowing the type
at compile time. Plus the strangest feature of all - a mechanism for turning
off the optional type checking so old code will still compile, even if it
won't work right.

Optional type declarations for function parameters, checked and used by the
compiler, would make sense. PyPy could use them to optimize basic numeric
types. Within a function, you can usually infer types. From an optimization
perspective, the only types worth distinguishing are probably int32, int64,
slotted classes without inheritance, and everything else. "Everything else"
needs the current dynamic dispatch mechanism; the other cases can have hard
machine code generated.

But this optional stuff? That looks like a solution to a political problem,
probably avoiding it being totally ignored. The Python designers used up their
goodwill budget in the 2 to 3 fiasco.

~~~
choppaface
In a lot of contexts for Python, static types don't add anything. A great
example is the Norvig spell checker: [https://norvig.com/spell-
correct.html](https://norvig.com/spell-correct.html)

Furthermore, the duck typing paradigm further warrants Python's position here.
In the above example, the spell checker code can work with a variety of string
types, or even lists, or even perhaps the Tensor-like things found in
Tensorflow and Pytorch. Without duck typing, the author has to use generics
and the user has to play a type game that can become obscene-- see Java.

So that's a dominant mindset that sets the context here; things no doubt look
strange from a different perspective.

I agree it's odd that the interpreter doesn't enforce static type declarations
by default. I've seen now two professional code bases where the authors used
type hints everywhere and I found the code actually disobeyed the hints. (I
guess nobody ran mypy regularly).

That's probably what the Python committee anticipated: a few people would use
this feature correctly, and hopefully build polished linting solutions as
well. So they gave them an affordance for type hints. But most people would
not want this at all, so it's entirely opt-in until it's something polished.

~~~
phailhaus
> Furthermore, the duck typing paradigm further warrants Python's position
> here.

Going to have to disagree with you here. Have you ever used Typescript? You
can have duck typing and a proper static type system at the same time without
devolving into Java. Furthermore, you can write Typescript without using types
at all! Simply have everything default to `any`, and it reduces back down to
Javascript.

Python unfortunately decided to go in a direction that is the worst of both
worlds. You can add type hints, but they do nothing without a 3rd party tool
(?!). This means I can enter a code base, see type hints, and _still_ have no
guarantee that the types are what they claim to be. Instead, we have to rely
on company-specific build processes to give us that static guarantee. And if
you're not lucky enough to work in a codebase that runs mypy before merging,
you might as well throw those type hints out the window.

~~~
choppaface
I’ve used Scala which offers a similar experience. You still fall into the
type game for generics, for example when using the just the Visitor pattern.
Dealing with JSON is also a pain in any language with static types.

I don’t believe you can really have duck typing and static at the same time
unless you throw out the static checks and mark things as “Any” (or the
equivalent). The beauty of Pythonic duck typing is that interfaces need not
have any type declaration and furthermore the user can use dicts with complex
structure (and declaring that structure with type annotations wouldn’t add any
value over examples and/or unit tests).

What makes Scala and modern C++ great is they offer nice interfaces for
expedient use of the static type system. But there’s still plenty of room for
the Pythonic approach of no static types at all.

~~~
phailhaus
> declaring that structure with type annotations wouldn’t add any value over
> examples and/or unit tests

Types can be exported, which means that consumers of your library have an
actual guarantee of what you are returning, rather than just hoping that the
docs are up to date. Javascript is the ultimate duck-typed language given that
everything is really just an object, and TypeScript is extremely effective at
managing it. I can't see why Python would be any different.

~~~
choppaface
right, a lot of python code either doesn't result in libraries (or even
documentation), or library APIs often tend to focus on dicts or integral
types. successful libraries like numpy and pandas focus on a very small set of
well-defined objects and leverage duck typing for containers (and runtime
casting for interop with native code).

I'm not saying arguments in favor of static typing and annotations are
invalid, but rather it's a mindset that can make much of python look very
foreign. the norvig spellchecker is a key example of swath of python code
where static types are essentially irrelevant.

------
acbart
I started teaching my introductory Computer Science class with static types
this past Fall. Although there's a slightly larger initial learning curve, the
long term benefits are pretty massive. I have to do more data analysis, but my
sense is that certain conversations go much, much easier.

~~~
M5x7wI3CmbEem10
which language?

~~~
mixmastamyk
Pascal was a great language for this and still is.

~~~
acbart
You will find very few students eager to learn Pascal in 2019 as their first
introductory language.

I say this as someone who grew up in the 90s with my father's Pascal textbook
and a deep love for Turbo Pascal.

~~~
mixmastamyk
Most kids don’t know much about languages. But its hype cycle is past, I
agree. It fits the requirements given above however. A brand new Pyscal
lovechild would be awesome.

~~~
acbart
Well if we're talking about K-8, that's probably true. I don't know about high
school. For undergraduate students, you better believe that they'll find out
what languages are "real" and "not real" very fast. Whether they're at all
accurate is another matter, but they pick up opinions. Whether it's their
cousin at another school or that one kid in the class who's been hacking since
he was five or just a grumpy senior, someone will come in and fill their heads
with something. I have my own set of (hopefully well-educated) biases I give
them, but I don't like having to argue too much with them when we're starting
out.

A new Pascal would be nice. I loved how much we had to set things up. You
start off the entire program by listing out all the variables and types you
need! What a fun concept. That would be nice to enforce with my students.

~~~
mixmastamyk
Yep. I wouldn’t worry too much on popularity though. After Pascal I took
Fortran as an elective and it was decades past its prime. I didn’t begrudge
the opportunity, I enjoyed learning. We should be learning to program rather
than a specific tool. One popular and one academic lang is a good trade off.

------
stared
Can I use Python type hints in a package?

I.e. is there some way to transpile code (much alike TypeScript to
JavaScript), so that the code can be installed by lower versions of Python
than 3.6?

~~~
dveeden2
The type hints work with older Python 3 versions. They just get ignored and
mypy can use them. However the type hinting syntax for Python 2 isn't the
same. For Python 2 you have to use comments. I don't think there is a way to
convert between Python 2 and Python 3. Maybe a good reason to say goodbye to
Python 2.

------
vkaku
I'd look at this as the way to bridging Python toward a more statically
analyzed future.

------
The_rationalist
When will python have it's typescript?

~~~
hsaliak
.. that’s what the article is about

~~~
ospider
Mypy is way more limited than TypeScript.

