
Statically-typed error handling in Python using Mypy - Agathos
https://beepb00p.xyz/mypy-error-handling.html
======
staticassertion
Here's my opinion, having written many thousands of lines of mypy code.

* Third party libraries are still not typed, which sucks

* Inference is weak. Sometimes an 'if' statement narrows accurately, sometimes it doesn't.

* Generics are extremely confusing, moreso than any other typed language I have used. Any Generics are even more confusing.

* Fundamentally, Python is structured such that the concept of interfaces are coupled tightly to implementation of classes, which is severely limiting. As an example, a module A can declare a class, a module B can declare an interface (Protocol), but module B can not implement that interface for A (without modifying A's code).

* Errors from mypy are still very very light. There are very few "Try XYZ to fix it", and it's just a single line of read text that amounts to an assertion.

* Still weak support for JSON (use Dict[str, Any])

* Slowly improving but still not amazing support for Self types

I could go on.

Mypy is awesome but it still feels very immature, and as it should since it's
pre 1.0. It does not come close to matching the experience of other languages
with static types.

~~~
Caligatio
* Still weak support for JSON (use Dict[str, Any])

Python 3.8 added TypedDict which finally helps this particular shortcoming.

~~~
avremel
I was surprised to find out that MyPy will throw an error if you use a
variable to access a key
([https://github.com/python/mypy/issues/7178](https://github.com/python/mypy/issues/7178)).

Optional keys are also a nuisance.

~~~
dharmab
How are optional keys a nuisance? I find that using the patterns:

    
    
        if "optional_key" in my_dict:
            do_something_with(my_dict["optional_key"])
    

Or:

    
    
        optional_value = my_dict.get("optional_key")
        if optional_value:
            do_something_with(optional value)
    
    

Works as expected. Nested optional keys can be a bit annoying, although:

    
    
        my_dict.get("optional_parent", {}).get("optional_child")
    

seems to work.

~~~
avremel
I was referring to defining optional keys.

If you have a dict with some keys which are optional, you need to create a
separate subclass (with `total=False`) just for those optional keys.

With TypeScript, I can just use `key?: type`.

[https://mypy.readthedocs.io/en/latest/more_types.html#mixing...](https://mypy.readthedocs.io/en/latest/more_types.html#mixing-
required-and-non-required-items)

------
roseway4
Having recently found myself switching between Python and Scala, I've come to
value IntelliJ's type hinting and red squiggly prompts, not to mention
compile-time errors.

While mypy and type hinting support in PyCharm and VSCode is great, it's not
the seamless experience as with typed languages and the lack of (optional)
runtime typing still allows all sorts of bad things to happen. A "safe" mode
for Python, where typing and perhaps some other stuff is checked at run-time,
might be an interesting advancement for those in the community building larger
or mission-critical systems.

~~~
astrea
My opinion is that python shouldn't be used for mission- critical systems
anyway.

~~~
staticassertion
People are gonna dogpile you, but if you'd said this about JS they wouldn't
have blinked.

~~~
pytester
Coz there's a difference between "dynamic type system" and "type system that's
thrown together with bits of string and rubber bands":

[https://www.destroyallsoftware.com/talks/wat](https://www.destroyallsoftware.com/talks/wat)

~~~
staticassertion
Python has is own 'wat' stuff, especially Python 2.

[https://stackoverflow.com/questions/3270680/how-does-
python-...](https://stackoverflow.com/questions/3270680/how-does-
python-2-compare-string-and-int-why-do-lists-compare-as-greater-than-n)

"CPython implementation detail: Objects of different types except numbers are
ordered by their type names; objects of the same types that don’t support
proper comparison are ordered by their address."

~~~
detaro
> _objects of different types always compare unequal, and are ordered
> consistently but arbitrarily_

Seems like a completely reasonable thing to do?

~~~
staticassertion
> When you order two incompatible types where neither is numeric, they are
> ordered by the alphabetical order of their typenames:

????????

~~~
detaro
That's entirely an implementation detail. The thing that matters for
programmer experience is the specification of behavior: There's going to be
_an_ arbitrary order. You certainly can argue that you think it would be less
surprising if trying this just failed, or that promising a stable order ties
down implementations too much, but I don't think providing an order is that
surprising.

None (as far as I remember) of the common "wat" examples about JS are
interpreter implementation details, but specified behavior.

I'm mildly curious why CPython chooses to implement it this way, but if I had
to guess: I'm assuming it is to provide a stable order between objects of
different classes with the same hash() value. Hash-value being what I suspect
is what the quoted answer misrepresents as "their address" (the address being
in CPython the default fallback for objects that do not implement a hash()
function), and being a good candidate to establish an arbitrary order.

~~~
staticassertion
Sorry, I'm not sure I understand at all the difference between this and the
'wat' stuff.

------
foxes
I think in the long run, python, ruby, js, etc are all bad ideas. You are
solving high level problems with these languages, correctness should be the
main priority.

Realising that type checking is a good thing, then bolting it on, I feel is a
crappy solution, when you should just use something designed better from the
start.

I completely disagree that there is a "productivity toll". If you dont know
how to speak a language then obviously it seems difficult. I think you save
time in the long run not dealing with all the type errors.

~~~
jtdev
>> “ I think you save time in the long run not dealing with all the type
errors.”

Have you written and deployed production code in the languages you mention
above? Did you encounter a substantial number of type errors?

~~~
staticassertion
There is no good definition of a type error. It certainly goes beyond
TypeError(Exception).

Is a '.close()' method being called twice a type error? It is in a language
that can express states in the type system.

Is SQL injection a type error? It is when you use refined types in your sql
library interface.

The vast, vast majority of errors I run into are errors that, with effort and
the right type system, I could turn into type errors.

The point being that asking "would those be type errors?" is a really big
question that is, when answered simply, "yes".

~~~
mixmastamyk
There aren't any mainstream languages that have the ability to detect the
ultra high-level errors you describe. Maybe typescript is getting better.

Pyflakes and unit tests will detect the great majority of potential errors.
Mypy gets you closer to zero.

~~~
yawaramin
There are very usable, well-supported languages out there right now that do
have such type systems. Saying they're out of consideration because they're
not mainstream is a circular argument.

~~~
coldtea
> _Saying they 're out of consideration because they're not mainstream is a
> circular argument._

No, it's a pragmatic argument. There's no circularity it.

If they're not mainstream, then they're neither "very usable" or "well-
supported" to any extend that a mainstream language would be.

~~~
yawaramin
Here's the circularity:

\- There are no mainstream languages with powerful and convenient type systems

\- But there are less mainstream ones

\- But I won't use those

\- There are no mainstream languages with powerful and convenient type systems

~~~
coldtea
That's circularity alright, but not as in a "circular argument" though. It's a
feedback loop.

That's however a totally legitimate engineering choice.

Engineers picking on a language shouldn't bet on less mainstream incomplete
environments with less tooling and libs and options and devs, just so that
they can raise them into the mainstream.

That might be a "tragedy of the commons" thing, but it's not an engineering
obligation to be an early adopter.

~~~
yawaramin
They're not _obligated_ to by early adopters, but then you hear comments like:

> There aren't any mainstream languages that have the ability to detect the
> ultra high-level errors you describe.

And it's like, well there are great languages that can do that, but if you
restrict yourself to that tiny 'mainstream' subset, you'll never know it.

And moreover I think engineers (or rather, companies) are way too conservative
about this stuff–picking 'mainstream' tech can be a touch-and-go proposition
at any time. Just because something is mainstream, doesn't mean it's the right
choice for your project.

------
karlicoss
Hi, I'm the author! Happy to answer your questions here!

~~~
dbrgn
Hi, thanks for featuring my result library!

Regarding your criticism that `result.ok()` returns `None` if the value is not
an `Ok` type: That's what `result.unwrap()` and `result.expect(msg)` are for
:) (There's no unwrap_err and expect_err so far, but PRs are welcome.) The lib
is strongly inspired by Rust (see [https://doc.rust-
lang.org/std/result/enum.Result.html](https://doc.rust-
lang.org/std/result/enum.Result.html)), that's why `result.ok()` returns
something option-ish.

I like your `isinstance` approach btw! Will have to think about it a bit more.
Reminds me a bit of Typescript as well. Does mypy have type guards
([https://www.typescriptlang.org/docs/handbook/advanced-
types....](https://www.typescriptlang.org/docs/handbook/advanced-
types.html#type-guards-and-differentiating-types))? Maybe that could be used
to create helper functions that check for success/failure and which help mypy
to derive the correct type.

~~~
uryga
> Does mypy have type guards?

not yet, maybe in the future. might be possible to emulate some use cases with
Literal Types. tracking issue:

[https://github.com/python/mypy/issues/5206](https://github.com/python/mypy/issues/5206)

------
Riverheart
Appreciate how in depth this is, looking at all the possible ways of handling
the issue. Makes the final result more interesting knowing the journey it took
to get there.

------
sandGorgon
Has anyone evaluated libraries like Pydantic ( [https://pydantic-
docs.helpmanual.io/](https://pydantic-docs.helpmanual.io/)) and Encode
Typesystem
([https://www.encode.io/typesystem/](https://www.encode.io/typesystem/))
versus MyPy as a real life thing ?

~~~
fermigier
These are completely different objects: data validation libraries.

I think naming the latter "Encode Typesystem" is misleading (and your question
is an indication that I am right).

~~~
sandGorgon
why is it misleading ? didnt understand. i just didnt want people mistaking
"typesystem" as a word rather than a project name by Encode.

its a very common name.

------
macca321
I maintain a library for doing exactly this in C#
([https://github.com/mcintyre321/OneOf](https://github.com/mcintyre321/OneOf)).
I tried to replicate it in Python as I'm writing it at the moment, but
couldn't quite manage it. Can you have a multi generic typed Union in python?

~~~
Recursing
Do you mean something like this?

    
    
      T1 = TypeVar("T1")
      T2 = TypeVar("T2")
      GenericUnion = Union[T1, T2]

~~~
macca321
Yes I can't quite remember why I couldn't get it to work. I want to make
something where this is valid, but couldn't manage it:

    
    
        def foo() -> OneOf[Bar, Baz]:
          if blah():
            return Bar()
          return Baz()
    
       
    
       def bar_to_blah(bar:Bar) -> Blah :
         ...
    
       def baz_to_blah(baz:Baz) -> Blah :
         ...
    
    
       one_of_bar_or_baz = foo()
       blah = one_of_bar_or_baz.match(bar_to_blah, baz_to_blah) 
    

but the critical thing is to get a type error if you add another case to foo's
return type, or change the generics to different types.

