I’ve written a lot of production Python and also a lot of thoroughly and rigorously statically typed code in C++. I use both regularly but there are clear tradeoffs in this regard. The main reason to avoid static type checking is that the programming language makes it difficult to do it effectively due to trading it away for other benefits. This does not change the intrinsic value of static type checking.
Python works great for cases where the scope of a particular function is tightly constrained around a tightly constrained set of inputs. In this scenario, it is relatively easy to test fitness for purpose, and that testing can replace static type checking as a practical matter. Many bugs sneak through in Python that C++ would find at compile-time, but that’s okay because the domain is so narrow that you can squeeze them out in testing with minimal overhead.
The minute you start trying to build a more generalized library of functionality in Python, the combinatorial bug factory starts to become a major drag. While you can add type checking, it is clumsy, slow, and less thorough in practice than you would expect with C++. Python is not designed to provide the degree of static type checking something like C++ can. Many types of code benefit from thorough static type checking, and for those use cases something like Python has a high cost.
This is fine. I understand the problems for which both languages are best suited. Right tool for the job and all that. Static type checking is generally beneficial in software but it isn’t always worth paying the tariff if the problem domain is narrow enough. It is quite possible to overfit static type checking to simple cases that don’t justify the effort.
I’d probably agree with you 5 years ago, but in those 5 years typescript has shown that you can take a completely dynamic language, add a very powerful and well thought out type system and the resulting language is so much better for non-trivial projects.
So if you ask me, who has been programming Python since before new-style classes, if types make sense for Python, the answer is a resounding ‘yes’: just steal as much typescript typing as possible.
At first glance, JavaScript/TypeScript and Python seem like similar cases, but in practice, I have never found them to be so.
The JS/TS community has enthusiastically adopted static typing, to the point where (a) the type system in TypeScript is reasonably clean and intuitive for everyday work, (b) almost every major library uses static types and specifies them in its documentation, (c) almost all major tools and development environments pick them up and work well with them, and probably as a result of those things, (d) almost everyone now uses static types extensively, at least for new development.
The Python community seems far more hesitant. I don’t know why. Maybe it’s because we’d just had such a long transition from Python 2 to Python 3 and introducing static typing feels like another big change? Maybe it’s because Python’s static type system simply hasn’t been as clean and tidy as TypeScript’s, particularly before the recent generics changes in 3.12? Whatever the reason, in practice none of (a)–(d) above reliably holds in the Python world. Personally I do still prefer to use static types in new Python code, but it’s frustrating and often needlessly time-consuming when popular libraries don’t.
While there are countless cultural and technical reasons why TS ended up more successful than Python here, I think the single most significant aspect was that TS decided to go with structural system while Python went with nominal. While nominal systems in general are fairly popular and well-received, the chief thing here is that structural types match how the language was being already used. Especially for Python they had before embraced "duck typing" as their idiom and nominal types are in conflict with that.
This is my single greatest complaint about Python's type system. Protocol should have always been the primary building block rather than just being relegated to something you use for "duck typing". Subtyping fits much better with things like the dynamic approach of the Django ORM, for example.
Not sure what you mean, while it’s true that it would have been nice that Protocols were always the primary building block, as of python 3.8 they _can_ be depending on your use of Structural Typing https://peps.python.org/pep-0544/
Yes. Great to read this type of nuanced comment. This nails it. Sometimes components are generic enough that type checking does not add any value. Type checking is useful when you want to frame your logic in terms of your business domain. But you don't want your business domain to leak into low level reusable components. Why constrain them to a specific domain when they could suit multiple domains without any effort.
Just think of a function which generates a random number for a lottery app. Do you want to call the function 'generateRandomNumber' or 'generateLotteryNumber'? There is no reason to constrain your function to the 'lottery' use case when it could also serve an infinite number of other use cases.
The function which is used as a dependency shouldn't concern itself with the specific business case towards which it will be applied.
As you move towards the leaves of the dependency hierarchy, the abstraction becomes more about technology and less about business domain.
It doesn't work very well for generic concepts. For example, let's say I have a dependency which exposes a function which takes a URLDefinition as an argument.
Maybe the dependent logic already has a similar type to represent URLs but maybe it's called URLInfo... The property names might be slightly different. Maybe URLInfo has a Host and Port property starting with capital letters but URLDefinition properties are all lower case.
IMO, this is a huge mistake. URLs should simply be treated as plain strings that way you don't need to worry about unnecessary interface mismatches. Inventing complex interfaces instead of just representing concepts as primitives adds unnecessary friction.
The approach you suggest is commonly referred to as 'primitive obsession' and is considered by many (including me) to be an antipattern. Having primitives everywhere means you have to repeat validations everywhere, methods will have more parameters, and it's much easier to accidentally pass these parameters in the wrong order.
You can pass simple 'info' objects. So long as they're just raw objects/data and don't expose methods. The main idea is that interactions between components is about messaging/communication.
Parent components give their children the necessary info to do their jobs but the implementation details of how they do it is fully encapsulated within the children components.
When components start passing other components to each other, they're no longer merely communicating with each other, but they are attempting to micromanage each other's functionality. It's a failure of the separation of concerns principle. There are always alternative approaches; for example, the child can emit/trigger an event to let the parent component know that a low-level condition occurred, but let the parent decide how to handle it because the parent is responsible for that higher level of abstraction.
This is not a new concept I invented, it's essentially what Alan Kay (the inventor of OOP) said decades ago. He said that OOP is about messaging.
There can be exceptions to the rule, but the idea is to keep those exceptions to a minimum.
Regardless - this is very much a strawman. I could (try to) pick apart the example, but I'm not going to. Sometimes a complex interface is required, sometimes a simple one.
Scepticism is warranted for partial type checking. Something I've noticed in partially-typed situations is they are the worst of all worlds. The type checker soaks up a lot of time, but the silly bugs that type checking would have caught still get through. Having 1 library that doesn't support type hints is a major problem if I'm trying to use a type checker.
Therefore, I think "... the authors have no desire to ever make type hints mandatory, even by convention." is useless. I'm sure the authors are honest about their desires, but frankly either everything gets type checked or practically nothing does. It is unlikely that there is a social equilibrium where some things get checked and some do not. Not only that, but clever libraries that aren't written with type checking in mind are often a devil to hint retroactively.
Strong agree that half-assed type checking is the worst of both worlds. Most of the effort of real type checking but only a fraction of the real-world benefit.
I think there is a big difference in what we type-check. The interfaces of public methods should be declared and type-checked. What happens inside a function not so much.
I agree with this. For example, it leads to weird situations where a linter with a strict type checking configuration marks some code as a type error, but it runs just fine.
Which, in turn, leads to you ignoring type errors entirely, since you don't know which one are spurious and which ones aren't.
The wrong assumption in that page is that Static Typing implies the need for type hints or annotations.
In fact, type inference is a thing.
Most of the languages with Hindley Millner type system (such as oCaml, F#, Haskell, PureScript) do feel dynamic (e.g., developer does not need to hint the compiler) yet they are purely static typed.
What is the argument against having Python checking the types, without asking the developers to change their code?
Haskell has to pay _a lot_ to achieve full type inference. A price that is way too high imo for a feature that nobody uses anyway. Among many other things it precludes subtyping, which I assume Python relies on heavily.
I suspect almost every Haskell uses full type inference almost constantly, if only to avoid having to write out top level types by hand. I know that applies to me. I've never considered that I paid a price for it. For me it was just something that came free with a language that is nice in other ways too. What price are you seeing? (For me, absence of subtyping is a feature.)
Personally, I don't use type checking because the benefits are extremely marginal given how I write software.
- I avoid complex function/method interfaces. My coding philosophy revolves around relying on simple primitives as arguments and return values and keep complex state fully encapsulated.
- I use a TDD e2e approach which gives me excellent code coverage and catches 'type mismatches' early as such issues cause functionality problems that are caught in my tests. Type checking is redundant.
- My software is made up of small modules. The lower level modules are generic enough that it doesn't make sense to restrict to specific types or assume a specific business domain.
At a psychological level, I find that type checking tends to encourage developers to design complex function/method interfaces which lead to worse, harder to maintain software and encourages leaky abstractions and passing around 'live instances'. It encourages developers to add unnecessary constraints which make low level modules less versatile and cover fewer use cases than they could otherwise have.
Interface names of low level modules often end up unnecessarily referencing the specific business domain which should be a higher level concern.
Thats what I call "Uncle Bobs ossification disease" where you spend too much time on process and not enough on getting shit done.
Because everything is broken up again and again it just piles and piles, making it harder to change things. Everything is intertwined through (often unwarranted) reuse and even the non-reused stuff is smeared over dozens and dozens of functions that are used only at a single call site.
But wait there's more, for every of these functions theres half a dozen tests that ossify some implementation detail (because function boundaries are no longer meaningful).
You end up with thousands of lines of code that do very little except for being the physical incarnation of "the process" god.
That stuff is so 2000.
Type systems give you robustness and whole program consistency proofs without a single line of extra code. No manually writing tests to ensure type safety and you get help writing semantically meaningfull functions, instead of whole object graphs that need to be called in a complex dance just to avoid having 4 function arguments.
If you saw how little code my projects require, you would realize that this is incorrect.
I've built a decentralized cryptocurrency exchange that can work with most blockchains in just 5000 lines of code. Has been running for 4 years with zero bugs.
I've written a cryptocurrency from scratch with only about 4000 lines of code and have not had any bug reported after 3 years of continuous operation. On the other hand, projects like Bitcoin and Ethereum are hundreds of thousands of lines.
I've also written a no-code platform in less than 4k lines of code which supports complex views with real time updates, pagination, indexing, access control, multi-tenancy and has been bug-free since it launched. I built the whole thing in about 3 months part time.
All of these built with plain JavaScript/Node.js. I can support all of these projects in parallel because I rarely need to update them.
Not so, they're quite short and feature-rich.
When I read the code of most other people's projects, I'm often shocked at how over-engineered they are. A lot of code merely wraps some other logic or there are unnecessary abstractions which add unnecessary layers of indirection.
Very fragmented codebases like that have turned "inline function/method" into my favorite automated refactoring. I'm not an IDE person generally, but I open one up for that specific use. My favorite game to play is to see how many times I can inline with unchanged or net negative lines of code in a commit.
Getting rid of dumb tests is often where the real line savings is. No need to test your specialized internal function that's just wrapping filtering an array. Like, yes, array filter does in fact filter according the predicate you give it.
Personal opinion here but I’d put TypeScript in the S-tier for “static typing overall enjoyable-ness,” Java in D-tier and it pains me to say Python in F-tier.
Yeah, I've recently removed type checking in a Python project because it was causing more false positives than true positives, and working around the type checker's complaints was just getting too tedious.
I'm a big fan of type checking in general, and Typescript is a great example of a type checker that allows you to statically define all the dynamic idioms that you'd normally use. But for various reasons, that just doesn't seem to have happened in the Python ecosystem. Instead, you can either write normal dynamic Python without types, or you can restrict yourself to a vaguely Java-like subset of Python with hard-to-use generics and confusing rules about forward references.
I wonder if it would have been better to have taken the Typescript approach and have Mypy and all its annotations developed as a separate language, rather than as part of the Python language, with the expectation that they could eventually be merged at some point. As it is, it feels like both sides have been very constrained: Mypy has struggled to innovate on expressive type-level syntax because it only operates on standard Python code, and Python has struggled with balancing the various goals that different groups have for youry annotations.
The result is a kind of mess where it's still painful to type basic things like generics (yes, that's changing, but not if you need to support older Python versions), but you also have weird syntax that requires type annotations, even if they're meaningless (like with dataclasses).
When I removed the type checking stage, that was with Pyright, which I'd used because of these sorts of recommendations. It has some advantages over Mypy, sure, but it still ended up being more hindrance than help. Particularly with things being of type `Unknown`, which produced all sorts of errors that I could never properly disable.
Yeah the formulation is quite biased, but to some extent there is some truth in the statements:
> Your codebase is old, large and has been working fine without static type checking for years. While Python’s type system is designed to allow gradual adoption of static type checking, the total cost of adding type annotations to a large extant codebase can be prohibitive.
That's a convoluted and quite dishonest way of warning that you're trapped if you don't enforce types from the start; but it does have the merit of being on the list.
None of these reasons are particularly compelling for any case I've encountered, other than your code base is old and / or poorly designed, and you chose to write it in a language without static type checking.
Yep. Speed is one factor, but also standardisation.
I’ve always been using mypy, but heard good things about pyright. It’s what VSCode uses by default as well, unless you jump through hoops. So I have pyright in the IDE and mypy everywhere else, yet they don’t agree with one another in various cases. Exhaustiveness checking and type inference in structural pattern matching, and async TaskGroups are some examples. One of my code bases has mixed “type: ignore” and “pyright: ignore” (forget the exact syntax) for that reason. Pretty bizarre and not something you’d see in any other language!
I wonder why mypy isn’t the blessed standard and pyright was able to catch up on (and in parts overtake) it. Mypy had a large head start and enjoys tons of development effort. Good job to Microsoft I suppose?
Also, in Python (unlike TypeScript or PHP) you need to be sure that the code regularly passes type checking to have confidence in your type annotations.
I've dealt with my share of code with types that were almost correct but not really and it's been quite frustrating.
There are cars and motorbikes, and many other things. They have different benefits and costs.
Python is a motorbike. It is small, convenient, nippy, fuel-efficient. It is also terribly unsafe, holds no luggage and no protection from the weather.
Statically-typed languages are cars. Stable, safe, predictable, require a lot of infrequent maintenance, weather-proof. Not convenient in heavy traffic, but hey, you put on the heater seats and the stereo and you don't mind queuing.
Now, I find the effort to add typing to Python like trying to make the motorbike safe. You double the wheels to have a full 4-wheel vehicle, add a roof and windshield... And you end up with something that has th upsides of neither a car nor a motorbike.
Where people find it convenient, sure. But the evangelism of "types everywhere" seems to not get it.
To steal someone else’s comment from above, I feel like TypeScript has disproved this, and that you lose almost nothing and gain a huge amount with a high-quality optional typing system
I'm not really sure what is meant by "avoiding static typing" in Python -- it's just like saying "avoiding unit tests", which is done by not running them (or even writing them) in the first place.
Python is strongly typed, but it does not mandate static type analysis -- unlike most other typed languages, where it is mandatory at compilation time. But if you choose to use it, there are tools which will run it for you at any point in time (mypy being the reference implementation). You don't even have to write any special annotations for that: the checker will try to infer the type when possible, or use Any when it isn't.
The crucial thing to understand is that Python is still dynamic at runtime -- there are no checks that an argument is of the correct type defined for the corresponding parameter. So your program can fail the static analysis but correctly run in practice, unlike most other statically checked languages.
Another effect of the above, which is often overlooked, is that many developers tie themselves into knots to get the type annotations right on unit tests. This is completely missing the point, as unit tests and static typing are at completely cross purposes: while static analysis evaluates, as the name implies, the code in its static state, the unit tests only make sense when they are executed, i.e. at runtime.
So the practical rule of thumb, learned from practice, is:
1. Regularly run the static analysis on your application (and library) code, adding any necessary annotations, to weed out various problems that might arise from mismatched types.
2. Write thorough unit tests -- ideally with plenty of property tests with tools like hypothesis [0] -- and run them regularly.
Python is a tool that is meant to serve you. Python is a big tent, multi-paradigm language that generally allows you to do things in the way that best suits your needs, as best determined by you.
^^ this excerpt from the post made me spit my wine out ... this is what https://raku.org is for!
> Pleasing static type checkers requires a non-zero amount of busy work
Heck yeah! Why would I want to find problems ahead of time when the whole thing could blow up at runtime with cryptic errors. Or better yet, no errors because I’m such a ducking elite programmer that I never make mistakes. Read the code. The code don’t lie!
This is a lame, low effort blog post. Jesus Christ.
Your comment is incredibly hostile. They're expressing their opinion on the language they use and like. A language that's always been that way. I find it incredibly tiring that every language goes to the fase of people complaining it's not other language and fighting to change it instead of using another one.
Python works great for cases where the scope of a particular function is tightly constrained around a tightly constrained set of inputs. In this scenario, it is relatively easy to test fitness for purpose, and that testing can replace static type checking as a practical matter. Many bugs sneak through in Python that C++ would find at compile-time, but that’s okay because the domain is so narrow that you can squeeze them out in testing with minimal overhead.
The minute you start trying to build a more generalized library of functionality in Python, the combinatorial bug factory starts to become a major drag. While you can add type checking, it is clumsy, slow, and less thorough in practice than you would expect with C++. Python is not designed to provide the degree of static type checking something like C++ can. Many types of code benefit from thorough static type checking, and for those use cases something like Python has a high cost.
This is fine. I understand the problems for which both languages are best suited. Right tool for the job and all that. Static type checking is generally beneficial in software but it isn’t always worth paying the tariff if the problem domain is narrow enough. It is quite possible to overfit static type checking to simple cases that don’t justify the effort.