There’s nothing to stop calling functions with incorrect datatypes at runtime and I’d consider this one of the single biggest weaknesses of python right now
PHP went with runtime typechecking, and it isn't pretty. The type system is much less powerful than Python's because it's really hard to add complex runtime checks to an already existing dynamic language. You can make a function check whether its argument is an array, but you can't make it check whether it's, say, an array of integers, because that would require walking through the entire array, or making every array do expensive bookkeeping in case it's needed later.
If you want more complex types you have to resort to phpdoc annotations, which are only useful for offline checkers, like Python's.
When I wrote PHP we used Psalm for offline typechecking. Between that and PhpStorm the runtime checks didn't add a lot of value, and they even made mocking a lot harder at times.
On the other hand I imagine runtime checks are more useful for legacy codebases that are too messy for static analysis. I didn't work on those.
And type hints have been designed with this in mind. It will stay that way.
If this is feature makes you unhappy, Python will make you unhappy. It's important to know what are the values that are important to you and chose your toolbox accordingly.
Couldn't a Python interpreter enforce type checking and raise errors whenever a type doesn't match? I'd be happy to use such an interpreter in many scenarios.
Or you could use Java which it seems is what you wanted to use in the first place and leave Python alone.
If you do it differently, you are not implementing python.
When you’ve worked on a sufficiently large and complex code base maybe you’ll understand.
Edit: coded in python for > 10 years. I prefer it over other languages, warts and all.
- the dev is thinking about the API
- the dev has the knowledge and experience to do it
- the dev has the resource/constraints that allow to do so
- dev has an interest in doing so
- this is a good investment for that particular project
Thinking in black and white does not make for good language design.
I'm glad Guido designed Python and said no to so many requests to make it no Python. Because he sure been pressured to.
> When you’ve worked on a sufficiently large and complex code base maybe you’ll understand.
I also understand Python projects that have a favorable cost/benefit ratio for type hints are not that many.
When you're used to hammer you see everything as a nail, and people loving a specific language feature tend to want it everywhere.
A huge number of dev in the Python community are not professional nor even experienced programmers. And a equally big number of projects have a size or a nature that makes type hint a bad value.
When you’ve worked on a sufficiently diverse project base maybe you’ll understand.
Or when I'm doing data wrangling in jupyter and I never use type hints cause it's all just REPL rapid prototyping and I get autocomplete from the objects.
It's good for checking the internal logic, but being optional is important in the python context.
You can enforce typechecking of your code on the CI with a tool like `mypy` if you want additional safety. But you'll never get the same guarantees you'd get in OCaml or Rust. It's a tradeoff between flexibility and safety.
Python is not Java. It does not enforce type annotations by design. It's not a weakness.
Basically my latest project that I was thinking about is to write a symbolic executor for python that would simply preserve all of the required-type information for all of the execution paths of the program.
Also the other project is to write a tool that allows you to walk all of the possible executions of a given piece of code to find which functions are being called from it - incredibly useful for refactoring of the old code.
Does something like this exists already that you've used and found useful?
- jedi: https://github.com/davidhalter/jedi
- nuitka: nuitka.net/
- python language server: https://github.com/palantir/python-language-server
- pyright: https://github.com/microsoft/pyright
- pyre: https://pyre-check.org/
- and of course mypy: http://mypy-lang.org/
I don't think any of them qualify, but they may all have little pieces.
Also, if you need to analyze Python syntax, baron will help: https://pypi.org/project/baron/
A type-flow analysis? Due to Python language features such a meta-programming and runtime code execution, you may find that at the end of this all your reported types will come out as 'Any' and the tool will be useless!
b) You’re doing metaprogramming wrong if it goes against the assumptions of the initial language you started from.
The literature tells us more than a couple of levels is likely not tractable. That’s not going to enough is it?
I'm not an expert in Python, but I am in dynamic analysis of other similar languages. Python lets you redefine methods, doesn't it? How do you accomodate the fact that while your routine is running someone could be redefining all your methods on another thread?
Nobody's yet managed what you're suggesting being able to do. Maybe you've got some new idea that nobody's even thought of before?
You accommodate that by not doing that because you’re not an idiot.
It’s like asking “why do you need static analysis tools when at any point in time your RAM bit can flip and now you need to model that, too”.
There's no need to be abusive.
> It’s like asking “why do you need static analysis tools when at any point in time your RAM bit can flip and now you need to model that, too”.
It's not like that, because your language semantic model doesn't include random bit flips, so you can ignore that as hardware failure.
But your language semantic model does include redefining methods on a concurrent thread. So you need to handle that.
Otherwise you aren't statically typing Python, you're statically typing a new language that looks a bit like Python, but really is something else.
I don't need to handle that at all if I say that "you can only redefine methods during initial startup which is defined as importing all of the necessary modules". At which point I can run my static analysis tool and it will preserve semantics.
If you're doing meta-programming by redefining methods used in other threads in one thread, then you're clearly redefining the whole idea of your programming language.
> There's no need to be abusive.
People like you (Ph.D in Ruby) have spent too much time doing their doctorate. You went down the rabbit hole of specialisation and now you think you can generalise your knowledge to everything.
I am going to disappoint you:
a. Python doesn't (practically) do multithreading
b. I think you're doing something really wrong if you're trying to do monkey-patching of methods from one thread to another (I don't care which language you're programming in)
c. Plenty of people in this thread have given links to
actual tools resolving the problem you're saying is unsolvable
d. I have an impeccably clear picture of how I am going to solve the given problem - the only reason I have asked the question is because I simply wanted to see prior work on the subject.
Yet you're forcing your opinion. Opinion which is not based on a single quote/whitepaper or a URL.
So I am not being abusive.
But to be even more clear: I am not trying to resolve the problem in the general case (it's pretty obvious you can't). I am trying to resolve the problem which fits 99.9% of the scenarios I have to deal with on a daily basis. And sadly there are _no_ tools for that at all.
For example start with Olin Shivers' doctoral work on k-CFA, and why he needed a k! And then follow the trail of citations from there forward.
Then look at David Van Horn's doctoral work on analysis:
> There is, in the worst case—and plausibly, in practice—no way to tame the cost of the analysis.
> The empirically observed intractability of this analysis can be understood as being inherent in the approximation problem being solved, rather than reflecting unfortunate gaps in our programming abilities.
> As the analyzer works, it must use some form of approximation; knowledge must be given up for the sake of computing within bounded resources.
That approximation is the 'Any' I was talking about.
And he's specifically talking about your idea of symbolic execution and why it doesn't work
> if we place no limit on the resources consumed by the compiler, it can perfectly predict the future—the compiler can simply simulate the running of the program, watching as it goes.
As a matter of fact all examples given in the papers you provide deal with generalised second-order cases of functions accepting functions which can be resolved by simply allowing to annotate said functions with the types by hand instead of inferring aforementioned types.
Functions with annotations had to be called with compatible types, functions without annotations were symbolically executed for each call site. Constructors were special cased so that the object attributes were set up correctly. Types for a stack slot or local could diverge and be narrowed depending on the path through the program.
The amount of metaprogramming used in practice in python made using it for real programs feel insurmountable. Also, I'm pretty sure performance would have been a problem beyond the toy programs I got it working on. But it was a fun exercise.
I'll have to dig up that code
Or maybe it’s needed less there because of typescripts type interference?
Very similar is https://google.github.io/pytype/ which also infers and adds annotations, as well as providing a type checker which in some ways works better than mypy (recursive types!)
I have no idea where is that claim about high productivity of these free form languages came from. In C++ in addition to the nice intellisense compiler checks everything before sending program to debug. So I do not need to do manual testing during runtime to discover typo in my code.
So all in all I'd say productivity in C++ is way better. And C++ is not a stellar example of easy language. Actually it is easy for me. Not because I am super-duper programmer but your basic coding with the help of standard library is a piece of cake. I do not really need to be an architecture astronaut and dive into any esoteric stuff. Simply not needed
To quote Martelli from Google talking about how the Youtube startup outran Google Video's might so much that they had to buy them:
"Eventually we bought that little start up and we found how 20 developers ran circles around our hundreds of great developers. The solution was simple! Those 20 guys were using Python. We were using C++. So, that was YouTube and still is." 
Google took months to copy new YouTube features, while YouTube could catch up in a week after Google innovated.
 Google Books, search term 'python interviews google vs youtube'
2) I do use Python but mostly for administrative/deployment/similar tasks as a scripting language. I see no proof that Python is "several times more expressive". I would really appreciate if you could kindly point me to a real study that shows exactly how it is more expressive and on what in particular the time is saved (you can of course omit the examples where Python is used as a glue language to stitch together few math libraries written in C/++).
I do have my own experience. Of course it is statistically meaningless. My friend and I have implemented simple GUI desktop app (custom video player with some features) as a bet. My buddy did it in Python and I did it in Delphi. I won: 1 hour vs 4.
IEEE Computer 33 (10), 23-29, 2000
80 implementations of the same set of requirements, created by 74 different programmers in various languages, are compared for several properties, such as run time, memory consumption, source text length, comment density, program structure, reliability, and the amount of effort required for writing them. The results indicate that, for the given programming problem,“scripting languages”(Perl, Python, Rexx, Tcl) are more productive than conventional languages. In terms of run time and memory consumption, they often turn out better than Java and not much worse than C or C++. In general, the differences between languages tend to be smaller than the typical differences due to different programmers within the same language.
" Most programmers in the script group used the associative arrays provided by their language and stored the dictionary words to be retrieved by their number encodings"
For C++/non-scripting they either used plain arrays or custom build tree in one case. So they had to write lots of code.
Sorry but there are associative arrays in C++ standard library and search and this and that. Building the program they've specified would be a no brainer in modern C++. This article is irrelevant for current situation.
You can still write python programs without them.
You can add them only partially.
You can also add them later.
Python stays Python.
MonkeyType lets you take a code base that is untyped, or partially typed, for historical or because it was not worth it at the time, and turn it into a fully typed project quickly.
It makes the transition between the exploration phase into the industrial phase much easier.
For me, it's kinda fantastic to be able to fiddle with a design, changing my mind again and again, without having to fight the type system, then once things are settled, add an additional safety net on top.
It's also very reassuring knowing I can just hack things, knowing that later on, if I want to take it to the next level, I have the option to change my mind about type hints.
- never become statically typed
- never make the type hints mandatory
- never enforce the type hints at run time
Python is jokingly said to be "the second best language for most things". To achieve this, it has to be able to be decent for many different purposes such as setuping machine learning models, scripting SIG system, doing test automation, powering a web site, provisioning linux servers, scrapping web page, batch renaming files and teaching code, analyzing numbers from a hdf5 file, being the client to a database/ftp/ssh/imap system.
It means it can't just be purely functional/OOP/imperative, be must borrow from each paradigm. Be statically typed or not, but gives features from both. Etc.
Because you don't want type hints when you are in jupyter or scripting your linux box. But you may want it when your Saas reach 3 millions lines.
Type hints are already mandatory if you want to use data classes. This is unfortunate - the two features really should be orthogonal.
At least when Guido was still BDFL, there was little resistance to type hints encroaching on unrelated areas of the language, and hence becoming less-and-less optional.