Hacker News new | past | comments | ask | show | jobs | submit login

What would you say are the biggest blockers to becoming realistic? I saw on the README they need tools in the Python ecosystem to start utilizing them, which I can help with starting with isort, beyond that I'd want to do whatever I can to help the project succeed.

Probably the biggest issue is that it can't run many libraries and frameworks because they use a lot of dynamic features, i.e. reflection and metaprogramming.

To be more specific: getattr, operator overloading, descriptors, heterogeneous dicts, decorators, etc.

Type checking and metaprogramming are fundamentally at odds [1]. Dynamic languages like Python have more of a focus on the latter. They later added type checking, but it comes at the "cost" of ruling out the more idiomatic metaprogramming and reflection features. In other words, static typing makes your source code bigger.

Well, optional typing to some degree lets you have the best of both worlds -- you can skip type checking of the hard parts. But optional typing doesn't let you compile your program to make it faster -- you need a fully-typed program for that.


I'm doing something similar to mypyc with https://www.oilshell.org/ (I actually visited Dropbox and chatted with them about it back in the spring.)

The difference is that I'm compiling Oil's Python source to C++ rather than to Python-C extension modules. So it doesn't depend on the Python runtime. It's not done but it's working well so far, and it's given me a lot of appreciation for which dynamic features Python programs actually use! (both my own and others)

Also note that mypyc was used to speed up mypy, which is a type checker. A type checker is a very particular kind of program that's different than 99% of the use cases of Python. So success on speeding it up is super impressive but it's not clear it generalizes.

The same is true for Oil -- my translation work doesn't generalize to arbitrary Python programs. Lots of people have died on that hill because it's really hard. You have a hard tradeoff between the kinds of Python programs you can support and the speedup you can give them. There are 10-20 projects over the last 2 decades at various points along that spectrum. In addition to mypyc, Oil's strategy was also inspired by Shed Skin, which is an impressive but mostly dormant Python-to-C++ compiler.


So in short I would say the problem is that nobody will be able to agree on a subset. You will have a lot of different fragments of Python geared toward particular use cases.

But Python will very often be more appealing than any of those fragments because it has a bigger ecosystem. One thing that I've appreciated more and more while designing a language is how much the network effects and inertia matter. It's why we're still using C and C++ after almost 50 years. I'm sure every day there is still a lot more C++ written than Go, Rust, Swift, and D combined, etc.

Python has a similar network effect and it will be around basically forever in its current form. Software doesn't really get rewritten or reduced -- more stuff just gets added on top.

[1] I wrote some posts about that tradeoff here: http://www.oilshell.org/blog/tags.html?tag=metaprogramming#m...

> type checking and metaprogramming are fundamentally at odds [1]

I would say that dynamic metaprogramming is at odd with type checking (and optimizations, and in general understanding the behaviour of a program statically). But of course metaprogramming can be done perfectly fine in a statically typed language.

There's less conflict if you release the features in the same compiler of course, but it's definitely not "perfectly fine".

My blog posts give some more color on that, but also see:

https://discuss.ocaml.org/t/the-future-of-ppx/3766 (breakage)

https://words.steveklabnik.com/an-overview-of-macros-in-rust (breakage)

Also, OCaml has had at least 4 different metaprogramming systems -- ocamlmeta, camlp4, ppx, etc. Rust is getting its second one post-1.0 as AFAIU.

There are lots of open problems related to metaprogramming and type system design because they interact heavily.

This blog posts also hints at that: https://nim-lang.org/araq/v1.html

In contrast, in Lisp, metaprogramming is "just programming".

There isn't anything specific of static type systems in those two links. Both ocaml and rust had powerful plugin systems that exposed the internal of the the compiler AST and they decided that they do not want to expose it as a stable interface. But exposing implementation details is not required to have a powerful metaprogramming environment. As far as I understand rust macros never did expose these internals and had no such issues.

Also compare with C++ metaprogramming (whose syntax is certainly awful, although it has been continually improving), but works perfectly fine with its type system (in fact most metaprogramming in C++ is done via the type system).

On the other hand CPython also exposes the runtime internals to plugins and de facto that prevents the language from evolving and alternative implementations to gain a foothold, so the issue of exposing implementation details preventing language evolution is not restricted to static languages.

edit: the hard part of typing and metaprogramming is making sure your metaprograms are well typed, i.e. the generated program is guaranteed to typecheck. This is great, but it is not a strict requirement, if you are happy with syntax macros a la rust or unconstrained templates a la C++, there is no particular issue. Your genrated program will be still be typechecked at compile time, which is still better than having a runtime error because of a bad metaprogram.

The point is that they all struggle with the design of metaprogramming. Why are there 4+ different systems in OCaml as "addons" whereas in Lisp and Python it's integrated in the language?

It's exactly analogous to types being integrated in OCaml, Rust, etc. whereas in Python, JS, Ruby, and PHP static typing it's an "add-on".

Scala also has a newer system https://scalameta.org/

C++ has new dramatic new proposals 20+ years after templates, addressing really basic use cases:


The point is that no language has gotten it right so they keep introducing new systems and breaking old ones.

C++ templates also have the mistake where type checking is done after template expansion. That's why you get terrible error messages.

If you're not convinced, that's OK, but try watching the talk by Yaron Minsky here:


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact