
Reified Generics: The Search for the Cure - wstrange
https://gbracha.blogspot.com/2018/10/reified-generics-search-for-cure.html
======
_hardwaregeek
I'm nowhere near qualified enough to critique this article in a meaningful
manner, but I've been thinking recently that types and compile time functions
go hand in hand.

Type systems are essentially an attempt to embed a logic that can prove
various invariants about the code. What varies across type system to type
system is A. what you can express and B. how much you can check. The issue is,
most programming language designers decide these two things for you. That's
why stuff like reification vs type erasure bites designers; the decision is
irreversible.

But what if it wasn't? And this seems to be what this article is proposing.
That if you can write compile time functions that deal with types as a
construct, you can leave decisions like type erasure and reification to the
programmer. Granted, that essentially forces programmers to act as language
designers. And I doubt most programmers want to learn stuff like type rules or
how to prove soundness.

However it does have some really cool implications. Like right now, Ruby has
this whole debate over adding types. And sure, Ruby could just add normal
static typing with some annotations and it'd work. But it kind of doesn't make
sense for such a dynamic language to use such a static construct. What'd make
more sense (to me) is some sort of metaprogramming on code that allows for
checking invariants. Like being able to load a class, then run it through a
checker at runtime. I've been reading a book on dependent types (The Little
Typer) and the whole idea of types having types with a meta-type of sorts
(universes in dependent types, meta classes in Ruby) seems very applicable to
Ruby.

~~~
danharaj
The main issue is that designing a good type system is a dark art.

The more powerful a type system, the easier it is to manipulate terms but the
harder it is to reason about types. That's the fundamental tradeoff.

Being able to specify your own type language is like having an untyped
language governing your term language.

There are promising directions though. In 20 years they might even have
matured.

~~~
YorkshireSeason

       That's the fundamental tradeoff.
    

I'd like to refine this to: _the fundamental tradeoff is between expressivity
of the typing system and computational complexity of type-inference_. Hindley-
Milner's polymorphic typing system "has hit a sweetspot" (B. Pierce) between
expressivity, type-inference. Almost every extension to Hindley-Milner makes
full type inference undecidable.

~~~
danharaj
Type inference is just one part of it. That's taking a term and asking "what
type could this have?" Another thing you can do is turn it around and ask
"what terms have this type?". That question also gets complicated easily.

And that's _principled_ type systems. If you put ad hoc garbage into the types
it might be inferrable and whatnot but still be ridiculous to reason about.

~~~
YorkshireSeason

       what terms have this type?
    

This is a very interesting question, but one that is rarely asked by compilers
or by programmers, or indeed by theoreticians.

I conjecture that the complexity of both questions is related, although I
admit that I'd be hard-pressed even to define properly what the complexity of
"what type could this have?" might be.

~~~
danharaj
Oh, it's more common a question than you suggest. If I am writing a block of
code and I come to some hole in context, for example `foo (bar _) baz`, and I
know what type of thing has to go in that hole, then the very next thing I ask
is "What term do I need there?".

This is a big question for theoreticians working on new languages. The problem
is called canonicity in that context. Here is an example of such a proof of
"What terms have this type [in this context]?":
[https://arxiv.org/abs/1607.04156](https://arxiv.org/abs/1607.04156)

And finally, compilers _will_ ask this question if they're nice to you :) Here
is GHC recently gaining the ability to ask this:
[https://phabricator.haskell.org/D3361](https://phabricator.haskell.org/D3361)

The questions are related by proofs of cut elimination / normalization, which
is the compatibility between a term and its context. Complex types describe
more complex interactions between terms and contexts. Higher rank polymorphism
has undecidable inference because a higher rank type can be used in _very_
complex contexts.

~~~
YorkshireSeason
I know that this question is being asked and reseached -- indeed I do research
in this direction. But it's not well understood, and it's not nearly as widely
asked a question as the type-inference question which the java compiler asks
every time you type something like

    
    
       int n = 2 + 3;
    

Note that 2+3 needs to be type-inferred.

    
    
       cut elimination / normalization,  
    

I'm not sure this can be reduced to cut-elimination, because the question
"What terms have this type?" can be asked for non-terminating programs as well
-- indeed it becomes much more interesting.

    
    
       https://arxiv.org/abs/1607.04156
    

Aside: I'm not following HoTT closely, I guess the problem whether canonicity
of the natural number type holds in HoTT in general (= can be derived from the
axioms of HoTT alone) is still open?

~~~
danharaj
> I'm not sure this can be reduced to cut-elimination, because the question
> "What terms have this type?" can be asked for non-terminating programs as
> well -- indeed it becomes much more interesting.

Oh please share some references :) I think in that context what matters is the
progress and preservation parts of cut elimination.

> Aside: I'm not following HoTT closely, I guess the problem whether
> canonicity of the natural number type holds in HoTT in general (= can be
> derived from the axioms of HoTT alone) is still open?

I haven't either, but I think that's the case.

~~~
YorkshireSeason

       share some references 
    

The PBE / IPS (= programming-by-example / inductive program synthesis)
literature cares about this a great deal. The approach to PBE / IPS in the
tradition of [1], essentially seeks a unique representative of each
equivalence class of programs w.r.t. to the usual typed notion of program
equivalence.

Since most interesting examples of PBE / IPS use recursion, [1] goes through a
hierarchy of recursion principles, e.g. map, filter, fold, full recursion. The
order of choosing recursion principles matters a great deal for controlling
overfitting, the _bête noire_ of machine learning.

The basic problem is this: the answer to the question "What terms have this
type?" is almost always an infinite set with a great deal of redundancy. E.g.
2+3 and 3+2 are both an answer to "What terms have type Int?". But both
answers are contextually indistinguishable, so in practise we only one to see
one of them. Since contexual equivalence is typically undecidable there are
limits to how clever you can be in a computer program to avoid redundancies.
Moreover, in PBE / IPS you typically don't want to have just values, you are
interested essentially in _short_ programs inhabiting a type. So cut-
eliminiation, at least as conventionally understood, doesn't quite do the job.

[1] J. K. Feser, S. Chaudhuri, I. Dillig, Synthesizing Data Structure
Transformations from Input-Output Examples.

~~~
danharaj
Thanks! This looks like a really interesting part of PL theory that I haven't
looked at before. The problem you describe immediately reminds me of Equality
Saturation; I wonder if it's been applied in this space:
[http://www.cs.cornell.edu/~ross/publications/eqsat/](http://www.cs.cornell.edu/~ross/publications/eqsat/)

~~~
YorkshireSeason
That's interesting, although from scanning the abstract, I don't immediately
see the connection between ES and PBE.

~~~
danharaj
Well the common technical problem is that of representing program fragments
along with their equalities so that they can be utilized in the case I linked,
and pruned in the case you suggested. It's just a musing though. I look
forward to reading up on PBE in depth.

------
whateveracct
Haskell has shown that you can have erased generics and have reified generics
opt-in (i.e. Typeable and Generic). Works perfectly.

And the vast majority of generic Haskell code does _not_ have either of those
constraints, which makes you think how important reified generics as-default
really is.

~~~
masklinn
> Haskell has shown that you can have erased generics and have reified
> generics opt-in (i.e. Typeable and Generic). Works perfectly.

Haskell has little to no RTTI, which makes erased generic a non-issue.
Java/C#/Dart have _a lot_ of RTTI, which makes erased generics problematic.

Without RTTI/reflection capabilities, generics reification become a
performance tradeoff, with it they become a logic/correctness issue.

~~~
int_19h
It's not just RTTI. There are some idioms in OOP languages that legitimately
ought to apply to generics, but don't if you erase.

For a fairly common example, a class implementing several specializations of
the same generic interface. Suppose we have Convertible<T> for something that
can make a T out of itself - it would be perfectly reasonable for an object to
be both Convertible<int> and Convertible<string>, but you can't do that in
Java. And this doesn't require RTTI checks at call site at all - it might be
that the caller is itself generic, and is just trying to accept something
that's Convertible<U> for its own U.

~~~
bunderbunder
It also happens with method overloading. In Java, you can't have a class that
has both of these methods at the same time:

    
    
      Integer sum(List<Integer> values) { ... }
    
      float sum(List<Float> values) { ... }
    

because, as far as Java is concerned, both of these methods have exactly the
same argument list.

You also have no way in the language to express things like "Map<String,
Option<Integer>>.class", which has some ergonomic implementations. (e.g., see:
[http://gafter.blogspot.com/2006/12/super-type-
tokens.html](http://gafter.blogspot.com/2006/12/super-type-tokens.html))

The gotcha here, though, is that Haskell doesn't have these problems, or the
one with implementing multiple specializations of the same generic type, even
though it erases types much more completely than Java does.

I'm not much of an expert on language design, but that leaves me thinking that
there's probably a lot of stuff we lay at the feet of Java's type erasure that
is actually more properly attributable to having a type system that is broken
in some fundamental way.

Or maybe it's the way that type erasure interacts with dynamic dispatch in an
object-oriented language? Since the dynamic dispatch means that there are
decisions about datatypes that a language like Haskell or ML can sort out at
compile time, but that an OO language like Java must delay until run-time?

~~~
int_19h
In this case I suspect Haskell just passes the appropriate "view" of the
value, since it knows what the target type class is at any given point, and -
most importantly - it can't be "downcast" later through that same value; you
can't do a _runtime_ check for whether a given Convertible<T> is also
Convertible<U> (either explicitly, or observable via dynamic dispatch). In
Java, it's possible, so even if the code never actually does that anywhere, it
always has to be pessimistic and assume that someone somewhere might.

But I'm not sure that truly counts as type erasure. If that's how things are
done, then the type class of every such value is effectively encoded by the
set of function values that are passed around, and could be queried at runtime
- the Java equivalent would be to tag every reference with its class,
including all generic parameters used to instantiate it.

------
nicktelford
I haven't seen anyone mention it yet, so I'll just point out that the kind of
userland reification described in this article is also possible in Scala using
implicits:

    
    
        class List[T](implicit tag: ClassTag[T]) {
          def elementType: Class[_] = tag.runtimeClass
        }
    

You can do something similar for your own types in Java, albeit with less help
from the compiler:

    
    
        public class List<T> {
            private final Class<T> t;
    
            public List(final Class<T> t) {
                this.t = t;
            }
            
            public Class<T> elementType() {
                return this.t;
            }
        }
    

The problem is that you rarely own the definition of all the types you
want/need to reify.

------
skybrian
It's not directly related, but this reminds me a bit of Zig's treatment of
compile-time parameters [1], which seems somewhat more natural than generic
parameters in most languages.

Zig doesn't go all the way, though. Instead, a type parameter could be any
combination of a compile-time and runtime parameter, depending how it's used.

I do wonder how that affects calling conventions, though? If have separate
linking then we probably need to avoid changing the calling convention if a
function is modified so that a parameter is used in a different way.

[1] [https://andrewkelley.me/post/zig-programming-language-
blurs-...](https://andrewkelley.me/post/zig-programming-language-blurs-line-
compile-time-run-time.html#compile-time-parameters)

------
Groxx
So I totally buy the argument that you can reify your own generics with a
system like this. It's nice, I like it.

...but I don't see how this feeds into the compiler without a "shadow type
system"[1]. If you have to specify the types everywhere, or don't have
compile-time checks, it's not really generics. It's just an Object-container
that has validation funcs executed at runtime, which can already be done
relatively easily in basically every language. Did I miss something?

[1]: or compile-time code generation, but that isn't mentioned. and that
strikes me more as a macro system than a type system (though I don't know if
that's a real distinction, if your macros are sufficiently powerful™)

~~~
chriswarbo
> If you have to specify the types everywhere, or don't have compile-time
> checks, it's not really generics.

Specify which types everywhere? If you mean specifying the concrete types
(e.g. `Integer` and `List[Integer]`, then I'd agree. If you mean type
variables (e.g. `T` and `List[T]`) then I disagree: that's a perfectly valid
way to do generics, and having to annotate stuff is just a deficiency of
inference (no worse than, say, Java).

The key is that we're allowed to use the argument names and function/method
names in our types, thanks to the 2 given rules. We don't need to lift those
functions up to the type level, since we're only checking that the inputs and
outputs line up.

To me, this seems to be a form of dependent types. Note that "dependent types"
only refers to the ability of having types refer to values, e.g. dependent
function types like `(n : Int) -> Foo n` or dependent pairs like `(n : Int,
Bar n)`. Most dependently typed languages _also_ treat types as first-class
language constructs, allow the type-checker to run arbitrary code (which
usually implies that computations are pure and everything
terminates/coterminates). Those latter aspects don't seem to apply here: it's
just about using value-level names in our types.

I think this works quite nicely, from a theoretical point of view; but I
wonder what would happen in more complicated situations. When type checking is
purely syntactic, we can easily get stuck with mismatched things like "should
have type 'T', got type 'Pair[T, U].first()'". That's where we need type-level
computation, to reduce things down to normal forms so that the syntactic
checks (are more likely to) work.

~~~
Groxx
> _If you mean specifying the concrete types (e.g. `Integer` and
> `List[Integer]`, then I 'd agree._

Yep, that's exactly what I mean (since that's the only example in the article
that seems to imply any kind of compile-time safety).

re dependent types: interesting! yeah, it does seem like normal generics could
be a type-dependent type in a case like this - I hadn't really thought about
that before. though that also seems like a "shadow type" tho, which the
article is trying to eliminate... so we're back at the starting point :|

~~~
chriswarbo
> that's the only example in the article that seems to imply any kind of
> compile-time safety

The type-checked example is this:

    
    
        var lst : List[Integer] := List(Integer).new();
        var i : Integer := lst.add(3) + 4;
    

To me, this isn't so different from something like Java:

    
    
        List<Integer> lst = new List();
        Integer i = lst.add(3) + 4;
    

The major difference is the extra argument in `List(Integer)` (compared to
`List()`). That's kind of unfortunate, but note that it's a _value_ , not an
annotation. It's still generic, since we could just as easily pass in some
variable (e.g. `T2`) that we've received as an argument, or pulled out of some
other method, etc.

This extra argument (a run-time representation of the type) is actually to be
expected. Values (including functions) which are generic (in the Java sense),
also called "parametrically polymorphic", or as you say "type-dependent
types", can be understood as taking those parameters as extra arguments (which
is precisely what this article is talking about); although the value itself
_cannot_ change its behaviour depending on that argument. For example,
`List<T> reverse(List<T> input)` cannot work different for, say,
`List<Boolean>` compared to `List<Integer>`.

This is really clear in Haskell. The Glasgow Haskell Compiler uses an
intermediate language called "GHC Core", where generic values are literally
represented in the same way as functions. There's a nice overview at
[https://alpmestan.com/posts/2013-06-27-ghc-core-by-
example-e...](https://alpmestan.com/posts/2013-06-27-ghc-core-by-example-
episode-1.html) (the "id'" example is the first generic one).

> it does seem like normal generics could be a type-dependent type in a case
> like this

Generics are definitely type-dependent types. However, when I said "dependent
types" I was referring specifically to value-dependent types, e.g.
[https://en.wikipedia.org/wiki/Dependent_type](https://en.wikipedia.org/wiki/Dependent_type)

Note in that Wikipedia article's introduction "dependent types are used to
encode logic's quantifiers like 'for all'". Compare that to the generic
example in that GHC Core link above, where the keyword "forall" gets inserted
by the compiler ;)

The Wikipedia article also states "Deciding the equality of dependent types in
a program may require computations". That's what I was getting at with the
system described in this article: it doesn't seem to allow for type-level
computations, without which we may end up getting errors that "expected type
X, given type Y" when those types are 'obviously' the same (to us, but not the
the checker, since it's not able to compute).

I know Bracha is a fan of 'pluggable type systems', where we can use a variety
of type systems to check things about the same program (perhaps requiring
different annotations). However, I'm not sure if these 2 rules are enough to
perform meaningful reductions on types (e.g. to spot that `Pair(T1,
T2).secondType()` is equal to `T2`). I'd imagine we'd have to "lift" more and
more expressions into the type level (e.g. `return`, then `if/then/else`, and
so on) until we have a full-blown theorem prover like Agda!

~~~
chriswarbo
Reading this back, I'd also suggest a comparison with Hindley-Milner type
systems (
[https://en.wikipedia.org/wiki/Hindley%E2%80%93Milner_type_sy...](https://en.wikipedia.org/wiki/Hindley%E2%80%93Milner_type_system)
)

These type systems are used by ML and its descendents (OCaml, Haskell, Rust,
Reason, etc.) and are generally seen as a "sweet spot". They feature:

\- Types depending on types (AKA type constructors)

\- Values depending on types (e.g. type classes)

\- Type-generic values (AKA parametric polymorphism)

\- Higher-kinded types (AKA higher-order types, or generics at the type-level)

\- Inference of all types

\- Erasure of all types

Things which Hindley-Milner _can 't_ do include:

\- Inference of values (AKA implicit arguments, as found in Scala and Idris)

\- Erasure of values

\- Types which depend on values (AKA dependent types)

These things are important for this article, since it represents types as
values (which the two given rules let us access from our type annotations). In
particular, if we could infer (some) values, we could avoid having to specify
arguments like in `List(Integer)`.

Going "beyond" Hindley-Milner is a very active area of research. There's a
nice talk about this at
[https://www.reddit.com/r/haskell/comments/390dyx/conor_mcbri...](https://www.reddit.com/r/haskell/comments/390dyx/conor_mcbride_type_inference_needs_revolution/)
(which i admit is a bit symbol-heavy); full disclosure, I was in the audience
for that talk ;)

~~~
Groxx
Thanks for all the info! I definitely do want to learn more about this kind of
thing, it's always interesting and I'm always glad for more links :)

To circle all the way back around tho, from the intro to the article:

> _A while back, I discussed the problem of shadow language constructs. I gave
> examples of shadow constructs such as Standard ML modules, traditional
> imports etc. Here is another: reified generics._

The article seems to be about _trying to eliminate_ shadow types, e.g. meta-
type info carried by the compiler but not otherwise visible. By turning
generics into type-generator funcs. My question / confusion is around "it
seems like you can't type-check that at compile-time without being either so
explicit it's no longer generic, or with shadow types, so I don't see how this
helps". Or indeed changes anything.

Generics as type-generating funcs makes sense, it lets you choose erasure-or-
not however you like. Perfectly on board[1]. But how do you type-check it at
compile-time without "shadow types"? (if the answer is "read my comments", I'm
happy to! I haven't gone through them super carefully yet.)

[1]: well... in concept. I'm not sure how libraries would be able to use this
in a standard way, seems like it'd lead to multiple competing "generics
systems". But that's also useful sometimes. I like languages that let you do
that kind of thing, though the ecosystem does not always benefit from it.

~~~
chriswarbo
> Thanks for all the info! I definitely do want to learn more about this kind
> of thing, it's always interesting and I'm always glad for more links :)

No problem. Most of my blog posts start out as rambling HN comments ;)

> how do you type-check it at compile-time without "shadow types"?

From my understanding of the article, the 2 rules allow type checkers to refer
to the value names. To me, the key parts of the (typed) example are these
lines:

    
    
        public var List = (function(T : Class) {  
          ...
            public datum : T;
            ...
            public add:(e : T) : T {
            ...
    

In the expression (type annotation) `T : Class`, we're introducing a function
argument called `T`. `T` refers to a _value_ , which we can tell because it's
on the left-hand-side of the type annotation (`Class` is on the right-hand-
side, so we know `Class` is a type).

In the other two lines, `T` is being used as a _type_ (again, since it's on
the right-hand-side of a type annotation). This is what the first "rule" lets
us do.

As far as I can tell, this (along with the second "rule", which lets us refer
to method names in our types) avoids the need for "shadow types" (or a "shadow
language" for types), since we only introduce/define/pass-around things using
the normal value-level machinery (functions, methods, classes, etc.), and our
type annotations simply _refer to_ those value-level names (we don't re-
implement functions, methods, classes, etc. for types).

My concern is that "rule 2" lets us write things like:

    
    
        function pickAType(T1 : Class, T2 : Class) : Class {
          if (canProve(collatzConjecture)) {
            return T1;
          else {
            return T2;
          }
        }
    
        var x : pickAType(Int, String) = 42;
    

This is well-typed if and only if the Collatz conjecture is provable (e.g. by
enumerating all proofs in some formal logic and checking whether any is a
proof or disproof of collatzConjecture). If it's provable, then
`pickAType(Int, String)` reduces to `Int`, which matches the type of `42`. If
it's not provable then `pickAType(Int, String)` reduces to `String`, which
doesn't match the type of `42`.

This is just an example: we can replace "prove Collatz conjecture" with any
arbitrarily-difficult calculation. The point is that _checking_ these types
requires _running_ the function which, in this case, requires the type checker
to understand `if/then/else`, functions like `canProve` which don't fit the
constraints of "rule 2" (that inputs and output should be `Class`); presumably
`canProve` will use things like recursion, and `collatzConjecture` will be
some sort of datastructure, which a type checker would _also_ need to
understand. In short, we'd need an awful lot more rules than just the 2 in the
article. The straightforward way to do this is to just have our type checker
just invoke the normal language interpreter, but that raises more issues.

For example, side-effects: what if `pickAType` chooses its output based on,
say, reading a random number from the OS kernel? It seems unwise to allow
side-effects when type-checking (except maybe for "type providers"
[https://docs.microsoft.com/en-
us/dotnet/fsharp/tutorials/typ...](https://docs.microsoft.com/en-
us/dotnet/fsharp/tutorials/type-providers) ). However, if we're running
arbitrary value-level code as part of the type-checking process (e.g. by
invoking the normal interpreter), that would require the value-level language
to _also_ be free of side-effects (or else we would need an alternative
interpreter, which raises the "shadow language" problem again).

In particular, object oriented systems, especially those like Smalltalk, Self
and the author's Newspeak, model function calls, property lookup, value
construction, method calls, etc. as 'sending messages between objects', which
is an I/O effect that can have all sorts of repercussions for the meaning of
other code. For example, imagine the method call `True become: False` (
[https://gbracha.blogspot.com/2009/07/miracle-of-
become.html](https://gbracha.blogspot.com/2009/07/miracle-of-become.html) ; or
in Newspeak, where booleans are immutable "value objects", some equivalent
behaviour-changing call). Even in a system with no I/O, I think this would
make types non-compositional, i.e. if the expression `f` type-checks to type
`A -> B` and `x` type-checks to type `A`, the expression `f(x)` might not
type-check, and even if it does it might not have type `B`, since the code run
while type-checking one sub-expression may alter the behaviour of code run
while type-checking the other.

Note that these aren't insurmountable problems: we could approach it like most
other things in dynamic languages and say "yes it can be broken by doing that;
so don't do that".

Alternatively, perhaps more in keeping with Bracha's ideas of pluggable type
systems, we might be OK with type systems that are limited (e.g. unable to
reduce `pickAType(Int, String)` to either `Int` or `String`), or uncomputable
(if we _do_ run `pickAType`), or even containing "shadow" interpreters which
may, for example, over- or under-approximate types (e.g. allowing `42` since
it _might_ have type `pickAType(Int, String)`, or disallowing it because it
_might not_ ), or which implement some alternative evaluation method or effect
system (e.g. calling out to SMT solvers or proof assistants, mocking I/O
calls, etc.). If they're pluggable, then we're free to ignore those which
aren't appropriate for our project (based on how much confidence we want in
the result, whether we'll be doing shenanigans like `pickAType`, etc.)

------
zmmmmm
This sounds very academic. Meantime in the real world, people in Java-land are
constantly killing themselves trying to invent workarounds for type erasure
induced problems. The latest example I know of being Micronaut [1] which touts
itself in some sense as being a whole platform for re-reification.

[1]
[https://objectcomputing.com/news/2018/10/08/micronaut-10-rc2](https://objectcomputing.com/news/2018/10/08/micronaut-10-rc2)

~~~
AstralStorm
This is because the decision to type erase was very weird for a strong static
typed language...

You inject weak typing (unexpected conversations) into a strong type system by
doing that.

~~~
stewbrew
"Weird" is a strong word. Once the compiler has proven the codes satisfies
certain type constraints you usually don't need the type information anymore
... except when you do of course.

