
Where Do Type Systems Come From? - philix001
http://blog.felipe.rs/2017/07/07/where-do-type-systems-come-from/
======
a-nikolaev
Watch Oregon Programming Language School lectures "Basic Proof Theory" by
Frank Pfenning.

[https://www.cs.uoregon.edu/research/summerschool/summer15/cu...](https://www.cs.uoregon.edu/research/summerschool/summer15/curriculum.html)

Very clean and easy to follow video lectures on the relation between, types,
programs, and logical proofs. One does not need functors and monoids to
appreciate the beauty of functional type systems. (And to see why such type
systems are indeed discovered rather than invented.)

~~~
z1mm32m4n
I second this. Frank's material is always thorough and approachable. He's
designed and taught many courses at CMU, and his lecture notes for them are
never less than impeccable.

------
dvt
It's a common misconception that Russell/Whitehead "invented" type theory. In
fact, Frege had already made the very insightful distinction between
functional and non-functional types in the 1890s -- _this_ was the key
development that Russell based his hierarchy of types on. See "Function and
Concept" (1891)[1]. It was a growing and communal sentiment that a
(meta-)theory of types would make certain mathematical concepts more
palatable.

If anything, I think the conceptual father of type theory is Gottlob Frege,
and Alonzo Church was the first to apply it concretely.

[1]
[http://fitelson.org/proseminar/frege_fac.pdf](http://fitelson.org/proseminar/frege_fac.pdf)

~~~
rntz
It seems to me Frege understood the need in mathematics to talk about many
different kinds of thing - numbers, truth-values, functions, and so forth -
and to distinguish which kind of thing you are talking about; but not the
necessity to use types (or other methods) to avoid circularity. Indeed, it is
precisely the mistake Frege made in his attempt to axiomatise logic and
mathematics and which led to Russell's paradox, that motivated Russell (and
Whitehead)'s theories of types.

Frege certainly articulated a clear notion of what a function _is_ , which is
significant.

------
simplify
If you haven't thought much about type systems but want to understand what the
big deal is, I wrote a post specifically for you [1]. It draws motivation for
wanting a good static type system from first principles.

[1] [https://gilbert.ghost.io/type-systems-for-beginners-an-
intro...](https://gilbert.ghost.io/type-systems-for-beginners-an-
introduction/)

------
georgewsinger
Nice article. I especially liked:

> program3 fails because runFunction can only run first-order functions and
> runFunction is a second-order function – a function that takes a first-order
> function as a parameter.

Here I had no idea that JavaScript implicitly typed `runFunction` that way.
That's cool.

Also, I never thought of "higher-order functions" as breaking into a countable
hierarchy of nth-order functions, which is an interesting thought. In Haskell
the hierarchy would start off as

    
    
        zerothOrderFunction :: a
        firstOrderFunction :: a -> b
        SecondOrderFunction :: a -> (b -> c)
        SecondOrderFunction' :: (a -> b) -> c
    

In general, an nth-order function is a function which either

1\. Takes an (n-1)th order function as an input and returns a simple type.

2\. Takes a simple type as an input and returns an (n-1)th order function as
an output.

The upshot is that typed programming languages not only catch bugs, but
prevent you from legally expressing many non-sensical expressions (analogous
to the set theory paradoxes). For example, consider the expression

    
    
        (\x -> x x) (\x -> x x)
    

If you expand this expression, it reduces to itself:

    
    
        (\x -> x x) (\x -> x x) == (\x -> x x) (\x -> x x)
    

In a purely untyped language, this expression would be legal. But what would
be its meaning? Arguably, it is non-sense, and should be excluded from the set
of legally expressible expressions. The way to do this is through a type
system. And indeed, if you typed this statement into a Haskell REPL you would
get a type error; this statement can actually be proven to be untypable
(IIRC).

On the other hand, this means that typed systems are in some sense _strictly
less expressive than their untyped counterparts_. It would therefore be
interesting if somebody found an expression which was both (i) meaningful and
(ii) _only_ expressible in an untyped language. You would then have an
argument for untyped languages :-)

~~~
kmill
> > program3 fails because runFunction can only run first-order functions and
> runFunction is a second-order function – a function that takes a first-order
> function as a parameter. > Here I had no idea that JavaScript implicitly
> typed `runFunction` that way. That's cool.

In case it's not clear, it's just that it eventually ends up with a run time
error (the talk about runFunction being a second-order function is just an
explanation for why you should expect it to end up with a run time error). The
evaluation is

    
    
           program3()
        == runFunction(runFunction)
        == runFunction(1)
        == 1(1)
        == Error: func (with value 1) is not a function

~~~
oldandtired
Depending on your system, 1(1) is quite sensible, That is applying 1 to a list
of parameters just returns the 1st parameter. This is a matter of the
semantics that one uses when application is run against any value.

1(1) -> 1

Just because the common idea of application is that it only applies to
functions does not mean that application applied to other types is nonsensical
and should end up with a runtime error.

If your language allows application to be defined for different types, then
the type system should be capable of determining the type that returns from
application.

A practical example of a language in which application is valid for integers
is Unicon/Icon. Both functions and integers have specific semantics with
regards to application. And failure is an option.

------
bogomipz
What a great piece!

I wish the author would expand on this piece with either more installments or
a even short book.

I find myself interested in type systems as it relates to programming language
design but I haven't found much middle ground between the basic types
described in introductory texts about a language and the opposite extreme
heavy academic texts such as the ones the author is breaking down in this
article.

Can anyone recommend any other such middle ground resources on type systems
and type system theory?

~~~
naasking
What do you consider basic? Algorithm W for ML type inference is pretty basic,
but powerful too. Or are you looking for something even more expressive?

~~~
bogomipz
What I meant by basic was the description of types provided by a language -
usually in an introductory text you might read when learning a new language. I
probably didn't articulate that correctly.

But I guess what I was referring to as a "middle ground"qa any resources for
learning about types systems written in a similar approachable tone like this
article.

This was another article I read recently that I thought was similarly
accessible on the subject of types systems:

[https://medium.com/@thejameskyle/type-systems-structural-
vs-...](https://medium.com/@thejameskyle/type-systems-structural-vs-nominal-
typing-explained-56511dd969f4)

So I guess I'm wondering if there exists such a book or series that might
allow one to further their knowledge of type systems without requiring
university study.

~~~
naasking
Types and Programming Languages is the go to book, and it's very accessible
despite being a textbook:

[https://mitpress.mit.edu/books/types-and-programming-
languag...](https://mitpress.mit.edu/books/types-and-programming-languages)

You can find some earlier PDF drafts online if you Google.

~~~
alexashka
I have the book and I must warn people, it is not THAT accessible - this is
not evening reading for an hour, this is work.

~~~
naasking
You'll have to define accessible then. Learning anything takes work.

------
Animats
There's another approach, from Boyer and Moore. Boyer and Moore built up
mathematics from constructs at the Peano axiom level (zero, add1, etc.) plus
recursive functions that must terminate. It's constructive mathematics; there
are no quantifiers, no ∀ or ∃. [1] They built an automatic theorem prover in
the 1970s and 1980s that works on this theory. (I recently made it work on Gnu
Common LISP and put it on Github, so people can run it again.)[2]

In Boyer-Moore theory, all functions are total - you can apply any function to
any object. Types are predicates. Here's a definition of ORDERED for a list of
number:

    
    
        (DEFN ORDERED (L)
          (IF (LISTP L)
              (IF (LISTP (CDR L))
                  (IF (LESSP (CADR L)
                             (CAR L))
                      F
                      (ORDERED (CDR L)))
                  T)
              T))
    

If L is not a list, it is considered to be ordered. This makes it a total
function, runnable on any input, even though the result for the "wrong" type
is not useful. This provides the benefits of types without requiring a theory
of types. It's a very clean way to look at the foundations of mathematics.
It's simpler than Russell and Whitehead.

When you prove things using definitions like this, there's a lot of case
analysis. This gets worse combinatorially; a theorem with four variables with
type constraints will generate at least 2⁴ cases, only one of which is
interesting. Most of the cases, such as when L is not a list, are trivial, but
have to be analyzed. Computers are great at this, and the Boyer-Moore prover
deals with those cases without any trouble. But it went against a long
tradition in mathematics of avoiding case analysis. That made this approach
unpopular with the generation of pre-computer mathematicians. Today, it would
be more acceptable.

(It's fun to run the Boyer-Moore prover today. In the 1980s, it took 45
minutes to grind through the basic theory of numbers. Now it takes a few
seconds.)

[1] [https://www.amazon.com/Computational-Logic-Robert-S-
Boyer/dp...](https://www.amazon.com/Computational-Logic-Robert-S-
Boyer/dp/1483236528) [2] [https://github.com/John-
Nagle/nqthm](https://github.com/John-Nagle/nqthm)

~~~
kazinator
If every operation can have any type arguments applied to it and does
something sensible with no compiler or run-time error ... good luck debugging,
surely.

What happens with OOP? Every class has to understand how to "bark", not only
the dog class?

If any class can somehow "bark" without throwing an exception, that may not be
in alignment with the programmer's intent, or promote the furtherance of his
or her goals in any way.

For intance, the intent may be that the programmer wanted to ask the local
variable _dog_ to "bark", but misspelled it as _ndog_ and the integer which
counts the number of dogs was asked to bark instead.

There is much value in identifying the problem that an integer doesn't bark.

~~~
Animats
Boyer-Moore theory has "shells", which are like structures. See page 39 of
[1]. Since this is a pure functional language, values cannot be altered.
Shells have constructors, a type predicate, and can have restriction
predicates on values.

    
    
        Shell Definition.
        Add the shell ADD1 of one argument
        with bottom object (ZERO),
        recognizer NUMBERP,
        accessor SUB1,
        type restriction (NUMBERP X1),
        default value (ZERO), and
        well-founded relation SUB1P.
    

It's not intended that you run programs in Boyer-Moore theory, although you
can. It's a proof tool.

[1]
[https://www.cs.utexas.edu/users/boyer/acl.pdf](https://www.cs.utexas.edu/users/boyer/acl.pdf)

~~~
kazinator
I will have to read this to understand what we can prove with this; or rather,
what kinds of wrongs in a program under this theory are usefully proved to be
false.

\---

Ouch; did you see that "overfull hbox" that got rendered out in the first line
of paragraph 3 of the Preface? :)

------
pier25
Reminded me of Gödel's incompleteness theorems.

First incompleteness theorem

Any consistent formal system F within which a certain amount of elementary
arithmetic can be carried out is incomplete; i.e., there are statements of the
language of F which can neither be proved nor disproved in F.

Second incompleteness theorem

For any consistent system F within which a certain amount of elementary
arithmetic can be carried out, the consistency of F cannot be proved in F
itself.

~~~
igravious
Honest question. What in the article prompted you to think about Gödel and his
theorems? Why were you reminded?

~~~
munificent
I'm not sure why pier25 was reminded, but Russell's type theory and Gödel's
incompleteness theorem are closely related. They both arose in response to the
foundational crisis in mathematics [1].

Russell stumbled onto Russell's paradox (among others) and it shook
mathematicians' confidence that everything in math was built on top of a
perfectly consistent and stable foundation. If you can define a set that it
"the set of sets that don't contain themself" then what other kind of crazy
talk can you say in math? How do you know proven things are true and false
things can't be proven in the face of weirdness like that?

Russell tried to solve the problem by inventing type theory. Types stratify
the universe of values such that "the set of sets that don't contain themself"
is no longer a valid statement to make.

Meanwhile, Gödel went and proved that, sorry, no, math is _not_ consistent and
complete. There are statements that are true but which cannot be proven.

[1]:
[https://en.wikipedia.org/wiki/Foundations_of_mathematics#Fou...](https://en.wikipedia.org/wiki/Foundations_of_mathematics#Foundational_crisis)

------
incan1275
I share the author's frustration with wikipedia sometimes - people usually go
to wikipedia for a distilled, comprehensible description of the subject
matter. What he quoted was certainly not comprehensible, even to someone well-
educated in CS foundations.

~~~
owebmaster
I like it because of it. If what I found in wikipedia was the "easiest"
description, I'd find it lacking. It is better to not understand everything on
the first read than understanding almost nothing because of lack of
profoundity.

------
threepipeproblm
I loved this because I have read most of the source material in the context of
logic, but never made the lead to type theory in computer science.

------
agumonkey
Types are close to adjoint functors / adjunctions and partial evaluation.
Assigning restricted information to part of a structure to gain knowledge
through limitation (math).

~~~
igravious
The category-theory window onto the world of types only appeals to a small
subset of human minds.

For the average programmer you may as well be spouting gibberish because the
average programmer will have no way to evaluate the claims (if any) you are
making. Note, I am saying that you _may as well be_ and not that you _are_.
Please do not misunderstand me.

Types systems certainly are formal theoretical systems but I personally have
come to believe that the majority of coders are ill-served by the mathematical
leanings of type theorists.

I'm not sure I can explain myself better than that at the moment.

~~~
agumonkey
Do not worry I understand 99% of your message. It's indeed a land far far away
from the everyday coding of the majority of programmers. Unless they start
digging, which I did. If you take code as data (lisp roots showing) you start
to want to reason about it and quickly you end up reading about FP,
denotations, different forms of evaluations, the value of metadata (type or
else).

Now I believe there's an artificial split between math leaning people and
pragmatics, the former end up as PhD, the latter in IT or close. But in
reality the average coder could understand and even enjoy the land of
abstractions, it's just that the river he swims in isn't flowing there so one
has to run against the flow.

Not to say that ideals are the only-tru-way.

~~~
igravious
I am delighted you responded so positively to what I wrote and didn't take
what I wrote negatively which you could easily have done.

Let's for arguments sake say that there are two camps (broadly speaking), the
pragmatists as you say and the theorists/idealists let's call them. It reminds
me of the difference between someone like Torvalds and someone like Stallman.

Thing is we need both! You're right, the split _is_ artificial. The Linux
kernel couldn't wait for someone to come along and create a type-awesome
version of C. I mean, Rust seems to be the first attempt to take what type
theorists have learned and apply it to a systems programming language. In the
meantime software needs to be written and we have the tools we have.

If the fruits of type theory are going to filter through into software
development I'm not sure it should come laden down with category theory (as
awesome as that is) or the lambda calculus (as mind-bending as that is). I
could of course be dead wrong.

~~~
agumonkey
A guy tried to make a C-level formal language, even sexp based at first. After
years he quit, saying it's probably impossible to have both (he wrote a long
long article about the reasons, he didn't leave without explaining every
problems in details).

Usually ideas filters in tiny bits, kind like genes. See closures, forever in
lisp, but now in every languages while lisp is still niche.

The issue with theorists is that they see the world in abstract algebra /
combinatorics, it's not fluff, it's just extremely high level thinking with
extremely short notation[1]. It looks like straight BS until you spend enough
time seeing that it translates into reality. Say something like prolog, where
a few rules give you graph coloring. It's not theory only, it's actual
application of different semantics.

[1] also, as in any group, they developped a culture and taste for some
things, expressing everything in the form of arithmetic expressions. F^n <=
iteration of F n times, it's a loop in average coder lingo.

~~~
jcranberry
This was fantastically well stated.

------
sgt101
Type Systems come from Russel - yup. But the notion of Type has an interesting
origin in the west as well (I would love to read/understand histories of this
concept from other cultures, but I am ignorant for now).

My reading is that it was invented by Scotus as Haecceity ! This was required
by Catholic Christianity because of the difficulty that The Creed introduces
about the identity of God - there are three entities which represent God, the
Trinity - how to account for this? Well; the thisness of God is joined with
the thisness of man, the thisness of the creator and the thisness of the thing
which is motion (I have never understood The Holy Spirit). You can think of
this as multiple inheritance! Theologians then had to account for "why three"
as you can carry on making aspects of god with this mechanism infinitely, god
the mother, god the lover, god the hunter and so on. But there are three -
why? The answer was provided by Scotus's student Occam, entities should not
multiply beyond necessity and hence there are three aspects of god because it
is necessary for creation that there are.

The fun bit it that this procession of thought is somewhat guessed at because
writing things like this down or debating them publically was a quick route to
the afterlife via a bonfire!

~~~
lists
> The fun bit it that this procession of thought is somewhat guessed at
> because writing things like this down or debating them publically was a
> quick route to the afterlife via a bonfire!

Theatre and Philosophy have always been able to have a lively chat with one
another

------
Ar-Curunir
This article was very well written. It finally clicked for me why "hugher-
order functions" are named the way they are.

Any more articles in this vein?

------
winter_blue
"Even though theoretically, type theories and type systems are not enough to
prevent all the problems in logic and programming, they can be improved and
refined to prevent an increasingly number of problems in the practice of logic
and programming. That means the research for better practical type systems is
far from over!"

This is great point, and I think it is absolutely worthwhile to put time into
researching better, more powerful type systems.

Tony Hoare said[1] that his research into formal systems and his hope that the
pr Framing world would embrace these new innovations that increase safety and
reliability was futile, but I think what we need _is a new approach, with
particular care given to practicality and adoptability_.

[1]
[https://en.wikipedia.org/wiki/Tony_Hoare](https://en.wikipedia.org/wiki/Tony_Hoare)

------
smcl
I feel kinda alone on HN, Lobste.rs and LtU in not having in-depth knowledge
or opinions on type systems. I get that these underpin the technology we as
programmers use every single day, but I'm a little ashamed that I can't get
excited about the subject and feel like it's too late for me to bother trying.

~~~
currymj
There’s the technical aspects of the fact that every language has to have some
notion of “type”. And seemingly interpreted languages might be JIT-compiled
etc. This is of interest if you care about the implementation of languages.

Then there’s the opinions that users of languages with more elaborate,
expressive type systems have, like how some people really enjoy Haskell or Elm
because they feel that the type system helps them express their ideas clearly,
avoid errors, and aids refactoring and maintenance.

If you’re worried about this one, don’t I guess? If you can use dynamic
languages to achieve your goals, and you like them, then that’s fine! There
are plenty of languages you can play with if you want to get a feel for
programming with types. Even Java 8 and C++11 are decent at this point (I’m
sure a Haskell programmer is fuming right now).

Then there’s like, a few thousand people in the world who have well-informed
opinions on research into the theory of programming languages, the Curry-
Howard correspondence between types and proofs. Also a lot of Hacker News
posters who have heard these words. Some of them pretend like they know what
they’re talking about.

~~~
acchow
> There’s the technical aspects of the fact that every language has to have
> some notion of “type”

This isn't true. There are no types in lisp or untyped lambda calculus.

~~~
naasking
Actually, there's one type.

~~~
acchow
If you're going to shove the system into a typed model, then sure there is one
type. But then you've kind of missed the point...

~~~
naasking
If you're going to answer the question of whether a language has types, then
you're already trying to see how it can be shoved into types. It's presupposed
by the question itself.

You literally have to count the types by analyzing the grammar. So for the
lambda calculus, you have lambda=1, halt. So does a single type mean no types
or literally one type?

What's the advantage of thinking that 1 type actually equals 0 types? I don't
see any, so in my mind, all languages are unityped or have a richer type
structure. Whether a richer type structure is desirable is a separate
question.

------
drc0
> That’s the equivalent of writing type annotations for programming functions.
> And the goal is avoiding bugs instead of logical contradictions.

mh, given Curry–Howard correspondence, aren't those the same? so the goal is
indeed not having logical contradictions?

~~~
Sharlin
Yes. But it's a rare programmer, or even a programming language designer, who
thinks of well-typed programs in terms of proving theorems.

------
zzzcpan
"Even though theoretically, type theories and type systems are not enough to
prevent all the problems in logic and programming, they can be improved and
refined to prevent an increasingly number of problems in the practice of logic
and programming."

It's actually just a belief. Nothing suggests that type systems and type
theories can be improved to be practical at preventing bugs. I'd say it's the
opposite, even with as much understanding about the nature of bugs as we have
today, they don't look very promising, unlikely to make it even into the top
ten of other different approaches.

~~~
swsieber
There are type systems that prevent race issues, use after free, and null
pointer exceptions.

So saying that they aren't practical at solving bugs is a little disingenuous.

~~~
AnimalMuppet
Well... how much of actual software is written using those type systems? Less
than 1%? So such type systems might _be able to_ prevent bugs, but in
practice, they _don 't_.

Why aren't they used? Probably existing code bases, inertia, and ignorance
play a role. I suspect, though, that at least part of the problem is that most
programmers find those type systems too hard to use. In that sense, the type
systems aren't _practical_.

And if you're going to blame the programmers for that, well, if your plan for
making programming better requires different programmers than the ones we
have, your plan isn't very practical, either.

~~~
hyperpape
Today less than 1% of all software is written in null safe languages. But I
believe Swift has non-nullable types, and it's the promoted language for a
really big ecosystem. There's also Rust and Scala, but those have less of a
captive audience. I do have high hopes for Rust, which also is data-race safe.

I think Java 8 and optionals show it doesn't have to be that hard, it's just
that there's too much old code that relies on nulls for the Java ecosystem to
ever be fully null safe.

Use-after-free is solved in a language without manual memory management, so
that's actually quite common.

Programmers get comfortable with new ideas over time. Higher order functions
and type inference used to be obscure concepts. Today they're par for the
course. I don't know if we'll all use dependent types some day, but I think
we'll keep getting more powerful types in mainstream languages for a while.

------
sriram_malhar
I found Tomas Petricek's essay on "Against a universal definition of 'Type'"
very informative. He argues that the word 'type' has shifted shape many times
since Frege/Russel, in that the intuition behind them is different. He also
argues that multiplicity of definition is a good thing.

[http://tomasp.net/academic/papers/against-
types/](http://tomasp.net/academic/papers/against-types/)

------
kronos29296
Nice article about type theory.

> Why there’s so much research around types if perfectly applying them to
> programming languages is impractical?

Somehow Haskell does this perfectly. Whaddya say to that?

~~~
coldtea
I say, citation needed. Who said "Haskell does it perfectly"?

Not to mention the mental overhead of Haskell (which is also not optimal).

~~~
aleden
Yeah, what the fuck? Most software engineers just want to get shit done
quickly so they can go home to their wife and kids.

~~~
aleden
If you think this merits a downvote please tell why. Excuse my french.

------
visarga
Types aren't just for programming and philosophy. Strongly Typed Neural
Networks are also a thing.
[https://arxiv.org/abs/1602.02218](https://arxiv.org/abs/1602.02218)

------
visarga
TL;DR - Type theory == being careful about the domain a function can be
applied in (adding meters to seconds or strings to sets should not be
possible).

------
miguelrochefort
Don't they teach this stuff in school?

~~~
0xCMP
The school I went to barely taught C/C++ and the absolute minimum of PHP, CSS,
Linked Lists, and Hash Maps. Very sad that so many actually smart people can't
graduate knowing much just doing their course work. Let alone imagine those
who lack off a bit and still pass. Unless you're programming on your own, like
I was along with a few others, you graduate possibly in debt and completely
unprepared.

So no, they never get in to this stuff.

------
OJFord
Distinctly unimpressed by this post. The author seems to have an axe to grind
with mathematicians as a class, which, as we would have been told in school,
isn't 'big or clever'.

The whole of programming, nevermind types, perhaps the most mathematical part
of modern programming, arises from mathematics. There's some good history
here, but the early paragraphs in particular are a display of ignorance if not
arrogance.

The author quotes Newton, the very chap who's said to have said he merely
stood on the shoulders of giants (to 'see' such insight). Any programmer in
the 21st century stands on the shoulders of mathematicians and computer
scientists of the 20th,; who were in turn standing on the shoulders of the
mathematicians of the 19th centuries.

~~~
bogomipz
>"Distinctly unimpressed by this post. The author seems to have an axe to
grind with mathematicians as a class, which, as we would have been told in
school, isn't 'big or clever'."

I thought it was intended to speak to people who might be intimidated or feel
obtuse when they encounter really dense academic texts when trying to learn
more about type systems as it relates to programming. As such I really
appreciated it.

I didn't think the author was grinding any axes at all, quite the contrary.

~~~
OJFord
I've no problem with that, it was the opening quotation, accompanying graphic,
and following paragraph which read - to me, though I appreciate I may not have
read it as it was intended - quite disrespectfully toward mathematicians.

Among whom I cannot count myself, for whatever it's worth.

