
A Path to Enlightenment in Programming Language Theory - jcurbo
http://steshaw.org/plt/
======
igravious
What is always missing for me in these lists is a tutorial for knocking
together a small lambda calculus + dependently typed interpreter in C. Not ML,
Not Ocaml, not, Haskell, not any language that has _already_ got lambdas in
it!

Always when I read these I see, "get your Haskell compiler and..." and I'm
like, if I knew Haskell well enough to do that I wouldn't be doing this
tutorial in the first place.

Case in point, article says "particularly for programming pracitioners who
didn’t learn it at school." \-- Correct! I taught myself Basic, I taught
myself a bit of x86 assembler, most importantly, I taught myself C. My working
mental model of a computer is the C machine model. And even now, now that I
know Ruby with its lambdas, I know that Ruby is written in C. And the Linux
kernel. And Nginx (C++ maybe? Dunno). And on, and on.

Show me the tutorial where I can build, _in C_ a bare bones working (tail
recursion) implementation of the lambda calculus with type inference and
dependent (algebraic?) types and I'll shake your hand and call you a champion.

Closest I've seen is Make A Lisp -
[https://github.com/kanaka/mal](https://github.com/kanaka/mal) but I don't
think it has the sweet types and tail recursion and other good stuff. PS:
bonus points if Regexen are native type :)

(Open to suggestions (I am))

~~~
exDM69
> knocking together a small lambda calculus + dependently typed interpreter in
> C. Not ML, Not Ocaml, not, Haskell, not any language that is _already_ got
> lambdas in it!

This wouldn't be any more educational and just plain painful. "Research"
programming languages tend to work very well for building compilers, that's
why they are used in many toy programming language projects.

Dealing with tree and graph structures in a language like C would be a lot of
work, a lot of effort going into memory management which is not at all useful
towards the goal of learning how to implement a language.

To give a practical example: last time I was working on a toy compiler using
Haskell, I needed to calculate the strongly connected components in a directed
graph to perform dependency analysis (rearranging "letrecs") that is required
to do type inference. Had I been writing the compiler in C, it would have
required a week's worth of engineering to get done. With Haskell, I could grab
Data.Graph.SCC and be done in a few hours. Arguably I could have picked up a
graph library in C, but then I'd have to spend time understanding how memory
management works in the library and so on.

I understand the appeal in using C or just an imperative programming language
but quite frankly, that would really hurt the pedagogic aspects of a project
like that.

edit: Here's the relevant code if you're interested:
[https://github.com/rikusalminen/funfun/blob/master/FunFun/De...](https://github.com/rikusalminen/funfun/blob/master/FunFun/DependencyAnalysis.hs)

~~~
Silhouette
The flip side of this is that if you're writing something in C, you actually
do have to understand how details like closures are implemented on real
hardware. These are not trivial problems, yet many programming language design
papers and articles effectively brush the whole issue of actually implementing
these things under the carpet.

Go ahead and search for that particular example. There is _almost nothing_ on
the Web about practical, real world techniques for implementing that sort of
programming language feature, despite its ubiquity in functional languages and
the increasing presence of -- or emphasis on -- higher order functions and
related tools in mainstream imperative languages. The same could be said of
many other non-trivial language features, say laziness, or various concurrency
models.

I think this is regrettable, because it creates a real barrier for anyone
who's interested in learning about these features and maybe writing their own
languages one day, but who doesn't come from an academic background where this
kind of material was taught. It also makes it unnecessarily difficult for
someone from an imperative programming background -- which is probably still
the overwhelming majority of programmers -- to understand the real costs and
performance implications of using these kinds of features.

~~~
exDM69
> The flip side of this is that if you're writing something in C, you actually
> do have to understand how details like closures are implemented on real
> hardware.

If you're writing a compiler targeting real hardware and not a high level
virtual machine, you'll need to learn how to implement closures regardless of
the "host" language you write your compiler in.

But you're correct, when implementing an interpreter, it's so much easier if
you have a garbage collector and an object system of some kind at hand. You'll
have to implement this yourself if you're working with C.

~~~
Silhouette
_If you 're writing a compiler targeting real hardware and not a high level
virtual machine, you'll need to learn how to implement closures regardless of
the "host" language you write your compiler in._

Right, but a lot of the tutorial material in this area doesn't really consider
that issue, even though it's rather fundamental to practical applications.
Instead the explanation often settles for some sort of indirect dependency on
the implementation language's existing version of a similar feature. I think
this is unfortunate, not least because it reduces us to a model where most
people implementing these ideas depend on some sort of magic run-time
environment instead of learning how to actually write that run-time.

~~~
ufo
Thats because tutorial material is going to keep things simple. If you really
want to implement a garbage collector on your own then you should go looking
for an advanced compiler book instead of a tutorial.

------
jcurbo
I spotted this on Twitter [0] and it was timely for me because I've been sorta
building my own learning path lately. After learning Haskell a few years ago I
got really interested in PLT, type theory, and other higher math (like CT) and
also things like formal program verification and automated solvers. So I
started trying to decide on a good self-study path (as a guy with a day job
who's not going to be starting a Ph.D. anytime soon). I ran across this very
helpful Reddit thread [1] and so far I'm thinking of the following path. I'm
starting with _How to Prove It_ , because I never did a lot of proof solving
in college (and that was >10 yrs ago anyway) and it's a good basis for further
work. Then I have two branches.

The first is for abstract algebra and CT, because I find them interesting and
they're useful for understanding Haskell, that currently consists of Pinter's
_A Book of Abstract Algebra_ , then _Conceptual Mathematics_ by Lawvere and
Schanuel, then Awodey's CT book, which is a path straight out of that Reddit
thread.

The second is for PLT/type theory, and starts with _Software Foundations_ by
Pierce, then _Types and Programming Languages_ by Pierce, then _Practical
Foundations for Programming Languages_ by Harper (for completeness, as I have
heard pros and cons for both), then maybe moving to _Advanced Topics in Types
and Programming Languages_ and such. Mix in some Oregon Programming Language
Summer School (OPLSS) [2] videos here and there when they fit. (OPLSS sounds
great and I'm glad their videos are online)

[0]
[https://twitter.com/manisha72617183/status/61301658769664819...](https://twitter.com/manisha72617183/status/613016587696648192)

[1]
[http://www.reddit.com/r/haskell/comments/29reb7/first_or_sec...](http://www.reddit.com/r/haskell/comments/29reb7/first_or_second_edition_of_introduction_to/)

[2]
[http://www.cs.uoregon.edu/research/summerschool/](http://www.cs.uoregon.edu/research/summerschool/)

~~~
groovy2shoes
Pinter's _A Book of Abstract Algebra_ is one of the best math books I've ever
read. It's short, to the point, and doesn't assume you're an idiot. I learned
a lot from it and I can't recommend it enough.

------
javajosh
I'd like to find a jargon-free path. There's too much gravitas, and hence ego,
in the words.

It's kind of like astronomy - you can (and should) start with looking up, and
notice that the stars rotate around the north star. Terms like right
ascension, azimuth, celestial spheres, plane of the ecliptic, etc...these are
impressive and important sounding and they really get in the way of
understanding.

It's not arbitrary picking astronomy as my example, because in the end
_computers are physical systems_. They are quite unique because they are
capable of actions that look like violations of the second law - e.g. you can
quite easily reverse a program that simulates gas diffusion, and simulate
putting that gas back in a jar - that is, you can define an incredibly
delicate system, and reset it for nothing, like being able to use a kitchen
for anything and then clean it instantly, for free. More generally, all of a
computers negentropy can be recovered for little or no cost after arbitrary
disorder! They are also unique because they can rapidly and accurately and
repeatedly traverse very specific state spaces, which is unlike anything in
the universe, including the human brain.

As a physical system, a computer extends through time, and it's state is in
general time dependent. Coupling signals across time is, I believe, what
programming is about. We have special names (jargon!) for different signals -
we call signals that come earlier "programming" and signals that come later
"runtime input". But only useful distinction is between signals that arise
from outside the system, and those which arise inside (that is, _programming_
and _input_ are the same; but input arising from _results of a function call_
are profoundly different). It's useful because only the latter can recurse,
which is both a source of error, but also the only possible way "intelligence"
(or any really complicated behavior) could possibly express. The only other
thing that matters is the shape of the system state - which constrains, in
some ways, the kinds of states the system can achieve (or, because it's a
universal machine, it constrains the general patterns of state traversal in
some hand-wavy sense).

I built something that demonstrates these ideas, if anyone's interested.

------
psibi
How much does algebra contribute to learning Type theory ? I'm currently
working on the How to prove it book. I thought, I would jump to TAPL after
finishing that.

~~~
mafribe
I also don't think studying algebra helps you directly with programming
language theory. However algebra is of indirect help _if_ you want to learn
category theory, because the latter is a direct generalisation of the former.
It is rather difficult for a non-mathematician to pick up category theory from
scratch without having first seen the the algebra that category theory
abstracts from. Universal properties, category theory's most important
concept, appears bizarre and disconnected from reality, unless you have seen
it working in the much simpler settings of groups, rings, modules etc.

To that caveat, I'd like to add that neither Rotman nor Birkhoff/MacLane are
ideal places to start learning algebra. There are plenty of gentler, more
modern books. In terms of tomes that are available online, I can recommend
Shoup's "A Computational Introduction to Number Theory and Algebra" [1].

As an aside, note that category theory is used only in the part of programming
language research that is about (pure) functional programming. In other sub-
fields (e.g. OO, logic programming and concurrency), category theory has not
so far proven terribly useful.

[1] [http://shoup.net/ntb/](http://shoup.net/ntb/)

~~~
ihm
> In other sub-fields (e.g. OO, logic programming and concurrency), category
> theory has not so far proven terribly useful.

I wouldn't say that. I don't know about OO and logic programming, but there's
a good amount of categorical/homotopical structure lurking around concurrency.
See for example [1], [2] or [3].

[1]
[http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.46.9...](http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.46.9946)

[2]
[http://www.researchgate.net/publication/2108154_A_model_cate...](http://www.researchgate.net/publication/2108154_A_model_category_for_the_homotopy_theory_of_concurrency)

[3]
[https://en.wikipedia.org/wiki/Directed_algebraic_topology](https://en.wikipedia.org/wiki/Directed_algebraic_topology)

~~~
mafribe
You mean Goubault-style homotopy stuff. I don't this this is very categorical,
e.g. [2] produces a single category, so I wouldn't call this an example of
categorical structure. The Gunawardena paper [1] doesn't exhibit categorical
structure either.

------
dschiptsov
It is only me, or type theory is overrated and oversold?

~~~
JesperRavn
It is certainly oversold. There is a typical discussion that goes

A: I'm a simple Python programmer. Please tell me why I need types.

B: Types let you catch many bugs that would otherwise cause compile time
errors or require compile time checks or testing.

A: But I've seen studies that suggest that static typing does not prevent
bugs.

B: When I said static typing, I meant dependently typed languages, which can
express almost any constraints at compile time, not just the basic constraints
expressed by static typing.

A: Great, you've convinced me. What dependently typed languages can I use in
my production system.

B: Errrr

As a mathematical theory, a way to think about programming, and a research
project for future languages, type theory is very interesting. But it's not
otherwise that useful for the practicing programmer.

~~~
nbouscal
I've spent a lot of time hanging out with pretty hardcore type theory people,
and have never seen any of them recommend a dependently-typed language to a
new programmer. In other words, you're pretty blatantly fighting a straw man.

~~~
JesperRavn
It's not about how experienced the programmer is. The problem is that people
do a bate and switch when it comes to type theory, promising the benefits of
dependent types even though there are no production languages that deliver
these benefits.

~~~
nbouscal
I've similarly never seen anyone promise the benefits of dependent types when
talking about simpler type systems. Why would they need to? Much simpler type
systems already have more than enough benefits to be worthwhile.

I'm not actually looking to debate you, your mind is made up and there's very
little chance I'll change it, not that it would matter if I did. I mainly just
want to point out for others reading this that your arguments are blatant
straw men.

Also just fyi, it's "bait and switch".

~~~
JesperRavn
_> Much simpler type systems already have more than enough benefits to be
worthwhile._

If that's your opinion, why did you ignore the fact that I addressed this in
my original post? You accuse me of having a straw man argument, and yet you
can't even engage with what I've written?

Anyway, you're right that neither of us is likely to convince the other, but I
have really seen arguments that go like the one I described, and you're really
doing me a disservice by ignoring the details of what I posted.

~~~
nbouscal
You "addressed" it by referring to "studies" that you didn't cite. There are
studies that go both directions on the question, none of which have good
enough methodology to be worth more than a tiny update in either direction.
Most of the studies are about extremely weak type systems (C, Java, etc.), and
therefore don't provide any evidence at all about type systems that are
stronger than Java but weaker than dependent types (e.g. Swift, Rust, Haskell,
etc).

Meanwhile we have a lot of people who have used both weak type systems and
strong type systems who claim that strong type systems make their jobs
dramatically easier, and we have a lot of people who have only used weak type
systems who claim the opposite. Take a survey of people who have actually done
non-trivial work using a language with a good type system, and you'll get very
clear results about their view of the benefits. Clearly an RCT would be
better, but given that training takes a long time, that just isn't feasible.

------
panic
In practice, most programming languages are not purely or even mostly
functional. So why is everything on this list about functional programming?

~~~
exDM69
Because imperative programming is not very interesting from a programming
language theory point of view. If this was a list about compiler construction,
it would look very different.

As we're seeing more and more language features originating from functional
programming trickle into mainstream languages, this research hasn't clearly
gone to waste even though everyone is not programming in functional
programming languages.

~~~
mafribe

        imperative programming is not very interesting from a programming language theory point of view. 
    

I'm afraid I disagree, the opposite is the case. Modern PL theory is mostly
about program correctness, and that is much easier for pure functional
languages. Until recently, nobody had a handle on program logics for
imperative languages, or reasoning techniques for operational semantics. The
only available reasoning techniques used to be based on denotational
semantics, following Scott's breakthrough domain-theoretic semantics of
lambda-calculus. That's why so much theory was developed in a functional
programming context, because FP is in some ways much simpler than imperative
or concurrent programming (even though compiling FP languages is harder). The
available textbooks reflect this state of afairs.

All that has changed in the last decade or so, but it will take another decade
or so before recent research insights into non-functional languages perlocates
down to accessible textbooks.

~~~
xjia
If program correctness is easy to prove, why we are not doing that when using
either C or Haskell?

~~~
sgeisenh
Check out compcert for a verified c compiler. And several Haskell projects
have been verified.

~~~
xjia
I know it is possible. But I wonder what stops us from doing it for most
programs we write from day to day. And, if proving correctness is indeed
important (I believe so), can we do something / what is the next thing to do
to enable us proving correctness in our daily job.

~~~
mafribe
What stops us is cost. Formally verifying non-trivial properties (in the sense
of CompCert) with current technology is several orders of magnitude more
expensive and time-consuming than programming and a bit of testing. And this
price is almost never worth it.

We hope that this cost can be lowered in the future.

