
Rich Programmer Food - niyazpk
http://steve-yegge.blogspot.com/2007/06/rich-programmer-food.html
======
mahmud
Pretty entertaining, if a bit melodramatic.

I really wish people didn't mystify such a basic programming skill. Compiler
hacking is something reserved for the wizards _only_ if you take the classic
definition of compiler implementation: an expensive engineering project,
targeting a new processor, architecture or OS.

In that sense, sure. You will be working in a lab with hundreds of developers,
and tinkering with a piece of multimillion dollar prototype.

In reality, however, "compiler construction" boils down to foundational
language theory, along with various tricks and techniques for translating a
set of very straightforward algebriac rules, to another set. Anytime you write
regexes or XPath to extract a set of fields from a documents and transform to
something "readable", you're almost writing a simple one pass assembler, using
some implicit "business rules" (i.e. all numbers should be floats, names
capitalized, etc.) for productions.

Compiler skills will give you the theoretical backbone to discover, refine and
assess those implicit rules. Not to mention techniques for transforming them
from one form to another.

To the list of skills made mystical and magical by people I would add Lisp.
It's not magic. I mention it because it just so happens to have compiler
construction lore aplenty.

The first Lisp exercises you will read in your lisp book of choice (often
requiring nothing more than basic English literacy and 6th grade arithmetic)
are the expression simplification and evaluation exercises. Transforming your
elementary school algebra rules (multiplication and division before addition
and subtraction, etc.) to something the machine can execute. The hardest part
is just understanding the words: if you have a hunch for what "expression",
"term", "rule", "left/right hand-side" and "precedence" might mean, you're
good to start.

Few chapters of a Lisp book will spare you volumes of traditional compiler
construction techniques, taught by rote methods.

The first time I attempted to read SICP I had convulsions and physical pain.
The whole time I had this inferiority complex nagging at me, telling me this
was something for "smart kids" and I was unworthy. But this stopped after I
went through the first few parts of chapter 1, and took in the playful tone
the text. I felt stupid afterward; like being afraid of a St. Bernard. It
looks big, but it's actually bubbly.

Don't listen to people when they say something is difficult or not for the
faint of heart. Put a saddle on Falkor and go flying!

~~~
kenjackson
The problem with SICP as a compiler book is that it doesn't really doesn't
teach you much about compilers. At least beyond Lisp compilers. Even something
relatively simple, like stack layout isn't really covered, at least that I
recall. Or intermediate representations (unless you simpler consider Lisp your
IL), or even how different parsing techniques will interplay with what you can
parse (easily at least).

It's like saying that reading CLR's algorithms text will teach you how to
build real databases, because it teaches you about trees and hash functions.

~~~
jerf
"stack layout"

At this point you've already gone off the rails, because you've implicitly
accepted that a compiler is "something that emits machine code".

The reason why compilers should be a required course is that people need to
realize that compilers that emit machine code are nothing more than a _special
case_ of compilers. A very interesting and important one for both practical
and historical reasons, but a special case nonetheless. The problem with this
definition is that then people think that it's not a big deal that they don't
understand compiler theory because it's just for the guys writing gcc and
stuff. Wrong, wrong, wrong, wrong, wrong. A compiler is a basic tool in the
computer science toolchest and I tend to agree that if you don't get them you
shouldn't call yourself a Programmer with a capital P. The definition Steve
gives is much more accurate.

~~~
kenjackson
Wrong. Understanding stack layout is much richer than emitting machine code.
Admittedly, throwing the term "machine code" around is already a loaded term.
Do you mean x86 code? Do you mean Java bytecode or CIL/MSIL? Do you mean C?

In any case part of understanding compilers is understanding what you're
targeting... unless your only goal is to accept or decline a program for a
given language.

And if you read the context of my post, my point was exactly what you said
regarding pragmatics of understanding compilers. Sure you can say "kill sets
are the figment of a design choice relating to languages for which destruction
can exist". But that's like saying you know computer architecture, but nothing
of registers, TLBs, caches, voltage, pipelining, etc..., because those are
simply remnants of a special case of computer architecture, albeit a popular
case.

If you came to me and said, "I'm a compiler expert" and you knew nothing about
stack layout, I'd seriously question your credentials. Maybe I'd be off base,
but I'm pretty confident I'd be right a lot more times than not.

Update: And I did want to add one more thing on the stack layout discussion.
Stack layout is a "fundamental" device in the Turing machine model for
handling important language constructs. Sure you can argue that with the
lambda calculus you can avoid some of this. But, I still think you miss out on
understanding a KEY part of compilers. It would be like leaving CS education
and only learning about Turing machines, and never the Lambda Calculus.
Obviously, you can rationalize it by saying its completely redundant, But I
think most of us would agree that you wouldn't have had a full education.

~~~
jerf
POVRay is a compiler. It takes a specification of a scene graph (give or take
some details), and emits an image.

A JSON library is two compilers. One takes text and emits some form of
language-specific representation, the other takes the language-specific
representation and emits text. Serialization in general is a special case of a
compiler.

A syntax highlighter is a compiler. One end of it looks like a normal language
compiler, but it doesn't emit machine code, it emits annotated text. There are
some fun intricacies related to that that have little relevant to conventional
compilers, in fact: [http://blog.sigfpe.com/2009/01/fast-incremental-regular-
expr...](http://blog.sigfpe.com/2009/01/fast-incremental-regular-
expression.html)

A web browser is a compiler. (Actually it's rather a lot of compilers.) It
takes a web page and compiles it into the necessary set of instructions for
the local display library to correctly display the page.

It is absolutely feasible to be an "expert on compilers" who doesn't know dick
all about stack layouts on real machines, because you can write 10 compilers
without any of them going down to machine language, or even byte code of any
type. It's not even that hard. Counting up, my score is at least 5. And while
I know what a stack layout is, what its for, and various tidbits about how
they work in C, I certainly couldn't stand up to a blistering question session
about it; after all, what I've learned about C stack layout was in the using
of C, not the programming of a compiler for it.

Yes, when I say things that compile to machine language are a special case, I
really do mean that less than 1% of all compilers fit that case. Easily.
Probably less than .1%. You are _surrounded_ by compilers and only a rare set
of special cases goes down to anything like assembler(/bytecode). For the
vast, vast majority of compilers, there _is_ no relevant concept of "stack
layout". The ones that do are an important and interesting subset, but when I
say "everyone should understand compilers", it's very much precisely _because_
that subset is relatively small. If the only value of knowing about compilers
came from being able to hack on GCC, it really would be worthless to know
about in the general case, but instead we are swimming in a sea of compilers,
and not understanding their principles means you are in some real trouble.

~~~
kenjackson
I think I understand now. You don't know what a compiler is :-)

We should be clear about the definition of a compiler. From the Dragon book,
arguably the definitive text in compilers. "a compiler is a program that reads
a program written in one language and tranlsates it into an equivalent program
in another language". From Wikipedia, "A compiler is a computer program (or
set of programs) that transforms source code written in a programming language
(the source language) into another computer language (the target language,
often having a binary form known as object code)."

What you're describing is something much more broad... something called a
"program". Programs take input and generate output. That process is NOT called
compilation. Now I'm being a bit of a jerk in how I'm saying this, but I think
the distinction is correct. What you're describing as compilers are simply
programs. Programs that take input and generate output. But they don't pass
even the first test of being a compiler, they don't generate new programs.

There's an additonal test to being a compiler. Take the input, I, and output
O. Input I is written in a language with semantics, call it SI. The output O
is written in a source language with semantics, call it SO. The semantics of I
of SI should be the same as the semantics O over SO. That is SI(I) == SO(O).
None of your examples maintain this. They all result in instances of a
program, not a new program..

By your reasoning Notepad is a compiler. It takes keyboard input and generates
text on a screen that is readable to humans. Pacman is a compiler. Takes
joystick input and results in a video and audio that will delight anyone who
was a child in the 80s.

And yes, compilers are a small set of the set of all programs written. But
that wasn't the debate.

~~~
jerf
The definition of "computer language" is very broad. Study your basic computer
theory again; they don't call those things "languages" for laughs. JSON is a
computer language; it sure as hell isn't a human language. Bitmaps are
computer languages, as are PNGs or GIFs. HTML is a computer language. And to
the extent the Dragon authors didn't mean it that way, I don't really care,
because your definition sucks for a fundamental reason that I outlined before
but will outline again: It convinces people they only need to know compiler
theory if they are outputting "object code". And that's an absolutely
_terrible_ thing for a definition to do. By that standard mine is _way_
better, in the way that a "good" thing is way better than a "bad" thing.

A compiler takes in a stream of symbols and translates it to another stream of
symbols. Common compiler techniques include intermediate tree representations,
parsing of symbols, various recursion-based optimization passes on the
intermediate representation and phases, and various serialization techniques.
Probably the most distinctive characteristic of a compiler is the existence of
a usually-tree-based (though not always) intermediate representation that is
neither the input language nor the output language, though in some cases you
can optimize past this. (You won't get good program-size-scaling this way, but
if your task is constrained it can be a good choice.)

There exist no software engineering terms that have universally agreed-upon
meanings. If you wish to insist that a compiler must maintain semantic
meanings, that is a valid choice, but I reject the idea that it is the only
choice, and I reject the idea that it's even a _good_ choice. A definition of
compiler that focuses on the use of the distinctive techniques of compilers,
the parsing, the intermediate representation, the manipulation of the
intermediate representation for some effect, the re-serialization to some
format, is much more useful for almost any use outside of a classroom. (Which
I do not wish to impugn; that is a valid use.) Because, to hammer it home one
more time, the techniques you learn in compiler class is useful all over the
place and I have concrete experience (as does Stevey) that way way too many
people have never learned those techniques. You aren't going to help anybody
learn more about these techniques if you insist that the only "compiler" is a
radical subset of the things that use compilation techniques. You do nobody
any favors by standing on your definition and perpetuating the idea that these
are esoteric techniques of limited use. (Note I did not say you _believe_ the
idea that these are esoteric techniques of limited use, I said you are
_perpetuating_ it.)

Notepad isn't a compiler that I can see; that's a strawman. It's a continuum,
not a binary yes/no, but it uses basically no compilation techniques, and I
wouldn't be surprised that at least the initial implementation used none at
all. The original was limited to 64KB, IIRC, and at that point even on the
computers of the era you could just directly manipulate an array. No
intermediate representation distinct from the input, no parsing, no
serialization, it's not checking any of the checkboxes.

~~~
kenjackson
I think it is fair to say that our definition of compilers and even languages
is different.

I think we also disagree about the point of definitions. I think they're to
make communication clearer. Not to motivate people to do something. I think
there are better ways of doing that than to obfuscate definitions, which
generally have well understood industry and academic meaning.

But I think these are things we'll just have to agree to disagree about... I
just hope your definition of "agree" and "disagree" are the same as mine. :-)

~~~
eru
He's just going with the academic meanings, where e.g. `regular expressions'
describe `regular languages'.

------
grandalf
The logic Steve uses, while true, fails to appreciate the benefit of selective
ignorance.

It's possible to reason effectively about a system without understanding the
turtles all the way down, as long as you understand the limitations of the
abstraction you are reasoning in, which is possible to do w/o dissecting
turtles.

I'd further argue that the "scientist vs engineer" personality distinction is
applicable here. The scientist is concerned with solving the primary problem,
while the engineer is concerned with optimizing all the turtles.

Take a physicist using Mathematica... does she need to know how compilers work
to effectively do cutting edge work with the tool? No. So why is any other
programming langage any different (unless you're amused by dissecting
turtles).

~~~
eru
Yes.

And if the physicist is implementing anything on top of Mathematica, she may
profit from knowing some things about parsing.

------
asolove
If you're interested in improving your skills and you read HN a lot, you're
probably saying to yourself: 1. I wish I could improve my Lisp and 2. I wish I
knew more about compilers.

Read Norvig's "Paradigms of Artificial Intelligence Programming." It proceeds
through various early AI problems, which are good motivation. It shows how to
solve some of them directly, then moves on to solving them with interpreters
for DSLs, and finally shows how to speed up those solutions through
compilation. The problem-specific portion of the code gradually gets smaller,
more powerful, and faster.

------
jules
I've been thinking about a compiler for Lisp/ML like functional languages. If
we take the source program, convert it to continuation passing style, then
lambda lift, we have an intermediate form where no closures appear, every call
is a tail call and where the stack is represented as data structures
explicitly. The next step is to convert every call site

    
    
        f(x)
    

into

    
    
        if(f == FUNCTION1) FUNCTION1(x)
        else if(f == FUNCTION2) FUNCTION2(x)
        ...
    

The reason for doing this is that now all function calls are to known
functions (there are no calls to function pointers anymore). Because all calls
are tail calls we can then convert every call to a jump instruction.

(note that this could cause code blowup with a lot of different functions and
call sites, so it would probably be better to do this transformation lazily as
you analyse what values f could take, and only actually do this transformation
if you know that f can take at most 10 different values)

What we have now is one giant control flow graph (state machine) representing
the entire program. All functions have been eliminated. The only thing left is
primitive operations and jumps.

Why do I think that such a representation is interesting for a compiler?
Because many things that previously had to be coded separately are now one
analysis/transformation. Because there are no functions left all analysis and
transformations automatically become interprocedural. And for example (loop)
unrolling and inlining become the same transformation. So this could possibly
make the compiler much simpler.

Could something like this work?

~~~
silentbicycle
Have you read Appel's _Compiling with Continuations_?

~~~
jules
I have not, but the table of contents sure looks interesting. Am I right that
they don't do lambda lifting / closure immediately? They seem to have a lot of
optimizations working on lambda expressions, and also optimizations working on
records. By lambda lifting immediately I am hoping that the optimizations on
records can take care of (some of) the lambda-optimizations.

~~~
silentbicycle
CwC is very much what you want.

The focus on lambda and record optimizations is because ML code is usually
heavy with sum types (tagged unions), and it's often possible to completely
unbox things at compile time.

------
sreque
I like Steve's PL posts a lot, but I don't understand why he has to hate on
static type systems so much. I appreciate and use both dynamic and static
languages all the time, and both have their own respective strengths and
weaknesses. There are some problems for which a static type system is
extremely helpful, and others where no type system can really express the
problem well. To say that a whole active branch of CS that affects the most
popular languages in current use is worthless is a little presumptuous of him.

~~~
jpr
To me, it seems that the purpose of a most compilers for statically typed
languages is to either generate a lot of type information (*ML, Haskell etc.),
or require lot's of redundant typing (the keyboard kind) (Java etc.), only to
throw it all away during compilation. This makes no sense at all to me.

~~~
eru
Being able to throw away the static type information is a good thing. (Also
Haskell throws away less than, say, Ocaml.)

The static types in Haskell are a way to mechanically prove certain
propositions about the program. You do not need to conserve the proof during
runtime---because your very aim was to statically ensure runtime properties.

------
thyrsus
Work all the demonstrations and exercises in Kernighan and Pike's Chapter 8 of
"The Unix Programming Environment". Less than 60 pages in a friendly large
font. You'll be glad you did.

~~~
mahmud
The Unix-way of compiler construction is convoluted, difficult and verbose.
Any Lisp book will teach you more about compiler construction in the first two
chapters than you can learn from entire texts on the subject that use Lex and
Yacc.

Chapter 8, at a more leisurely pace, and far more in-depth:

<http://www.cs.brown.edu/~sk/Publications/Books/ProgLangs/>

~~~
pjscott
I've been looking through that book, and it's just what I've always wanted
from a compiler book. Though maybe I should have expected that as soon as I
saw the name "Shriram Krishnamurthi" on the cover.

------
saturdayplace
I'm no programmer, barely a web developer, and have no idea what he's talking
about half the time. But jeeze this man can write.

I should go learn about compilers. Or improve my typing.

~~~
epochwolf
I'm in the same boat. I have a degree but I don't know anything about
compilers. I'm convinced that being a remotely intelligent programmer means
you are going to be feeling stupid most of the time.

~~~
watmough
One of the most useful things you could do at this stage is go write a lexer
to split some input file up into lexemes / tokens.

As an example, think splitting a CSV file up into numbers, strings, separators
and newlines. If you get this right, think how easy it would be just to parse
this data into a simple data structure and generate SQL to spit it into a db.

Right there, in that one lexing module, you've probably taken the single
largest step towards accomplishing the tasks that Steve Yegge talks about.

Of course there are tools to generate lexers, but if you're learning
something, not much beats getting your hands dirty with real code.

~~~
epochwolf
I think I've done that already. My first program parsed a network protocol.
<http://botdom.com/wiki/Dante> (The code is nasty but it works)

~~~
johnzabroski
So research how to make it less nasty. Check out the Lexer Hack article on
wikipedia.

It also helps to familiarize yourself with classical pathological syntax
issues for languages like ALGOL-68 (requires 'semantic actions' to parse
correctly) and early versions of FORTRAN (identifiers could have intermediate
whitespace, which meant that a typo could corrupt the meaning of the program)
and figure out how to do parse a language with an Offside Rule (and not all
offside rules are created equal; Haskell 98 is very complex). Then look at
techniques like scannerless/lexerless parsing (SGLR, PEG, etc.), combinatorial
parsing, etc. Understand what mathematical properties of a parser algorithm
and a parser generator you should look for to ease your job. Understand why
naive user of a parser generator tends to result in slower lexing/parsing than
hand-coded ones. Understand why for some applications we might be able to not
care. Understand how to embed good error messages in a grammar-based approach
vs. hand-coded solution. Understand the notion of unparsers, which, IIRC, are
not covered in any of the Dragon Book at all.

------
silverlake
I've spent 20 years working on various compilers, yet I still don't think it's
a big deal. Undergrads just need to work on any type of large systems project:
OS, compilers, distributed systems, DBs, whatever. The problem with the
compilers/programming languages field is that every idiot with a computer has
an opinion. It's frustrating that popular languages are arguing about features
that have been analyzed in depth 20+ years ago, e.g. closures really are the
greatest thing ever and should be in any modern language.

~~~
peripitea
Can you expand on why closures are the greatest thing ever? I've heard this
from a few places, but I've only run into a handful of real-life situations
where they were the perfect solution to a problem I was having. And even then,
they weren't solving an otherwise-difficult problem; they were just saving me
some lines of code. I feel like I might be missing something.

~~~
drblast
Writing medium-sized projects in Javascript, I often look toward the sky and
thank the originators of that language that they had the foresight to include
closures and first class functions. I use them all over and save a significant
amount of effort. Because my time is extremely limited, they make things
possible I couldn't otherwise do.

If there are a handful of problems where they're the perfect solution, then
any language should have them, simply because the overhead in implementation
and linguistic style is so low.

------
staunch
(2007) should be added to the submission title.

~~~
mahmud
Compiler theory operates at mathematical time-frames, not Web 2.0's.

Best compiler texts were written before 1985 :-)

~~~
staunch
It's just a custom here as I'm sure you know.

~~~
silentbicycle
So is "posting the obligatory xkcd comic". So is posting "[citation needed]"
or reminding people that "correlation does not imply causation" like a
goddamned robot.

 _That doesn't mean you have to do it._

~~~
blasdel
But with pg making comments that are just a link to the readability
bookmarklet, I think this battle might be lost.

------
seldo
I certainly agree that learning to type is the most important skill in
programming. It saves you a few milliseconds every time you type anything,
which adds up over thousands of keystrokes into a clear competitive advantage.
And unlike compiler design, learning to touch-type is easy.

------
mattmillr
As a programmer who didn't take the compiler class in school, what's the best
way to go about learning what it takes to write one? A book recommendation?
Find a CS dept nearby and take the class?

~~~
muhfuhkuh
They call it the "Dragon Book". Compilers by Alfred Aho. It's supposedly THE
standard for compiler design.

~~~
10ren
As an introductory text, it reads more like a summary of journal articles.

However, I would recommend chapter 2, which builds a complete compiler for a
very simple language (arithmetic expressions). In the first edition, that did
it in C, and it's very simple and clear. In the second edition, they did it in
Java, and making it object-oriented makes it much less clear. I therefore
recommend the 1st ed (plus, the hand-drawn dragon on the cover looks much
better than the purple cgi dragon on the 2nd ed.)

~~~
johnzabroski
Note that there is more than one hand-drawn dragon book cover by Aho & Ullman.
Principles of Compiler Design tends to precede "The Dragon Book" as commonly
referred, although the material overlaps a lot. The newer editions are much
less clear with regard to the trade-offs in parser design, e.g. I don't think
the section on concrete and abstract syntax uses the phrase "scannerless" or
"scanner" at all while I am pretty sure PoCD at least says "scanner" and
explains why you might separate the scanner/lexer from the parser. The Java
book is much less clear, even though it improves on the reasoning (IIRC, pg
~113 in the Java edition) for why you should keep the two separate (although I
personally consider these reasons to be myths) -- it is the fact that they
don't really build up to the idea that makes me wonder why they provide
justifications in the first place. They don't really debate whether concrete
syntax is that useful, etc.

I have to agree that the Java code is an academic object-oriented programming
example. It's not "less clear" so much as "less practical". For comparison,
see the ANTLR API reference and the interfaces provided there, which differ
just enough to show practical consideration for the use cases one might want.

------
fleitz
A large part of the problem is people only see the skills for compiler design
being useful for compilers. If they would start with something simple they
could avoid the compiler design class and write better code by using things
like ANTLR from the start. The next time your program consumes text input try
writing a grammar for it and processing the AST instead of half baking it with
a regex.

------
clyfe
Read this article a couple years ago and recognized it. The best part of it to
remember:

<<A good friend of mine with a Ph.D. in languages told me recently that it's
"very painful" to experience the realization that all those years of slaving
over beautiful mathematical purity have more or less zero bearing on the real
world.>>

------
ww520
Compiler class was the most fun class I had back in school, though there were
a number of things I didn't understand and skimmed over. It was only years
late that I re-read the Dragon book in depth that I understood some of the
topics. Too bad there aren't too many language or compiler jobs.

------
yanilkr
Many things in software world are built on abstraction. It's not really
possible to know it all and many times it is counter productive to get into
any project with a mind set of I have to understand everything here.

------
mkramlich
I want to write a compiler that can take a Steve Yegge post as input and
compile it down to a compact, concise form more readily executable by my
brain.

------
chrismealy
His part about rails made laugh out loud.

I get the feeling Steve is teasing us with his magical vaporware just for
laughs.

------
moondowner
Turns out to be a good PR to taking a course on Compilers on faculty. I'm
sold, I'll sign up for it.

------
aycangulez
I think a better title for this article would be "How monstrosities get
written".

------
oiuytghyuj
Well as a physicst I say you can't understand a computer unless you understand
the wavefunction of an electron crossing a potential barrier. So ner-ner-ni-
ner-ner.....

