
Writing a Compiler in Go - dcw303
http://squanch.org/writing-a-compiler-in-go
======
M4v3R
4 years ago Notch (creator of Minecraft) released specs for DCPU-16 -
imaginary 16-bit CPU that could run limited, but quite complete set of
instructions. The goal was to put this into his new game - 0x10c. People went
crazy about this idea and a whole community appeared with people building
assemblers, compilers, specs for new hardware and other tools for creating
software for this architecture.

I also got interested in it and found the same article from 1988 that OP is
referencing, which resulted in me writing a simple BASIC to DCPU-16 ASM
compiler, also in Go [0]. The code isn't of very high quality (it was just a
pet project) but if anyone wants to take a look, go ahead.

[0] [https://github.com/M4v3R/DCPU-Bas](https://github.com/M4v3R/DCPU-Bas)

~~~
Quequau
It's a shame that Notch dropped the 0x10c project. The concept seemed to me
like a great way to get kids more interested in actual computer engineering
(as opposed to just using computer applications).

~~~
speps
It seems TIS-100[0] would fill that gap pretty nicely (I've heard good things
from friends). It's by the same author[1] that introduced the concept of block
worlds (that Minecraft used for example).

[0]:
[http://www.zachtronics.com/tis-100/](http://www.zachtronics.com/tis-100/)
[1]:
[https://en.wikipedia.org/wiki/Zachary_Barth](https://en.wikipedia.org/wiki/Zachary_Barth)

~~~
overlordalex
There is a similar type game called Human Resource Machine[1] which I can also
recommend. It's made by the same people who made Little Inferno.

My one gripe with it however is that it takes the middle ground between
teaching assembly to people unfamiliar with programming and those who already
have some knowledge, and doesn't quite satisfy either.

[1]
[https://tomorrowcorporation.com/humanresourcemachine](https://tomorrowcorporation.com/humanresourcemachine)

------
etatoby
I would like to mention that most, if not all, of the Go compiler and related
utilities is written in Go itself, with high quality, open source code that
can be found in the "go" package of the standard library:

[https://golang.org/pkg/go/](https://golang.org/pkg/go/)

I would recommend using that as a template for one's own compiler, because
it's written in idiomatic Go (originally by the language creators) and is
cleanly split across abstraction boundaries (packages "scanner", "parser",
"ast", and so on.)

There's also a smaller, simpler parser in the "text/template/parse" package,
that is used to parse the templating languages in "text/template" and
"html/template". As an introduction to it, I recommend the talk "Lexical
Scanning in Go" by Rob Pike:

[https://www.youtube.com/watch?v=HxaD_trXwRE](https://www.youtube.com/watch?v=HxaD_trXwRE)

~~~
jerf
Actually, after a quick examination of the Go libraries, I'm not sure I can
agree that it's "idiomatic Go". I still see many things that strike me as
"artifacts of the C conversion" more than "the way someone would have written
this from scratch".

The careful package split is something to be copied, though.

~~~
stcredzero
Where is a good place to read "idiomatic Go?"

~~~
jerf
Much of the rest of the standard library. The compiler portion is a special
case.

------
zellyn
I can also highly recommend the Coursera Compilers class. Sign up for it after
it's over, and just work through things at your own pace.

The provided scaffolding code is either Java or C++, but with a bit more work,
it's perfectly reasonable to re-write it in whatever language you desire.
Here's mine, in Go:
[https://github.com/zellyn/gocool](https://github.com/zellyn/gocool)

Admittedly, it is the world's dumbest compiler (uses a single register, and
there's _no_ optimization), but it's still fun to actually "slay the dragon"
and stop believing that only wizards can build a compiler. :-)

~~~
pjmlp
When I did my compiler class at the university, our professor teached us a
nice way to do dumb compilers, that I always suggest as learning process.

Generating bytecodes in a suitable text format, that can be either be easily
validated and executed with a dumb interpreter, or translated into machine
code via a Macro Assembler.

Surely the code performance sucks, but it gives the students the gratification
of producing something that executes real machine code and that they can show
off to their friends.

------
59nadir
> I’ve been picking up Go lately, and I wanted to apply it to a project that I
> could sink my teeth into.

> What I came up with is very-much-not-idiomatic-go, but it does map
> (basically) line for line with the original Turbo Pascal.

Why not use this as an exercise in how to write idiomatic Go? The backdrop of
wanting to apply Go isn't exactly followed up on if you're just going to copy
completely different idioms without any regard for your new language.

I know it's touched upon later in the piece, but it really doesn't make sense
to make this an afterthought.

~~~
accnumnplusone
Agreed. Seems to be a lot of disregard for the idioms from people picking up
Go. I've recently had myself removed from a pairing where the project was some
sort of ghastly java oriented Go. Not enjoying it.

~~~
musha68k
That's a real shame indeed as the biggest strength of Go IMHO is exactly _the
culture_ it emerged out of.

Reading the source of the Go stdlib is a good exercise in preparation of
"letting go" of your idiosyncratic perspective. In that regard it doesn't
matter if you are coming from Java or Haskell or any other "world".

"Perspective is worth 80 IQ points." \- Alan Kay

(who probably hates Go but that's OK too ;)

~~~
accnumnplusone
I've read a decent amount of the std lib in the near 3 years I've been using
Go. I think the idiosyncratic perspective is that of the people who are unable
to let go of the language they're coming from and adapt to the new one.
Bringing ideas to an ecosystem is one thing, disregard is another. While I'm
here I'll throw in mention of npm-style 3 line function packages I've been
encountering. I think you've presumed too much about my perspective, and lost
out on that 80.

------
qnavry
Please make this accessible! (contrast ratio of 2.1 falls below WCAG)

[https://en.wikipedia.org/wiki/Web_Content_Accessibility_Guid...](https://en.wikipedia.org/wiki/Web_Content_Accessibility_Guidelines)

------
alehander42
This is awesome. Did you feel like Go is good enough for the task? I've always
thought its lack of expresiveness make it very unsuitable for compilers and
all kinds of pattern-matching/tree-representation-based code (opposite to e.g.
Haskell/OCaml)

~~~
sethammons
Go's compiler is written in Go.

~~~
jjnoakes
Which doesn't really say much about how suitable it is to write a compiler in
go.

Most C compilers are written in C. I would not argue that C is a language
particularly suited to writing compilers though.

------
pjmlp
> Let’s Build a Compiler, by Jack Crenshaw

Uau! Great idea, I went through those tutorials back when I started hanging
around USENET comp.compilers group.

I learned a lot with them, nice to see them being brought to newer
generations.

------
ryancox
Go also has a builtin yacc implementation:
[https://golang.org/cmd/yacc/](https://golang.org/cmd/yacc/)

    
    
      It is largely transliterated from the Inferno version
      written in Limbo which in turn was largely transliterated 
      from the Plan 9 version written in C and documented at 
      http://plan9.bell-labs.com/magic/man2html/1/yacc
    

For a real example in action, see the mtail project:
[https://github.com/google/mtail/blob/master/vm/compiler.go](https://github.com/google/mtail/blob/master/vm/compiler.go)

------
h8liu
(some shameless self advertising)

I built e8vm.io ([https://github.com/e8vm/e8vm](https://github.com/e8vm/e8vm),
[https://e8vm.io](https://e8vm.io)), which is all written in go, and it has a
full-blown compiler, with no optimizations though. The language has go-like
syntax and C-like run time.

shanhu.io ([https://shanhu.io](https://shanhu.io)) provides an online
playground for the language, which runs the compiler right inside the browser,
thanks to gopherjs.

~~~
h8liu
and for the DCPU fans, e8vm also has its own small mips-like instruction set,
which is currently the only target "platform" that my compiler supports.

------
nsajko
Gray text on black background is awfully unreadable.

~~~
doodpants
Agreed. And the blue color of the links is even worse; I actually can't read
them unless I move my head closer to the screen than is comfortable.

------
dalailambda
For an example of a language written in Go(besides Go), there's always
[https://github.com/ark-lang/ark](https://github.com/ark-lang/ark)

------
Scarbutt
This is nice but I had just used freepascal, Crenshaw's series is not about a
learning a programming language but about creating a compiler for a new one.

