
Why Language-Oriented Programming? Why Racket? - jessealama
https://beautifulracket.com/appendix/why-lop-why-racket.html
======
hardwaregeek
I was thinking recently about why I find programming languages so interesting.
The answer I came up with was that programming languages allow you to create
your own reality. You get to define how things work in this reality. Want
functions to be values that can be arguments and return values? Sure! Want a
lot of crazy symbols to do complicated math? Go for it! Want everything to be
dynamic? Why not.

The caveat that comes with this description is that sometimes people shouldn't
create their own reality. Whether it's making decisions that ultimately lead
to really screwy universes or just not being prepared to deal with the power
that comes with changing the very fabric of reality, some people will not mesh
well with LOP. And there's nothing wrong with that.

~~~
amelius
The problem is that people often create languages that are basically the same
as existing languages; no new concepts. New languages are usually a grab-bag
of features found in C, Java, Lisp and/or Haskell.

~~~
DonaldPShimoda
There's a quote on the whiteboard in my lab that is (perhaps falsely)
attributed to Larry Wall that says something like:

> Programming languages are distinguished not by what they make possible, but
> by what they make easy.

Practically all programming languages are computationally equivalent. The
design of a new language simply seeks to answer the question: what should be
_easy_ for the programmer? Various languages are just different answers to
this question.

~~~
ken
This seems unsatisfying. Most of the modern languages are choosing pretty much
the same sets of features. I've been using them for years (decades?), and I
couldn't tell you significant differences between most of these languages.

There's maybe 3 major islands of languages today. Within each island, they
make essentially the same things easy.

The answer to a question like "Python or Ruby?", "C# or Java?", or "Rust or Go
or C++?" mostly boils down to historical accident: what system did the
original designer pick, which usually means which one just happened to have a
good library for the primary goal of the first prototype.

~~~
blotter_paper
I'd add some caveats to GP's comment, but I think it's mostly spot on.

>"Rust or Go or C++?"

Does one of these make memory problems easier to avoid?

>"Python or Ruby?"

One of these prioritises having a single way to do most tasks, while the other
prioritises programmer expressiveness (simple example: "unless" is absent in
Python). Thus one is easier to onboard newbies with and have them relatively
quickly be able to read the code in their ecosystem, while the other allows
skilled programmers to convey semantics more efficiently.

I'll admit being mostly ignorant about the differences between Java and C#,
but at least in general the languages you contrasted do make different things
easier.

Edit: grammar.

~~~
lucio
>the differences between Java and C#

Not much.

~~~
cicero
Are you primarily targeting Microsoft platforms? If so, then C# might have the
edge. Otherwise, you might prefer Java.

------
mark_l_watson
I bought two of the author’s books and also enjoy his newsletter. That said, I
still have not drank the LOP coolaid, yet.

I almost always start Lisp development projects in a repl, getting low level
things I need working. Initially I don’t worry at all about making things
tidy. I keep this process going until I have something that works. Then I
start cleaning up code, perhaps factoring out common code, etc. If I see
enough value in the new code and if it makes sense, then I move parts out to a
separate library.

It is at this point, after experimenting with a problem and having good code,
that I think in some cases it would be good to step back and decide whether to
design a new language for the type of problem you just solved.

------
GorgeRonde
As a Clojure programmer I find Racket's "#lang" feature fascinating, all the
more so it seems to be doing exactly the opposite of what's recommended in
Clojure: favor data over functions over macros.

Personally most of the DSLs I write in Clojure are "private" (i.e. I write
them for myself to help me develop a library or an app) and thus they tend to
be small. This is why I favor functions over data: it allows me to reuse
common function combinators (e.g (fn or-fn [f g] (fn [& args] (or (apply f
args) (apply g args))))) so that I do not have to reimplement them which is
something you have to do if you approach the problem with a data-first
mindset. Instead, if I really want to prettify such function-based DSLs by
bringing forth the use of data to express the problem at hand, I write a DSL-
as-data parser and implement it as one or multiple DSL-fns.

More @
[https://gist.github.com/TristeFigure/20dd01b0d3415f34075cfc0...](https://gist.github.com/TristeFigure/20dd01b0d3415f34075cfc02a1918106)

How does this compare to Racket DSLs ?

~~~
juliangamble
Some notes on Data > Functions > Macros in Clojure:
[https://lispcast.com/data-functions-macros-why/](https://lispcast.com/data-
functions-macros-why/)

------
adjkant
An interesting tangent I think about sometimes:

I think when people think about "programmers/developers" vs "computer
scientists", this is where the difference could really show up in the future.
A developer who knows how to code and hack may not have the skill/design
knowledge to properly create a language/DSL, perhaps making a monster in the
process. A well trained computer scientist (academic, self taught, doesn't
matter) should have those tools and more importantly the design know how. A
lot of teaching (again in both academia and not) really lacks on teaching good
design for programming generally.

I think we're already starting to see this difference show up with libraries
and frameworks (somewhat already DSL's themselves), and if we keep moving
towards the direction of DSL's I think you will find many of the bright
programmers of tomorrow going away from writing code to make things but
instead code to make languages to make things. The final step of code to thing
(from a technical perspective) seems to become increasingly boilerplate more
than ever before as tooling keeps expanding. You still need to know the deep
internals for large scale things, but it's never been easier to spin up a
quick project that "just works" and does quite a lot. If
library/framework/language creators do their job, this should only get easier.

At a certain point I wonder if the actual programming ever becomes a "lower
tier" of the software world.

~~~
maxhallinan
> A developer who knows how to code and hack may not have the skill/design
> knowledge to properly create a language/DSL, perhaps making a monster in the
> process. A well trained computer scientist (academic, self taught, doesn't
> matter) should have those tools and more importantly the design know how.

The author of Beautiful Racket, Matthew Butterick, is a lawyer and a
typographer. He is not a computer scientist. And yet, he designed a DSL called
Pollen for creating web-based books. Pollen has been quite successful within
the Racket community.

My understanding is that Racket is predicated on the idea that people like
Matthew, as much as people like your computer scientists, are the best authors
of DSLs. They are the ones who understand the domains they're working in. If
the division you predict ever does exist, it is because the tools have failed
to make people who are not computer scientists capable of creating the DSLs
they need to solve their problems. It's not because languages are inherently
better designed by experts in the domain of programming languages.

~~~
adjkant
Why wouldn't the author qualify as a self taught computer scientist though? To
me your point more of a statement on the accessibility of computer science
first, and I do completely agree that the accessibility is important, which
the Racket/HtDP ecosystem does pretty well with.

That said, there's many ways to write code and learn to code, and I think of
the web programming bootcamp style or the cookie cutter college grads who go
through four years learning how to program in X language to work at fancy
company Y. My point is that many routes are just not focusing on the higher
level design skills that I think are needed to make good
libraries/frameworks/DSL's.

To clarify, I'm not saying you have to be a PL expert, simply good at program
and language design, which I think is what lacks in many places and could
create such a division. That skill is/should be accessible to everyone.

~~~
maxhallinan
Ok, I think I read more into your first comment than you actually said.

>many routes are just not focusing on the higher level design skills that I
think are needed to make good libraries/frameworks/DSL's.

I have observed that too. But I don't think this is about who is and isn't a
computer scientist, whether self-taught or formally trained. I think it's more
a change in the way people relate to programming languages. Perhaps
programming languages were commonly assumed to be principally an academic
topic. Perhaps it's not that more people are becoming computer scientists but
that more people are finding non-academic ways to relate to programming
language design. I think what Butterick did was to build the tool he needed to
do a job (write a book). So language design becomes just like any other form
of hacking.

>Why wouldn't the author qualify as a self taught computer scientist though?

Butterick himself is adamant that he is a lay person. He compares himself to a
"squirrel in a ferrari":
[https://www.youtube.com/watch?v=IMz09jYOgoc](https://www.youtube.com/watch?v=IMz09jYOgoc)
And that's his point in that talk - Racket makes it possible for even the lay
person to build the language they need.

~~~
wool_gather
He can view himself however he wants, but the man just wrote an article that
competently covers Turing completeness, regular expressions, and Lindemeyer
trees, among other things! He's definitely earned his comp sci merit badge, so
to speak.

~~~
maxhallinan
Computer science is a specific treatment of these topics that is based in
formalism. Compare John McCarthy's "Recursive Functions of Symbolic
Expressions and Their Computation by Machine, Part I" and Paul Graham's "The
Roots of Lisp". These papers cover exactly the same material. But only the
first is computer science because it uses a formal language to express the
ideas.

~~~
adjkant
If that's the definition of CS we're using then my original post is a very
egregious misnomer. I feel like that definition is way too restrictive though.
CS can be formalized and informal, but both are still CS IMO.

------
bsaul
I'm currently browsing the list of languages made with racket (here
[http://docs.racket-lang.org/search/index.html?q=H%3A](http://docs.racket-
lang.org/search/index.html?q=H%3A)) and it seems like most languages are just
lisp variants. I suppose they were made for educational or fun purposes, but i
wonder if i haven't missed something more deep as to why someone would want to
reimplement a lisp in a lisp language.

~~~
mpweiher
That was my impression as well: you can have any language, as long as it's
LISP.

Some of the more specialised languages are different, though.

~~~
krapp
> you can have any language, as long as it's LISP.

Let's be fair, many lisp programmers would be perfectly fine with that.

~~~
bsaul
ok but then since lisp already has an extremely small syntax, why not simply
define functions in the language rather than using the macro system ?

Unless your goal is to be source compatible with another lisp variant, of
course.

~~~
krapp
>why not simply define functions in the language rather than using the macro
system?

I don't know. Most examples I've seen from people extolling the virtues of
lisp macros and metaprogramming wind up just generating more lisp code in the
same language with the same syntax and semantics, so I don't know why you
couldn't just use functions in that case.

To be fair, I only have a surface understanding of one lisp variant (Arc) so
it's entirely likely I just don't get it.

~~~
gus_massa
A nice example is `match` that is one of the macros that comes by default in
Racket.

    
    
      #lang racket
      
      (define (show x)
        (match x
          [(vector a b c)
           (display "It's a vector, total = ")
           (displayln (+ a b c))]
          [(list p q)
           (display "I's a list, average = ")
           (displayln (/ (+ p q) 2))]))
      
      (show (list 8 2))
      (show (vector 1 2 3))
    

(Playable version
[http://pasterack.org/pastes/75154](http://pasterack.org/pastes/75154) )

The idea is that `match` can analyze the value of the first argument (`x` in
this case) and then it binds the variables (a,b,c or p,q in this case).

In this case I'm matching only a vector with 3 values or a list with 2 values,
but you can match more complicated and nested pattern.

The interesting part is that inside match you have the full power of the
language, you are not restricted to a few special hardcoded cases (like
`display`). You can send an hppt request, play a sound, save a value to disk,
calculate the prime factorization, transform the text to uppercase, anything
that is available in the language is available inside match.

But `match is just a normal macro. Racket is implemented in layers. The bottom
layer is implemented currently in C, but it doesn't define `match`. The bottom
layer is almost hidden and it is used to define some libraries and an
intermediate language that it is used to define some libraries and an
intermediate language, ..., and after a few steps you get the official Racket
language.

But if you like you can reimplement a macro like `match`, or a variant, or
your own macro, with all the magic to define local variables and use all the
power of the language inside it.

------
DyslexicAtheist
I like his style of writing and how elegant and powerful LOP is. The _jsonic_
snippet is insane (in a good way). In the wrong hands or for the wrong job it
could lead to a security disaster though. Still pretty cool to be able to
think out of the box like this. Makes me rethink totally the way we do things
and why.

    
    
      #lang jsonic
      // a line comment
      [
        @$ 'null $@,
        @$ (* 6 7) $@,
        @$ (= 2 (+ 1 1)) $@,
        @$ (list "array" "of" "strings") $@,
        @$ (hash 'key-1 'null
                 'key-2 (even? 3)
                 'key-3 (hash 'subkey 21)) $@
      ]

~~~
agumonkey
methinks a lot of the programming world at the interface
(semantics/syntactic/ergonomic) is .. implicit tree processing.

see html encoding through functions, or kotlin builders, or json dsl ..

------
quelltext
If you create whole new languages, i.e. external languages, not really
embedded DSLs, with their own syntax, how is Racket better than other
languages at implementing those?

Neither the sntax object, nor hygiene are of concern when you build external
DSLs, which the author's examples seem to be.

I have not yet seen a compelling argument for Racket there vs. for instance a
parser generator framework.

I might just be missing the point. Either way the author's convincing seemed
to have failed for me.

~~~
pge
In LISP-like languages like Racket, the code is a list. That is, a program is
just a list of tokens (a b c), the same data structure one would use for
storing any other kind of data (the equivalent of Python’s [a b c]). When
programs are themselves just lists, they are easy to manipulate with code. So,
in LISP-like languages, it is easy to write code that writes code. That makes
them especially suitable for domain-specific languages or LOP. You can write
functions that write code, making a new language.

I think PG has some writings on this topic, but I don’t remember if it is
online or in his LISP textbooks. His company Viaweb did this, IIRC, using a
DSL written in LISP to generate HTML.

~~~
amelius
> In LISP-like languages like Racket, the code is a list.

This is nothing special. In all languages, code is a list: a list of
characters, that is.

Just saying.

~~~
krapp
What the parent likely means is that Lisps, structurally, are akin to the AST
many other languages generate as an intermediate step to compilation.

------
jhorsager
I hear this alot about Racket as if this is the value proposition and Racket
stands alone with these features. Languages like Ruby are routinely used to
build DSLs (RSpec, Rails, ServerSpec, Chef...etc). I would like to hear
somebody address why Racket is superior to languages like Ruby for creating
DSLs, because it for sure doesn't stand alone. Metaprogramming is a feature in
a LOT of languages. I'm not arguing for those languages, I'm arguing Lisp
languages aren't the only ones capable of creating other languages and a
comparison would be more useful than repeated emphasis on unique ability for
the task which I don't think is necessarily true.

~~~
creatornator
In practice most Ruby "DSL"s are just regular libraries that you happen to be
able to use without the familiar foo(bar) function call syntax. It's the
difference between "Domain Specific Language" and "A Domain Specific
Language", or embedded vs. hosted/standalone DSL's [0]. The value that Racket
provides is facility in developing standalone languages, as opposed to the
embedded DSL's that people make in Ruby.

To address the examples you gave, RSpec, Chef, etc., those literally ARE just
using Ruby, no new syntax, no new semantics.

[0] [https://www.quora.com/What-is-an-embedded-domain-specific-
la...](https://www.quora.com/What-is-an-embedded-domain-specific-language)

------
wildermuthn
If I’m understanding the article correctly, it should be possible to create a
Racket language that compiles to any other language. A quick google does
reveal work has been done on C and Python, although it appears there are bugs
and various work-arounds that precludes a full and clean integration. The
Javascript integration appears dead (in terms of using the latest version of
Racket)

Although using Racket for this wholesale integration appears impractical, it
does suggests a killer-app: extending general programming languages. In other
words, using Racket as a transpiler. I.e., a better Babel.js, but for any
language.

I routinely use Python for machine learning tasks, but frequently am
frustrated by the inflexibility of the language. Instead of solving the
problem itself I have to first solve how to make the language do what I want
(usually resulting in code that lacks expressiveness and brevity, and thus
extensibility).

I’d love to use lisp to leverage ML work that can only be done effectively in
Python (Tensorflow, PyTorch, etc).

Can Racket be an effective transpiler?

~~~
xrayspec
IIUC Babel.js is essentially a fancy preprocessor for JS. So if by that
analogy, you mean "can I use Racket to design my own notation that compiles to
Python", yes of course.

------
bling0bling0
I find it a pretty amazing feat that the author has created a programming
language (Racket), then used it to create another language called Pollen[1]
for book publishing, then went ahead and used Pollen to publish three books.
Oh and the fonts are designed by him too[2].

[1] [https://docs.racket-lang.org/pollen/](https://docs.racket-
lang.org/pollen/)

[2] [https://mbtype.com/](https://mbtype.com/)

~~~
seotut2
No, the author hasn't created Racket. The author is the lawyer
[https://en.wikipedia.org/wiki/Matthew_Butterick](https://en.wikipedia.org/wiki/Matthew_Butterick).

Racket was developed by PLT Inc.

~~~
DonaldPShimoda
> Racket was developed by PLT Inc.

There's not a lot of information on PLT Inc. these days, so: Racket was
developed under direction of Matthias Felleisen, with his students Matthew
Flatt, Shriram Krishnamurthi, and Robby Findler serving as the core
development team. They also published the book "How to Design Programs" [0],
and Krishnamurthi published "Programming Languages: Application and
Interpretation" [1], which are both books that rely on Racket. (PLAI actually
uses its own language which is developed in Racket, called plai-typed, but
which relies on the DrRacket environment.)

[0] [https://htdp.org](https://htdp.org) [1]
[http://www.plai.org](http://www.plai.org)

------
mpweiher
About a decade ago, I would have agreed with this 100%, and I still am
fascinated and, quite frankly, awed by Racket and LOP (and looking forward to
Racket Fest 2019 here in Berlin[1]).

But.

(You knew there was a "but").

While I don't quite agree with the assertion that with the right tooling (so:
_this_ tooling), creating a language is as easy as creating a library, I don't
think it would solve our problems even if it were true.

Language _design_ is much harder than library design, even if the tooling is
perfect and transparent. In the extreme we're going to end up with what Alan
Kay calls _inverse vandalism_ [2] aka the Mount Everest Effect: making things
because we can, not because we should or they're good.

In essence, designing a (domain specific) language should be your _last_
resort, not your first resort. At least one that requires a parser or macro
system to implement. After all, when we create a library, we _are_ in fact
creating a domain specific language, or at least a domain specific vocabulary.
A framework goes a little further towards the idea of language. In fact, the
first version of Simula was a simulation DSL. It was the second version that
made the language into an OO language and the simulation "DSL" into a
framework. The advantage being that you can use the same language with
different frameworks for other domains.

If we look at natural language, it is very much the same: we use the same
language to talk about any number of domains, the most we tend to add is some
domain-specific vocabulary. Imagine the tower of babel we would suffer under
if every domain had a completely new and different language!

That effect was very much encountered and later described by the LRG at PARC
when they started using Smalltalk 72, which allowed every object to defined
its own language[3]. It was chaos! Turns out we are not very good language
designers in the heat of the programming battle. So they very consciously made
the language more restrictive later. However, the Smalltalk message syntax is
specifically designed to give you something akin to DSL, but without most of
the problems[4].

A similar effect was documented by Bonnie Nardi for end users. Here as well,
the expectation was that users would prefer domain-specific tools that were
tailored exactly to their needs. This, surprisingly, turned out not to be the
case. End users apparently vastly prefer general tools, even if the effort to
adapt them to specific tasks is greater (see also: people writing documents in
Excel...)

That said, current general purpose programming languages are probably not
quite flexible enough. I think there is a sweet spot to be found somewhere
between Smalltalk-80 and something like Racket.

[1] [https://racketfest.com](https://racketfest.com)

[2]
[http://worrydream.com/EarlyHistoryOfSmalltalk/](http://worrydream.com/EarlyHistoryOfSmalltalk/)

[3]
[http://stephane.ducasse.free.fr/FreeBooks/BitsOfHistory/Bits...](http://stephane.ducasse.free.fr/FreeBooks/BitsOfHistory/BitsOfHistory.pdf)

[4] [https://youtu.be/Ao9W93OxQ7U?t=627](https://youtu.be/Ao9W93OxQ7U?t=627)

~~~
kasey_junk
I largely agree but then I enter the modern ops stack where templated yaml is
the lingua franca.

It’s incomprehensible and for each system you are operating you have to learn
some arcane yaml incantations that have dramatic impact on your production
systems. It’s a world crying out for a set of well factored dsl.

~~~
mpweiher
Or a general purpose language with the ability do describe architectural
configurations?

~~~
nmadden
I think this is what Pulumi ([https://www.pulumi.com](https://www.pulumi.com))
is trying to achieve.

------
wool_gather
This is tangential to the article, but. Scheme was one of my first languages
and I would love to pick it/Racket up again. But the Racket IDE (like DrScheme
before it) is _painful_ to use. Especially compared to a more "native" Mac
editor.

Has anyone used Racket with success in another IDE or programmer's editor?

~~~
soegaard
Try `racket-mode` for Emacs. It has many of the features that DrRacket has.

[https://github.com/greghendershott/racket-
mode/blob/master/R...](https://github.com/greghendershott/racket-
mode/blob/master/Reference.md)

~~~
wool_gather
I'll give it a go, thanks!

------
dangom
Definitely interesting, and the examples look nice, but biased in that
absolutely no evidence is given to support that LOP makes anyone more
productive.

------
platz
> maximum precision

Sometimes maximum precision means having a first-class typechecker & type
inference. (You could build some of that too in Racket if you really need to.
Of course, _that_ code itself will not be verified or typechecked.)

> Why aren’t these lan­guages DSLs?

They aren't DSL's because they need statically-linked binaries, and good error
messages, and then your task is not as easy as desugaring to a LISP/dynlang.

~~~
adjkant
> Of course, that code itself will not be verified or typechecked.

Easy, just used typed Racket :)

~~~
platz
Ok, so now you have Typed Racket (that last point was a relatively minor
comment on the larger argument). Does implementing the above then correspond
to the strengths as described in LOP?

~~~
adjkant
I'm not sure I'm fully following what you're getting at due to the terseness
and my first comment was more of a fun note that you can always build on
language features, but generally speaking you could inherit and use the types
and checking of Typed Racket in your DSL and still add the syntax you want for
increased precision.

------
findingbuzz
so many programming languages... i wonder if there is someone that knows them
all

------
dullcrisp
I’m sorry to nitpick on a good article, but this article lost me when it
proved that HTML is a programming language by showing that it can be output by
a Python program. This seems to be a case of proving too much [0].

[0]
[https://en.wikipedia.org/wiki/Proving_too_much](https://en.wikipedia.org/wiki/Proving_too_much)

~~~
Retra
It may be proving too much, but it is also arguing the general point. Your
objection would be that this example would prove that _all_ strings are
programs, which might seem absurd. But in fact, all strings _are_ programs if
you haven't specify which interpreter they are supposed to be programs for.

Either way, that seems like a silly approach, IMO.

------
cubano
"Before you leave, please know: I’m hedged either way. LOP and Racket have
been an incred­i­ble force mul­ti­plier on my pro­gram­ming pro­duc­tiv­ity"

Well, wouldn't we expect that the author of a particular programming language
to be mad productive in it because, you know, they designed all the mental
models and assumptions that are baked into it?

You would never once stumble on syntactic ambuigity because, to you, there is
no ambuigity...or perhaps there is but you slyly added that bit just to fuck
with programmers heads...or who knows the reason.

I'm just postulating that for an author to boast about their productivity
_with the language they designed_ is somewhat self-serving.

~~~
kasey_junk
The author of this article is not the author of Racket.

Racket is built by a team and has been around a long time.

