
Racket – Lisp beyond Clojure - macco
http://slides.com/ronie/racket-beyond-clojure#/
======
DigitalJack
I find DSLs to be extraordinarily powerful for writing and working with my own
programs. I find them painful when I'm trying to work with other people's
programs.

Making a DSL lets me construct a language to express my program that meshes
with the way I think. Unfortunately, we all think differently, and what is
intuitive and friendly for me is hostile and awkward for others.

While reading someone else's program that uses a DSL may be superficially
easy, but the minute you try to get under the surface, you're left scrambling
to understand what a given term or idea meant to the author. There is no
language definition to rely on for common understanding.

There are exceptions naturally.

I know a lot of people will disagree, and that's fine of course. It's opinions
all the way down.

~~~
kd0amg
_While reading someone else 's program that uses a DSL may be superficially
easy, but the minute you try to get under the surface, you're left scrambling
to understand what a given term or idea meant to the author. There is no
language definition to rely on for common understanding._

I have generally found that this is the same problem as with user-defined
functions, and it has the same solution: programmers need to document their
code.

~~~
brandonbloom
Agreed. I have two follow up thoughts:

1\. Badly designed functions can do less design damage per-function than badly
designed languages per construct, but user-defined functions tend to have much
higher volume than both language constructs and their usages.

2\. Powerful techniques, including DSLs, lead to denser code. Denser code
takes more time to read per unit of code. Total reading time may be either
more or less, but the cold start ramp is always much steeper.

Combined, this means that people are less sensitive to suffering the
complexity caused by user-defined functions vs DSLs/etc because that
complexity has lower intensity spread out over a greater area. I'll refrain
from any further value judgement on this state of affairs; just wanted to
share some more insights in to the tradeoffs.

~~~
lisper
> Badly designed functions can do less design damage per-function than badly
> designed languages per construct

Really? When was the last time you successfully used a C library without
reading the documentation?

~~~
brandonbloom
Straw man. When was the last time you had to check the documentation for the
parameter evaluation strategy of a C function call?

I don't think it is a controversial statement than a badly designed macro can
do more damage than a badly designed function. I believe that this is true,
even normalizing for bad macro systems. This is not a statement against
macros, so much as it is a statement to consider when making tradeoffs.

~~~
lisper
It is controversial. The GETS function is probably responsible for a huge
number of security breaches with a cost possibly ranging into the billions of
dollars (there's no way to really know). What would you cite as a comparable
example of a badly designed macro?

~~~
GFK_of_xmaspast
Part of the reason gets() has been such a disaster is that people use it.

------
jnordwick
Unhappy with the current state of functional programming. I know I should't
be. I have a lot of options.

I went to school at UC Berkeley. My first language was Scheme. Then Elisp.
CLOS was in the first few too. I really didn't learn C or C++ until second
semester.

A few months ago I had to pick up enough Scala to figure out what some code
did in a project and I wasn't really happy with the language. I realize it is
multi-paradigm, but the functional part just seems a little bolted on at
times. The notation was clunky. Maybe it is because I prefer Lisp-ish and APL-
ish languages. And the Scala docs aren't very friendly. I actually find it
easier to write imperative code in Scala than functional code.

Clojure seems nice, but lacks a lot of things that you kind of want in a LISP
(proper tail rec). Racket has very little market pen.

Maybe I'm just picky, but I don't see anything I really want to put time into.

~~~
DigitalJack
I guess I don't understand what the big deal is with tail call optimization.
Could someone give an example where it really shines and clojure's loop/recur
just doesn't?

If you are looking to put time into a programming language that is interesting
in and of itself, I'd suggest Haskell.

~~~
jnordwick
Looping is fine for imperative languages. It should be the norm for
performance sensitive things since that is how the CPU works. It is just
easier than trying to make a smart compiler turn your recursion into looping
constructions.

However, for functional languages you really want TCO to work properly so you
can write your algorithms in a properly functional way. And you don't just
want self-calls to work, but full mutual recursion (A calls B that calls A
etc) to be properly optimized too.

~~~
optforfon
the CPU is just doing a jump to an instruction at the end of each iteration.

When you do tail call recursion you are doing just that. The assembly jumps to
a tag and in recursion you jump to a function name, which is also effectively
a tag.

So it's not really about mapping better to the CPU instructions or anything
like that

The added benefit is that with tail call recursion you can have mutually
recursive function (function A calls function B at the end and function B
calls function A at the end), which isn't possible with a simple iterative
loop, so the functional style is actually more powerful than an iterative
approach and still maps directly to "the way the CPU works"

~~~
jnordwick
You are forgetting about the function prologue, epilogue, and the maintenance
of the various pointers. It certainly isn't as simple as a jump as you imply.
To do proper TCO most implementation resort to function trampolines and other
self-modifying trickery to get similar to loop-like performance. Within the
confines of the JVM, without doing extensive source analysis, I doubt there is
even a way to do it.

~~~
optforfon
this is outside my field of expertise. Can you give an example of a tail call
that needs "trickery"?

I'm not 100% on this, but when you have a tail call I don't think you have to
remember the state of the stack frame. "function prologue, epilogue, and the
maintenance of the various pointers" aren't god given rules - they're things
dictated by something like the C ABI so you can resume the caller. So if
you're thinking in terms of C ABI then yes, it's a compiler optimization, but
in principle I think it's a zero cost abstraction (though I'd argue that the
iterative loop is the actual abstraction)

In tail call recursion you're ensured the caller will never resume!

When you enter a tail-recursive function you save (as you normally would) a
return pointer for the program counter and allocate registers/memory for your
function arguments. At the end of the function you've done all the computation
that that frame will ever do, so you are free to overwrite the argument
variables when calling the next iteration/recursive-step. The return pointer
doesn't need to be touched at all. Where is the complexity you're talking
about?

~~~
senex
When TCO is "lexically scoped" like loop/recur, the compiler can handle it.

When HoF are involved, you may have a case where a procedure calls a
procedure-parameter, which calls another, and another... Something about the
runtime has to recognize this or else the compiler has to accept more
constrains.

See, for example, how Gambit-C implements tail calls by compiling modules to
single C functions and how there is a trade off for procedure calls across
module boundaries versus Chicken Scheme's approach to essentially garbage-
collecting the call stack.

~~~
optforfon
Okay, but at that point you're talking about things that are way beyond the
capabilities of an iterative loop. I think my point still stands - that
implementing a tail recursion in place of a loop is not something you will
have to pay for. Both structures will map to the same instructions.

~~~
ufo
The difficulty with tail recursion optimization is related to calling
conventions. Some calling conventions expect the caller to do some cleanup
after the callee returns, which effectively means that no function calls
actually occur in tail position. For example, in the C calling convention the
caller caller is responsible for allocating _and_ freeing the stack memory
dedicated for the callee's function parameters. This system makes vararg
functions like printf easy to implement but makes it hard to do TCO. Another
example is Rust, where having destructors automatically run when a variable
got out of scope prevented them from implementing TCO in a clean manner. I'm
not familiar with the JVM internals but I think the limitations are going to
be similar to the ones I mentioned here.

~~~
steveklabnik
It's not just that, it's that last time there was serious talk of it, LLVM's
musttail wasn't properly supported across important platforms for us. So it
got put by the wayside, and there's always so much to do, nobody has worked
through the semantics now that support is better.

We did reserve a "become" keyword for this purpose though. Instead of
"return", you say "become", and TCO is guaranteed. That's the basic idea
anyway, we'll see what happens.

------
dasmoth
This is addressing a question I'm interested in, but on first read I'm
struggling to see the value that's being presented here.

\- "parameterize" is great, but seems exactly the same as Clojure's "binding"
on a dynamic variable. [Edit: modulo being "sticky" with regard to
continuations... but Clojure doesn't have continuations, so it's a pretty
subtle difference.]

\- Reader macros are an interesting feature, but do move away from the
syntactic regularity of LISP -- and were, AFAIK, explicitly rejected in
Clojure

\- Custodians _are_ kind-of interesting -- but I ended up having to do some
Googling [1] to find out exactly what they do...

Is there anywhere else I should be looking for a more systematic
Racket/Clojure comparison?

[1] - [https://docs.racket-lang.org/reference/eval-
model.html#%28pa...](https://docs.racket-lang.org/reference/eval-
model.html#%28part._custodian-model%29)

~~~
pareidolia
> \- Reader macros are an interesting feature, but do move away from the
> syntactic regularity of LISP -- and were, AFAIK, explicitly rejected in
> Clojure

That does not make sense to me since clojure introduced explicit irregular
syntax for things like maps and vectors. In Common Lisp the "we-the-language-
developers can introduce special typography but the users cannot" thing, in
constrast, doesn't exist .

~~~
weavejester
The majority of Clojure's special syntax is for data structures, and while
Clojure doesn't have reader macros, it _does_ have data readers.

------
chriswarbo
I've recently started porting some bash scripts over to Racket, using
[https://docs.racket-lang.org/shell-pipeline](https://docs.racket-
lang.org/shell-pipeline) and it's been pretty straightforward and painless so
far!

~~~
jonathonf
Are there (m)any advantages to doing this? I've seen a number of shell-interop
modules for various languages but they all appear to add an extra layer that
reduces portability or ease of maintenance.

~~~
jlg23
> Are there (m)any advantages to doing this?

[trigger warning: cynicism distilled from 15+ years of bitter tears]

More fun and better job security for the current maintainer. The former
because one hacks away in the preferred language, the latter because one
cannot be replaced easily by some unix geek unless s/he happens to speak the
same language fluently.

One can chose from various implementations in various languages (eg: psh and
scsh) but hacking your own is easy:

* Just re-implement the basic unix tools and shell semantics and invent some nifty convenience syntax and features along the way.

* Either don't document at all or document everything with at least 2 pages aimed at people who have never used a computer before. Both ways will prevent the only ones who _could_ replace you from even looking at your "shell in X".

* Never ever do this in a typed language, write a ton of unit-, functional- and integration tests to make up for that - of course all of them completely undocumented.

* On top of your creation, invent a fragile "shell syntax like" layer to make adoption easier.

* Under no circumstances create some new abstraction layer that goes beyond shell semantics that brings something substantial to the table and is well documented.

* Enjoy the time saved because you don't have to "man $tool" anymore, it's all your code anway.

* Under no circumstances ever write in pure posix sh again - the realization that with all the stuff you've learned along the way posix sh covers 99% of the use-cases in a more expressive way will be crushing.

~~~
SwellJoe
Remind me never to hire you for anything ever. You've truly internalized the
Tao of the BOFH.

~~~
jlg23
Thanks for the compliment. I'm usually hired for that attitude (and my ability
to hide it except for the few meetings where it really counts) ;)

------
noobermin
I'd think Common Lisp is the Lisp beyond Clojure.

Also, I just see slides that have (googleable) terms, is there a recording of
this presentation?

EDIT: see mkozlows' comment (THANKS!!!)

~~~
616c
I have been trying to get into the whole Lisp paradigm for 1-2 years now. I
own Realm of Racket, amongst a collection of Lisp books.

Racket is all the promise of developer-centric power tooling that puts Common
Lisp to shame.

I recommend you watch one of the core devs, Matt Flath, build a a hygenic
macro expander.

[https://www.youtube.com/watch?v=Or_yKiI3Ha4](https://www.youtube.com/watch?v=Or_yKiI3Ha4)

Notice how his talk, likely written in its own documentation language
Scribble, shows demos of DrRacket. I am more of an emacs guy, but do you see
the interactive colorized debugger pointing out variable binding and flow
control? That is the only thing that I have seen in the Lisp world that takes
SLIME and laughs at its dogged simplicity.

Also, as you watch this talk, observe how he takes the complexity of something
I am still certain I cannot do, build a macro system (let alone understand
macros others write), and boils it down to the essence with the color-coded
schematic to match his environment and visualize his thought process. I dare
say Rich Hickey et al would be impressed, tipping the hat to the Simple or
Easy talk he is famous for.

If I have bored you at this point, you will probably not check out Rackjure.
Someone basically implemented a language subset in Racket of Clojure. So it is
safe to say you can subsume Clojure with Racket.

I know we are all smug Lisp weenies, but the whole Brown/NEU PLT group who
works on Racket deserve serious praise. I listen to all the core devs and
watch their talks, because they push the boundaries of what a good computer
scientist is.

They just happened to choose Scheme/Lisp to make me feel dumb. Watch them
ditch it and go for Haskell. We will all be sorry then.

~~~
pjmlp
> Racket is all the promise of developer-centric power tooling that puts
> Common Lisp to shame.

Have you ever used a commercial Common Lisp like Allegro ?

~~~
nickpsecurity
We cant entirely blame them: those cost significant money while mainstream
stuff usually has free IDE's that are good without limitations. Allegro even
mentioned royalties last time I looked at them. Royalties!?

Only natural hobbyists overlook this. But, yes, LispWorks and Allegro are very
powerful environments. Comparing AllegroCache to Hibernate might be fun for
newcomers too. Haha.

~~~
pjmlp
I guess we were spoiled by being able to have seen such environments, back
when it was common to pay for software tools.

Even Macintosh Common Lisp would already have been a good experience.

[http://basalgangster.macgui.com/RetroMacComputing/The_Long_V...](http://basalgangster.macgui.com/RetroMacComputing/The_Long_View/Entries/2013/2/17_Macintosh_Common_Lisp.html)

~~~
nickpsecurity
That was pretty interesting. Especially that the Mac CL survived so long in
about the same form. The one thing that confused me was Apple acquiring
"Allegro" Common LISP. That's name of Franz's product. History of Franz says
nothing about Apple. I'm assuming they're different products with the same
name just to trip us archaeologists up?

~~~
lispm
Coral Common Lisp was once renamed 'Macintosh Allegro Common Lisp', because
Coral went into a marketing agreement with Franz Inc. Apple later bought Coral
and its products, and published MACL then as Macintosh Common Lisp (MCL).

MACL had technically nothing to do with Franz' Allegro CL.

~~~
nickpsecurity
Now that makes sense. I just couldnt see one not saying something about other
with same name. Tks for the tip.

------
qwertyuiop924
I'm not a fan of Racket, as I've made clear elsewhere. Its insistance on using
as many paradigms as possible frustrates me, and while I like "batteries
included" in general, I wish some things like contracts, custodians, OO
system, etc. had been moved out to other libraries to make core easier to wrap
your head around. I am also not a fan of syntax-case: it violates the macro
abstraction, and is overly complex for what it does, IMHO.

In essence, I wished that core had picked a paradigm and stuck with it,
instead of like 10 of them.

Practical or no, I'd take a simpler Scheme over Racket any day. Usually
Chicken, which is quite practical, and has a lighting fast call/cc
implementation (non-delimited, with delimited implemented in terms of call/cc
by an external library, if you want it).

But just because I don't like it doesn't mean you won't: objectively, it's
powerful, technically competent, and fairly well designed. It's just not to my
tastes, any more than CL is, and so it looks like I'll have to wait until R7RS
Large before I get a large lisp I like.

And yes, I said lisp, not scheme, because Scheme _is_ a lisp, and also to
annoy the people who say otherwise.

~~~
mballantyne
Can you elaborate on how syntax-case "violates the macro abstraction", and
maybe what you think is important about the quality it breaks? Not a
description I've heard before. I do agree it is complicated!

~~~
qwertyuiop924
The macro abstraction is that the AST is exposed as a series of lists, which
is what macros process. Instead, syntax-case exposes 'syntax' objects, which
are entirely distinct from any other datatype, and kind of complicated.

This isn't a horrific sin (new macro abstractions pop up like rabbits,
although they rarely change something so fundamental, at least in Lisp mavro
systems), but if you're going to replace an abstraction everybody knows and
understands with a more complex one, you better have a good reason, and I
don't think the benefits outweigh the costs.

~~~
ufo
I thought that the syntax objects resulted in better error messages, due to
keeping track of source code position, lexical bindings for variables and so
on. Isn't that worth the complexity?

~~~
junke
Talking about CL, all the things that are done with syntax objects are done
with environments (`&environment`). Type declarations, lexical bindings, etc.
You can even process the same code in two different environments if this is
useful. Other bindings from symbols to values can be stored in the symbols
themselves: things like original code source (as a string; I recovered deleted
code thanks to that in the past), original location, custom properties, etc.

~~~
qwertyuiop924
So like the sc macro system, but more featureful. In theory, the sc macros
could record this sort of data (maybe), but I don't think they record as much
as syntax objects.

------
shiro
A kind of nitpick, but the use of dynamic-wind with custodian in the slides
bothers me. Dynamic-wind shouldn't _directly_ be used to ensure cleanup, since
a continuation captured in the body may be invoked outside from dynamic-wind
and the body restarts, even if resources are gone.

In the R5RS age, light users looked for try-catch construct in Scheme and
could only find dynamic-wind and used it; but they're not the same. Dynamic-
wind is a low-level construct that can be used to implement try-catch idiom
and other stuff.

Now Scheme has guard, so it's better to use them to write cleanup stuff. It's
a bit verbose, though (you have to write cleanup both in the handler and the
end of body), so I personally wrote unwind-protect macro on top of it and am
using it.

------
raspasov
I really don't get the title of this. Why is it "beyond Clojure"? Maybe if we
can see the actual talk it will be more clear.

~~~
smnplk
Clojure ideas are beyond Clojure.

------
Ericson2314
I need types. But Racket has the world's greatest macro system and everybody
should imitate it, Period.

~~~
gus_massa
Have you tried Typed Racket? [https://docs.racket-lang.org/ts-
guide/](https://docs.racket-lang.org/ts-guide/)

~~~
Ericson2314
Somebody always asks :).

1) I want my libraries to be typed all the way down.

2) Improving the implementation of typed racket is harder/less likely than
fixing the macro systems of Rust and Haskell.

3) Network effects of those two languages.

------
edem
How I am supposed to read the slides? Top to bottom, left to right? It is not
intuitive.

~~~
kd0amg
Top to bottom. The "top row" of slides are section headers. Go down from a
section header to see the section. When you get to the bottom of a section, go
right, and it will put you at the top of the next section.

------
avodonosov
I don't understand - is it just 8 slides? That's all?

~~~
blandflakes
I don't understand why this slide format is popular, but I believe you go down
until the end of that column, then right, then down again, then right, and so
on.

~~~
antfarm
Or just press space to go to the next slide.

------
junke
Let's try with other languages:

    
    
        Rust - C beyond Go
    

See any problem?

~~~
imagist
Lisp is a family of languages that includes Racket and Clojure.

C is a language, not a family of languages that includes Rust and Go.

The Lisp family of languages also contains Common Lisp. In some contexts, it's
reasonable to assume "Lisp" means "Common Lisp". This is not one of those
contexts.

~~~
junke
Newcomers are always confused by the distinction, this doesn't help at all to
put them in the same big bag. This classification is not wrong, but mostly
useless when you see how the different "dialects" evolved (they are grown-up
languages, nowadays).

> C is a language, not a family of languages that includes Rust and Go.

But Rust and Go are constantly compared to C. We always talk about C-like
languages. Maybe I should have said "Algol" instead, but there is a family of
languages rooted at "C", and we don't call it the C family.

Go is "a compiled, statically typed language in the tradition of Algol and C".
"The syntax of Rust is similar to C and C++", but "Rust is semantically very
different from C and C++.[citation needed]" (wikipedia).

Likewise, Scheme is semantically very different from Common Lisp. And Racket
is not considered as a Scheme, even though it is related to it. You can't
copy-paste any Typed-Racket expression and run it with ChezScheme.

So, yes, they all belong to the same proto-language family, but constantly
referring to them as a unique family is doing a disservice to each of them.

Take Clojure for example: the wikipedia page says that it is inspired by C++,
C#, Common Lisp, Erlang, Haskell, Mathematica, ML, Prolog, Scheme, Java,
Racket and Ruby. There are plenty of influences that goes into a language, why
not talk about the other ones? parentheses?

~~~
imagist
> Newcomers are always confused by the distinction, this doesn't help at all
> to put them in the same big bag. This classification is not wrong, but
> mostly useless when you see how the different "dialects" evolved (they are
> grown-up languages, nowadays).

Sadly, language is determined by usage, which only incidentally correlates to
usefulness. And critically, there's a feedback loop here: a word is only
useful as it's used, because otherwise people don't know what it means, and
you've failed to communicate your intent. "Lisp" might _hypothetically_ be
more useful if it meant "Common Lisp", but it's not useful for meaning "Common
Lisp" (at least not on Hacker News) because when you say "Lisp" people assume
you're talking about all the languages with the parentheses, and you've failed
to communicate.

> But Rust and Go are constantly compared to C. We always talk about C-like
> languages. Maybe I should have said "Algol" instead, but there is a family
> of languages rooted at "C", and we don't call it the C family.

Right, the terminology commonly used for that is "C-family languages", not
"C".

> Go is "a compiled, statically typed language in the tradition of Algol and
> C".

Note how they don't say "Go is a C".

> Likewise, Scheme is semantically very different from Common Lisp. And Racket
> is not considered as a Scheme, even though it is related to it. You can't
> copy-paste any Typed-Racket expression and run it with ChezScheme.

Yes, but in a context not centered around Common Lisp, people use "Lisp" to
refer to all these languages. You can argue whether that's good or bad, but
you're not able to change it.

> Take Clojure for example: the wikipedia page says that it is inspired by
> C++, C#, Common Lisp, Erlang, Haskell, Mathematica, ML, Prolog, Scheme,
> Java, Racket and Ruby. There are plenty of influences that goes into a
> language, why not talk about the other ones? parentheses?

Parentheses, macros, some functional programming constructs.

Look, I'm not saying the terminology is ideal. I'm saying that, in this
context, "Lisp" doesn't mean what you and I want it to mean, and trying to
change that will a) fail to communicate and b) fail to change the meaning of
the word in this context.

~~~
Jach
I'll just leave two of my old comments here...
[https://news.ycombinator.com/item?id=10207199](https://news.ycombinator.com/item?id=10207199)
and
[https://news.ycombinator.com/item?id=3423646](https://news.ycombinator.com/item?id=3423646)

Short summary: I do think on HN it's reasonable to assume "Lisp" without
further qualification to refer to the lisp family. The usefulness of the lisp
family idea (which afaict is having sexps and maybe something like defmacro)
is questionable. Among Lisp and lisp-family practitioners outside of HN and
maybe even on HN itself (it'd be interesting to see a poll result) Lisp means
Common Lisp, as it has basically since Common Lisp was created, and the
distinction with other lisps usually made by saying things like "Clojure is a
Lisp", that article "a" being important here to say x is a member of the y
family and would be nice to have in the slide title.

I agree with your argument about common usage but I'm not sure what the common
usage on HN really is and what's a vocal minority using it incorrectly until
it becomes common. But we've already lost several wars like that, in the wider
culture ('literally' for one..) and in tech (a recent one that bugs me being
'isomorphic' JavaScript) so it's probably best to sigh in resignation and
maybe think/complain about the lisp/lisp family stuff once every few years at
most. ;)

~~~
imagist
> I agree with your argument about common usage but I'm not sure what the
> common usage on HN really is and what's a vocal minority using it
> incorrectly until it becomes common.

I just don't think "correctly" and "incorrectly" are words that have meaning
when it comes to semantics. I care more about your goals. If your goal is to
have "Lisp" mean "Common Lisp", then you can try to enforce that, but it's a
strange goal to fight for, and probably not an achievable one.

Personally, my goal is to understand what people are saying and communicate my
own ideas effectively. Toward that end, I just say "Common Lisp" or "CL" when
I mean Common Lisp, and Lisp-family languages or Lisp-like languages when I'm
talking about the larger group. And when reading, I use context clues to
figure out which the person is talking about.

Certainly in the OP, it's clear that "Lisp" refers to the s-expression-y
languages.

~~~
Jach
Here are some ways words can be incorrect:
[http://lesswrong.com/lw/od/37_ways_that_words_can_be_wrong/](http://lesswrong.com/lw/od/37_ways_that_words_can_be_wrong/)

In the end it's not a very fulfilling battle even if won, to preserve the
meaning of a word. I always liked the line "In the face of ambiguity, refuse
the temptation to guess" as a guide to be more explicit and ask for
clarification in general communication.

~~~
imagist
Yeah, 17-20 in that link do a better job of expressing what I wanted to
express than I did. :)

