
Writing a DSL in Lua (2015) - ColinWright
https://leafo.net/guides/dsl-in-lua.html
======
leafo
Hey, I'm the author of this blog post. Thanks for posting it whoever did.

I wrote this a long time ago, it's mostly targeted at Lua 5.1 but more recent
versions of Lua approach function environments differently, so keep that in
mind.

I've written & worked with quite a few HTML generation DSLs at this point. I
put them into two groups:

1\. Nested object return: the object is converted into HTML after executing
the template

2\. Evaluate and write to buffer: no return values, each "tag" is a function
call that generates code in a background buffer (nested html typically
accomplished by blocks or inline functions)

Approach 1 is what was used in this blog post. Approach 2 is what I ended up
using for my web framework:
[https://leafo.net/lapis/reference/html_generation.html](https://leafo.net/lapis/reference/html_generation.html)

Approach 1 is used by React. Approach 2 I first came across in a library
called Erector:
[http://erector.github.io/erector/](http://erector.github.io/erector/)

I prefer approach 2. It lets you use your programming language constructs to
do things conditionally, in loops, or whatever else your language provides.

I find that with approach 1 you tend to try to bend the language to convert
everything into an expression that can be returned into the nested object
you're building. The example I have off the top of my head is how React devs
will write some pretty nasty ternary operator expressions just to write some
HTML conditionally directly inside of a JSX chunk. (The alternative would be
pulling things out into temporary variables, which just creates a lot of
noise)

As a fun aside for how far you can take things, for my implementation of
approach 2 in my web framework, I ended up writing a syntax transformer that
could pre-render static HTML chunks to plain sting that can be appended to the
output buffer. Essentially removing those function calls and any HTML escaping
that would be necessary during evaluation time of the template.

~~~
wruza
Also, Dart has syntax for making ifs and loops in collection literals. Called
collection if and collection for. If only we could bring all really good
features into a single mainstream language and focus on creativity instead of
routine. Sigh.

[https://dart.dev/guides/language/language-
tour#lists](https://dart.dev/guides/language/language-tour#lists)

------
wruza
Function environments, resumable coroutines and dsl-friendly syntax is what
makes Lua feel so superior to js (in a sense of the language power, not in
syntax, optimization or minor design details). It is a shame that browser
vendors said “nah, too hard” for coroutines and resorted to generator hack for
asynchronous operation – and no one reputable complained! I don’t get it why
the execution model should suffer just because someone finds it tedious to
split their 300 C calls to few continuation callbacks.

The entire JSX thing could be replaced by something similar, seamlessly
connecting code and hyperscript. Callbacks could operate directly on “this”,
like in “onclick() {count++}” without making every damn “count” a silly
lexically visible [count, setCount] couple. Every missing conceptual feature
in js is a total collapse of an entire dimension of possibilities. And yet it
is “a language of the future”. How we got here?

~~~
kragen
The reason explicit asynchrony produces more reliable software than implicit
cooperative threading like Lua coroutines is explained at book length in the
doctoral dissertation of Mark S. Miller:
[http://www.erights.org/talks/thesis/markm-
thesis.pdf](http://www.erights.org/talks/thesis/markm-thesis.pdf)

He is one of the main participants in the ECMAScript committee.

If you disagree with his arguments, please explain why. But you don't seem to
have heard of them in the first place, simply assuming that the reason JS
isn't designed the way you would have designed it is because the designers
weren’t as smart as you are, explaining their choice as “someone finds it
tedious to split their 300 C calls to few continuation callbacks.” Sometimes
people will disagree with you for reasons other than being stupid or lazy.

~~~
oconnor0
Would you be willing to explain why explicit asynchrony is superior to
implicit cooperative threading? Rather than simply engaging in more
namecalling.

~~~
kragen
Mutable state is always a problem, but it's a bigger problem when there are
less constraints on when it can mutate. But Mark's dissertation explains this
and some related issues a lot better.

Also, I don't think I did any name-calling?

~~~
wruza
>Mutable state is always a problem, but it's a bigger problem when there are
less constraints on when it can mutate.

A very long-time project I’m idly working on with my life-long colleague
addresses this issue. It is real, as are all race conditions and threading
issues. But it doesn’t have to be solved at the language level.

One of solutions we proposed to ourselves was that an object storage should
manage shared access conflicts. You may see it in Redux and other stores that
serve a “frontend framework backend layer” duty today. Even with explicit
async, you will always have some high-level races that must be dealt with. And
these do exactly that.

The basic idea is that local state is strictly local and not a subject of
parallel mutation, and shared state is shared via internally
locking/serializing controller who knows better how to mutate, merge or throw.
It doesn’t push the problem down the code or execution model, since it’s
what’s it for.

~~~
kragen
Sounds like you'd benefit from reading Mark's dissertation. Not that he
_solved the problem_ but he did try a lot of candidate approaches, discuss
them with others, and comprehensively review the existing literature; and he
reports on several years of trying things similar to what you're thinking of.

------
giancarlostoro
This makes me want to learn Lua just so that I can see what kind of things I
can do with it. There's a DSL in Kotlin for HTML as well that looks
syntactically similar. I have not dived enough into DSL land yet, but this
looks kind of fun.

~~~
frabert
Rust's lexical macros look even more interesting:
[https://github.com/bodil/typed-html](https://github.com/bodil/typed-html)

~~~
1MachineElf
The same could be said about anything Bodil Stokke publishes.

------
jacobwilliamroy
As I've been learning more CLISP, posts like these just become more and more
confusing to me. LISP is every language that ever has or will exist. Some
macros are basically mini programming languages in their own right. I've seen
too much. Everyone needs to learn LISP now.

------
runawaybottle
I feel like if you use React right, it’s a pretty good for making DSL-like
stuff.

~~~
kroltan
As part of a previous job, I had to redesign the infrastructure monitoring
dashboards, but shenanigans did not allow me to install different software on
the dashboard server. So I wrote a React app that uses a custom renderer,
which, instead of writing to DOM or to a HTML string, renders into the
configuration format of the application.

We used a more traditional enterprise "serverful" architecture, so information
about services and hosts was important for availability purposes.

I do not recall which was the application, but I could then export that
compilation output via API to update the dashboard with new services.

I used a few layout components wrapping Facebook's Yoga layout library (a
library to solve flexbox layouts), and put them into high-level components,
such that the infrastructure guy could edit a plaintext file and run the
compilation to update the dashboard.

It was basically glorified XML, of course:

    
    
        <Host id="poseidon">
          <Service name="GlusterFS" id="gluster"/>
          <Service id="IDK" depends="gluster,somethingElse"/>
        </Host>
        <Host id="zeus">
          <Service id="somethingElse"/>
        </Host>
    

And it would generate a chart grouped by host, with arrows connecting any
depends-marked elements. It would also validate the resulting graphs, so you'd
get feedback if you missed declaring a service you depended on, and so on.

Conclusion: was a fun "paid side project", but it's way too much effort. In
retrospect using a proper XML (or even simpler markup) engine might have been
a bit quicker and did the job just as well. Plus, the dependency on JS tooling
at "runtime side" is very annoying.

------
kragen
This is beautiful. I never thought of chaining the parenthesis-free syntax or
using table arguments for a sequence like that.

~~~
pb82
You can accomplish something similar in C++: single argument constructors let
you skip the parentheses but that single argument can be an initializer_list,
for example 'Tag(std::initializer_list<Tag> children)'. Now create subclasses
of Tag for actual tags and you end up with a DSL that looks just like the one
in the article. There is of course a bit more to it, but it can be done.

~~~
kragen
Skip the parentheses? Do you mean like

    
    
        H1 x = "Topics";
    

Or is this an aspect of C++ I don't know about?

~~~
pb82
Yes exactly. And now if the constructor of your element, let's say a div,
doesn't take a string but a std::initializer_list you can write it like this:

    
    
      auto doc = Div {
        H1("Topics")
      };

~~~
kragen
Those look like parentheses though. Also braces, but the Lua version has those
too.

------
BiteCode_dev
While there are a few DSL that won the war, like CSS or SQL, it is very
unlikely that your DSL will prove worth it. 99% of the time, what you need is
a good high level library/API to deal with the problem with your current
technology.

Here is why:

First, you need to design it properly, and most devs are not good language
designers. The chances your DSL is going to be good is very low.

Second, unless you are incredibly talented and have little constraints, you
will not even remotely provide 1% of the tooling existing languages already
have for free. I've seen so many DSL without basic syntax highlighting nor
linter it's not even funny, but you need much more to be productive with a new
language. Even if you are using macros and claim to be able to use the tooling
for the main language you come from, stop a second and think about the last
time you had fun debugging a complicated macro magic.

Third you need to test it and document it. Do you even do it properly for your
regular code? It takes time and resources. A lot of it. Which could be better
spend on the project, or on the library/API you should be writing about.

Fourth, what about training? Because you can hire somebody for language X. But
for your DSL, you'll need to train them.

Then of course, you plan to support it for as long as it's used. Right? Right?

~~~
alberth
SQL is not a “DSL”.

It’s a “Declarative Programming” language and arguably the most popular one.

[https://en.m.wikipedia.org/wiki/Declarative_programming](https://en.m.wikipedia.org/wiki/Declarative_programming)

~~~
homie
a language being a DPL doesn't preclude it from being a DSL

~~~
alberth
I’m not certain that is true.

With a DPL, you describe what output you want without describing _how_ to
produce the output.

With a DSL, you have to describe how to produce the output.

~~~
dragonwriter
No, there is no requirement that a DSL be imperative rather than declarative.

