
Luau – A fast, small, safe, gradually typed scripting language derived from Lua - panic
https://roblox.github.io/luau/
======
JHelmsAt
In case anyone is interested: Roblox (who is making Luau) is hiring.

We've been growing fast, and are looking for strong C++, systems, and
application engineers. As a highly vertically integrate gaming platform, we've
got a lot of hard problems to solve - from programming language design (see
OP!) to debugger tooling.

You can email me at my username plus roblox.com.

~~~
andrewmcwatters
Hi JHelms, I'm the author of the largest pure-Lua game engine on GitHub,
Planimeter's Grid Engine 9 ([https://www.planimeter.org/grid-
sdk/](https://www.planimeter.org/grid-sdk/)). I'd love to chat, and sent you
an email.

------
tomp
I wonder how the interpreter works. In their _Performance_ page they write:

 _> It’s similar to Lua interpreter in that it’s written in C, but it’s highly
tuned to yield efficient assembly when compiled with Clang and latest versions
of MSVC. On some workloads it can match the performance of LuaJIT interpreter
which is written in highly specialized assembly._

One thing I've been experimenting for my own project, is using tailcalls. All
main compilers (gcc, Clang, MSVC) are able to compile that into _actual_ tail
calls (i.e. `jmp`, not `ret`, so that the stack doesn't overflow) (with
optimizations enabled), even for indirect calls (i.e. dispatching depending on
the next instruction). Function parameters (VM state) are assigned to
registers, alleviating Mike Pall's main complaint about writing interpreters
in a high-level language, that the compiler cannot optimize the huge loop as
well as a human can. I need to test this on a full interpreter with 50 or so
instructions, to see if the optimizations still apply.

~~~
rapidlua
I am currently exploring the similar approach. The idea is to compile to llvm
IR and then augment it. There's a musttail marker in the language to enforce
tail calls no matter what the optimisation level is.

Passing VM state via arguments is suboptimal since argument registers are
often clobbered and one would normally use callee-saved registers. I have
developed a patch for llvm allowing to specify registers used for passing
arguments and making more registers available for allocation without spilling.

I intend to reimplement LuaJIT VM in C with IR augmentation to verify the
performance.

~~~
tomp
Did you check out the "GHC calling convention"? They implemented it
specifically for this purpose.

[https://llvm.org/docs/LangRef.html#calling-
conventions](https://llvm.org/docs/LangRef.html#calling-conventions)

My understanding is, for "fast path" instructions (e.g. integer addition)
there's no register spilling necessary, so passing VM state in registers makes
sense. For more complex instructions, you rely on LLVM to optimize registers
as it would normally, with callee-saved registers. Of course, one must still
pass as little state as possible - e.g. just sp, cp, maybe top of stack /
heap, and then a pointer to *vm_state for all the rest.

~~~
rapidlua
> Did you check out the "GHC calling convention"?

GHC allocates registers for arguments from the fixed list where most but not
all registers are callee-save in other common CC-s. This is a big improvement;
still there are valid reasons to want it customisable.

E.g. in LuaJIT next instruction is decoded as a part of an instruction handler
before dispatching to the next handler. Instruction arguments are passed to a
handler; they shouldn't use callee-saved registers.

Another complication is the stack frame. There's a requirement to keep the
stack pointer aligned (16 byte on x86_64). Call instruction pushes the return
address and the pointer becomes misaligned. If the called function wishes to
call further functions it has to adjust the stack pointer to make it aligned,
even if it doesn't use the stack. The stack pointer is adjusted in a function
prologue, hence there's a slowdown even if a nested function call is on a cold
path.

Whether these micro-optimisations are worthwhile remains to be seen.

~~~
tomp
I'm not sure I understand your complaint about callee-saved registers. The
point is that _arguments_ are passed in registers, as opposed to on the stack,
and most calls are tail calls anyways (so `jmp`, not `call`/`ret`). It doesn't
matter if the registers are caller-saved or callee-saved, as the context is
destroyed anyways, the function never returns.

In case of GHC, the functions _could_ return, but the fast-path is that most
functions access VM state, and many functions have very little need for
registers; therefore, forcing registers be caller-saved would just cause most
functions to push and pop the stack without any reason. If non-standard
calling convention (e.g. C calling convention) functions are called only
rarely, then using caller- or callee-saved registers doesn't really matter.

~~~
rapidlua
We are on the same page irt tail calls. I assume that instruction handlers
(non-standard calling convention, tail-calls next handler) need to invoke
helper functions quite often. These helpers adhere to standard C calling
convention. VM state is in registers. Unless they are callee-saved, registers
have to be saved to stack and restored after each helper call. (LuaJIT VM uses
helper functions heavily.)

------
zeckalpha
Too bad MetaLua never took off, these could be macros embedded in Lua rather
than a separate language.
[http://metalua.luaforge.net/index.html](http://metalua.luaforge.net/index.html)

~~~
fsfod
It may not be exactly the same but Terra
[http://terralang.org/](http://terralang.org/) uses Lua a meta programming
language and the Terra language itself is effectively a typed Lua compiled to
machine code with LLVM.

~~~
infinite8s
Terra looks really interesting (Halide is another language along this meta-
approach - [https://halide-lang.org/](https://halide-lang.org/)). It's too bad
it never made it further than being used for a few PhD theses.

~~~
eggy
I like Terra, and I have played with it, but I have not found a use for it to
push me to really explore it. I'd love to hear from anyone using Terra for
cool projects. I had watched a talk about MAD-X [0] that I found very
interesting, but it is specifically for particle accelerator design in physics
[1].

[0]
[https://mediastream.cern.ch/MediaArchive/Video/Public2/weble...](https://mediastream.cern.ch/MediaArchive/Video/Public2/weblecture-
player/index.html?year=2016&lecture=487416c0&ftime=00:14:11)

[1] [https://mad.web.cern.ch/mad/](https://mad.web.cern.ch/mad/)

------
Jyaif
"Please note that at this time we are not ready to make Luau implementation
publically available in the source form"

1/ Code source is not available, so it feels like the purpose of the page is
just to brag about how good your interpreter is.

2/ s/publically/publicly/

~~~
whizzter
Not necessarily, since it's in production they might be hoping for a bit of
securty-by-obscurity if they haven't had time to get an audit the code
(runtimes are always hard to get 100% correct since they're so generic, esp
for a language such as Lua).

Seems like this runtime was announced to Roblox developers almost a year ago.
But i don't think they've put much thought or effort into open sourcing it and
it just happend to pop up in peoples feeds now since Arseny published an
article(#1) about his 8 years working for Roblox 2 days ago and somebody
probably took the link from that article (Quite an interesting read if you're
into gamedev).

#1 , [https://zeux.io/2020/08/02/eight-years-at-
roblox/](https://zeux.io/2020/08/02/eight-years-at-roblox/)

------
nafey
Possible to make this work on LuaJIT as well? If this is transpiled to Lua 5.1
it should be possible to make this work with LuaJIT runtime. Combined with
Openresty and gradual typing this could be a quite a good language for writing
Backend services.

~~~
minxomat
Take a look at teal, it does exactly that.

~~~
nafey
Any links? Teal appears to be an assembly language from my google search.

~~~
durub
Here it is: [https://github.com/teal-language/tl](https://github.com/teal-
language/tl)

~~~
nonbirithm
It's also made by the same person who created LuaRocks and htop.

I'm personally hoping tl or something will take off as the de-facto TypeScript
for Lua. I'm already convinced that types are better. I'd use types all the
time if I could. It's just that not a whole lot of people seem to care about
Lua, even with how fast LuaJIT is, so you have to compromise with projects
adding types to Lua that are only in their infancy, assuming they'll still be
around after a couple of years. TypeScript has proven that gradual typing can
be done right and had a bunch of innovations I never would have thought of
(the keyof operator for creating a type of "the set possible key names of T"
for instance). Having that kind of power in Lua on top of one of the fastest
JIT compilers on Earth is my dream.

~~~
minxomat
Together with Teal, LuaJIT can become a very safe platform. Vela (fork with
many improvements to LuaJIT) has optional, native recursive immutability built
into the JIT for Lua-land objects and also compiles next() to dig pairs out of
the nyi hole.

Also Moon+ (even more expression version of MoonScript) is considering
implementing the teal checker natively:
[https://github.com/pigpigyyy/MoonPlus/issues/15](https://github.com/pigpigyyy/MoonPlus/issues/15)

------
dman
Lua runs the same risks as Scheme which is another small language - of
multiple incompatible implementations cropping up because it is so easy to
create an implementation. [Dont mean to downplay the difficulty of writing an
implementation just to highlight that the engineering cost of doing so is not
prohibitive enough]

~~~
xanathar
I was almost semi-proud of having done my implementation, why did you ruin the
party.

Jokes aside, and not to boast about what I did, honestly, but Lua is (in
hindsight - I wish I realized before) full of gotchas which are definitely not
easy to implement (just one word which still makes my stomach hurt:
COROUTINES).

I honestly think there are simpler (and better) languages to target. With
hindsight, experience and pain, of course.

~~~
dman
You have every right to be extremely proud of what you accomplished - writing
your own compiler is extremely gratifying and you have my respect!

------
looki
The front page sample does not explicitly use the Point type in the
declaration of p. Typo or does the compiler actually match table variable
definitions to available type definitions?

------
jdonaldson
Very happy to see this, Lua has a bunch of great properties (small footprint,
speedy JIT, simple datastructures). It's great for quickly hacking together
some script functionality, and it (already) handles enormous scale in
enterprise applications (e.g. Cloudflare).

I maintain the Haxe Lua target, and have a good understanding of how Lua
relates to other languages (at least through a "Haxe" lens).

I think the main things I get hung up on is the 1-indexing scheme, and the
lack of true "null" fields on tables (Lua treats these as missing). This makes
Lua tricky to integrate with other languages, especially if they serialize/de-
serialize data. You essentially have to pick a work-around, and unless you
know what you're doing (and what you're doing 5 years from now), you're going
to feel a lot of pain.

I realize changing these things would make Lua less "Lua", but virtually every
other language Haxe supports has straightforward support for these behaviors.
It's really the only two nits that have marred an otherwise pleasant
experience working with Lua.

------
eggy
Interesting. Seems like a Typescript for Lua. The Roblox platform is a great
testing platform for it I am sure.

~~~
giancarlostoro
I gave this game a try last night, it was amusing all the different types of
games you could potentially play, I was wondering who developed each one. It
seems you can build your own games (game modes? unsure of the term for Roblox)
and earn some income on it. I had no idea it was done in Lua, might give it a
shot to see what is possible.

~~~
kitd
My son spends his life on Roblox. If you're ever after beta testers? ;)

~~~
exikyut
Hmm, that's a good idea - connecting interested kids directly with Roblox
developers and letting them play around in/QC alpha-stage worlds.

That would teach the value of iteration, failing forward and how to deal with
imperfection at a very early age.

Of course reality is not a vacuum and there would be a lot of noise though :)

------
otikik
If this sort of thing interests you, you might want to look at

[https://github.com/teal-language/tl](https://github.com/teal-language/tl)

It implements types on top of Lua. In a single (middle-sized) Lua file. And it
is open source.

------
tcbawo
I like the mention of an internal tool called "SECRET TOOL". It makes me
wonder why the name would be redacted, although I'm amused by the idea of that
actually being its name.

~~~
akavel
Don't know Roblox, but I guess this could be also a name of some kind of an
in-game-world tool? I.e. like a sword or pickaxe?

~~~
hannula
As a Roblox user, I believe it might be their tool that allows Luau code to be
stored on the filesystem and automatically synced to games. There are
references to "FileSyncService" in the binaries, but the engineers refuse to
open it up for some reason.

------
shaggie76
I'm curious what they do to the garbage collector; over the last few years
we've made quite a few optimizations to our Lua 5.1 codebase to reduce the
impact it has and yet just last week I was in there again squeezing blood from
that stone.

~~~
pansa2
What problems have you had with the garbage collector? Excessive pauses?
Insufficient throughput?

Many other game-focused scripting languages use reference counting instead of
tracing GC to eliminate pauses at the expense of throughput. Would that help
in your situation?

~~~
shaggie76
We use a hybrid of ref-counting for many things but Lua still needs to do
sweep phases in its GC that cannot be amortized across multiple frames. While
we do most of Lua's GC asynchronously some of the phases take so long that
they can stall the main thread that wants to execute some bytecode.

Here's an older tweet with some examples (I'm not the one tweeting but I made
the optimizations):

[https://twitter.com/sj_sinclair/status/1097920762147733506](https://twitter.com/sj_sinclair/status/1097920762147733506)

------
pansa2
> Note: our garbage collector optimizations are still in progress, so this
> section doesn’t document them.

It will be interesting to see whether they make any major changes to Lua's
garbage collector, which is an incremental tracing GC.

Several gaming-focused scripting languages use reference counting instead of
tracing GC, even though throughput is lower. This is because RC (almost?)
eliminates the need for stop-the-world pauses, making it easier to maintain a
consistent frame rate.

------
DarkWiiPlayer
So another typed Lua, huh? What are the main differences to teal?

~~~
implicit
Hello! I am the lead developer of the Luau type checker.

I actually hadn't heard of Teal! I don't think it existed when we started.

Judging from the Teal documentation, I'd venture that it looks pretty similar
to Luau in everyday use.

If I were to guess, I'd venture that the biggest differences are probably the
kind of type inference we do (Luau's inference engine draws inspiration in
equal parts from OCaml and TypeScript), a bunch of Roblox-specific features,
and that our inference engine is all C++ for performance.

~~~
DarkWiiPlayer
Oh, that's interesting :D

I've been meaning to try out roblox for a while and lately I've been getting
pretty hyped about teal (for context, the guy behind it is the creator of
luarocks and htop, among others) so I might see the differences up close
sooner rather than later :D

EDIT: (unrelated) Already not looking forward to people asking "Lua" questions
about typing and constantly having to explain that Lua is not just a roblox
thing xD

------
HighlandSpring
Missed opportunity to call it Lau (Lua was sometimes called such jokingly in
the Garry's Mod community)

~~~
andrewmcwatters
Created by "John Lua," another Facepunch gag.

------
monkeypizza
Video summary of the optimization process by the designer:
[https://www.youtube.com/watch?v=vScM-
nk5Avk](https://www.youtube.com/watch?v=vScM-nk5Avk)

------
boring_twenties
Can someone ELI5 why Lua is so popular?

I don't know anything about it, but just looking at it, it seems like a giant
mess. 1-based indexes. Semicolons required here, but forbidden there. Like...
why?

~~~
samatman
I use Lua(JIT) extensively.

cons:

\- 1-based indexing

\- you basically always want a local but you have to ask for one, it's a bit
noisy. 'strict mode' is easily applied to the global environment, and catches
most outright errors here.

\- there is an adequate amount of library and extension code, but less than
you'd expect coming from Ruby or Python. The lack of a blessed object system
sometimes means taking more effort to understand what other people's code
does.

pros:

\- variadic functions and multiple return values make for a truly dynamic
language

\- first class functions which are fast and work well: everything is a
closure, a whole file is just an anonymous function called a chunk

\- the table as a single compound data structure works very well, and
metatables are a unique and excellent solution to extending behavior. Array
portions are required to be compact (non-nil from 1 to #tab), which can be
worked around on the few occasions when it's inappropriate.

\- asymmetric coroutines.

\- first class environments, which are just, yep, tables. this allows for a
few really nice things.

\- fast. LuaJIT is very fast. Both Lua runtimes fit in a typical L1 cache, and
it uses a register VM, which is a good fit for real processors vs. stacks. No
global interpreter lock, all C level Lua functions receive a Lua_state as the
first argument and are fully reentrant.

\- in general the emphasis on minimalism means the whole language "fits in
head", and the tools provided are more than adequate to get jobs done.

\- the LuaJIT FFI, which parses C declarations directly, is simply a pleasure
to use, and is the only one of which I'm aware where you pay an absolutely
minimal penalty to use it. There's just nothing like it. More languages need
to have this.

miscellaneous:

Semicolon insertion is simply a non-issue, unlike JS it was implemented
properly. I have well under one semicolon per file, less than one per project,
even. It's very, very occasionally nice to write more than one statement per
line; that's what the semicolon does.

It's a nice language.

~~~
lifthrasiir
> [...] and multiple return values make for a truly dynamic language

If you are using a word "dynamic" as "once-good-but-now-increasingly-
considered-worse-and-worse", I agree. Lua is definitely too dynamic to be
safely used.

Weeks ago I had to debug a Lua code looked like this:

    
    
        commands = commands_template:gsub("<arguments>", arguments:gsub(" ", "\\ "))
    

If you can catch a problem, great! But most wouldn't be able to do so. And I
used Lua for a prolonged time, to the extent that I have made a type checker
which absolutely flags this case [1] and developed a very good list of
intrinsic problems of Lua which obviously includes this one and _still failed
to notice the problem_. Lua is a small language, but packed with lots of
similar problems.

[1] [https://github.com/devcat-
studio/kailua/blob/323caab59c06230...](https://github.com/devcat-
studio/kailua/blob/323caab59c06230af7ee22835a681064de0ab165/kailua_check/src/check.rs#L2668-L2669)

~~~
samatman
What is that even supposed to do?

I tend to use lpeg for any complex string manipulation. gsub has bitten me
once or twice by not thinking through whether something is a pattern nor not,
sure.

~~~
samatman
But, no: I meant that after the fiftieth time I had to modify every call site
of a Python function because I realized I needed to return a tuple rather than
a scalar sometimes, I came to _really appreciate_ multiple return values.

~~~
lifthrasiir
If that's the only real reason why do you keep using dynamic languages after
all?

------
forrestthewoods
Is Luau available for other developers to use? Or is this a secret VM exposed
to Roblox?

Put another way, if I don’t play Roblox is there a reason for me to care?

------
varshithr
[Offtopic]: The name Luau reminds me of an episode from It's Always Sunny in
Philadelphia :)

------
bakuninsbart
Most important question not asked or answered: Is it pronounced Lua-you or Lu-
au?

~~~
implicit
Author here. :)

It's Lu-au. It's like a party in your development environment!

------
G4BB3R
Why not local-by-default? And have a global keyword instead?

------
Symmetry
Seems much less of an invasive change than MoonScript is.

