
Lua 5.4.0 - smlckz
https://www.lua.org/manual/5.4/readme.html#changes
======
pansa2
It would be interesting to know how many projects are using regular Lua vs
LuaJIT, as the two implementations have diverged.

When both were on version 5.1, there didn't seem to be any downsides to
LuaJIT. Many Lua projects could benefit from it as long as they weren't trying
to run on a platform that disallows JIT compilation (e.g. iOS or a game
console).

However, now that LuaJIT is 2-3 language versions out-of-date, and has
different best-practices (e.g. around FFI), it's almost like the two are
different languages.

~~~
fwsgonzo
Performance-wise Lua is not too far behind LuaJIT that you should just "never"
use Lua. I used Lua for a long time without problems, and in the cases where
you need that extra performance you are just one "system call" away from
native performance for that specific operation. I used to benchmark these
things (for my use cases) for a long time, and Lua was always somewhere
between 50-100% slower than LuaJIT, which is _NOT_ a lot. It's a mistake to
think that people implement Mandelbrot as a script callback. :)

It's unfortunate that LuaJIT fell behind so much, but as far as game engines
who need a scripting language goes: You are either safe with Lua, or you have
other alternatives: JavaScript, WASM, dynamic libraries and so on.

~~~
cellularmitosis
I recently redid the graphs of the luajit benchmark results to highlight
performance across architectures:
[https://gist.github.com/cellularmitosis/55ff55b55008b2f1c42b...](https://gist.github.com/cellularmitosis/55ff55b55008b2f1c42b4d0e46f58c76)

~~~
fwsgonzo
Nice work, man. I was doing microbenchmarks where I among other things measure
the 90ns (JIT) and 120ns (Regular) function call overhead of Lua. My
benchmarks was compared to my own scripting backend where the call overhead is
4ns, and it always feels a little bit like cheating when you have a gigantic
headstart with each test.

I'm not surprised the JIT isn't that good, but this is crazy almost! Did you
compare against the latest version of Lua?

[https://gist.github.com/fwsGonzo/f874ba58f2bab1bf502cad47a9b...](https://gist.github.com/fwsGonzo/f874ba58f2bab1bf502cad47a9b2fbed)

In my microbenchmarks LuaJIT was always faster than Lua, but not so much that
I wouldn't stop using Lua. In my mind once you stop developing a language it
stops being interesting, because you are always looking for ways to simplify
and remove whole classes of bugs.

EDIT: I just built lua5.4 from sources and ran my benchmarks again, and it's
markedly worse than LuaJIT still, but it's better than before for sure. My
conclusion is that Lua5.4 is the fastest PUC-Lua yet.

[https://gist.github.com/fwsGonzo/2f4518b66b147ee657d64496811...](https://gist.github.com/fwsGonzo/2f4518b66b147ee657d64496811f9edb)

~~~
anonymoushn
Some of these benchmarks are maybe not how one would do things if they were
trying to go fast. An unfortunate idiom I wrote a thousand times at a company
that uses Lua and tries to go fast is

    
    
      local n = 0
      local t = {}
      -- to do an insert
      n = n + 1
      t[n] = blorp
      -- to delegate some inserts to another function
      n = insert_some_stuff(t,n)
    

A less messy thing you could do is

    
    
      local table_insert = table.insert
    

but this is not as great, because lua will still do a binary search to find
the end of the array each time you insert something.

~~~
fwsgonzo
That might be, but the C++ array append is bounds-checked which makes it half
as fast as it could be too. It would be 16ns if there is no checking. So, you
are right but I am trying to do a balanced approach here. These benchmarks are
made for me, and I want to write (more or less) normal code.

I don't want to write local table_insert = ... I'd rather drop Lua for
something else then. That said, it's cool that there are things you can do to
speed things up if you really have to.

~~~
anonymoushn
Fair enough! I think t[#t+1] = x is not as offensive and will get you a few
nanos less than the way with two table lookups, but if that's also not to your
taste then that's fine.

------
ufo
This release brings many performance improvements, including a new garbage
collector.

In terms of language features, the biggest new feature are the "to be closed"
variables. They can be used to ensure that cleanup routines are always called,
similarly to RAII in C++ or try-finally blocks.

~~~
anonymoushn
<close> is equivalent to golang's defer (either can be implemented in terms of
the other) except at the block level. imo calling it RAII mostly leads to
confusion. One mailing list user recently asked a bunch of questions around
using <close> for RAII. They expected RAII to work even for resources that are
never bound to a particular type of local variable, for example if one writes
do_thing(io.open("foo.txt")), where do_thing is not responsible for closing
files because sometimes it is used with files that will continue to be used.
They eventually concluded that the closest thing to RAII available was __gc.

Some users presented separate complaints about resources kept in to-be-closed
locals in coroutines not necessarily being closed. You can do something to try
to solve this, like setting the __gc and __close metamethods of coroutines to
call coroutine.close. A "real" solution would look more like dynamic-wind.
Notably golang doesn't attempt to solve this one, so maybe not solving it is
fine.

~~~
pansa2
> <close> is equivalent to golang's defer [...] except at the block level.

Does that mean it’s the same as C#'s `using` and Python's `with`?

~~~
anonymoushn
I wasn't deeply familiar with python's `with`, so I looked it up[0]

<close> differs from `with` in at least the sense that it doesn't have any
equivalent to __enter__ and doesn't create its own block. It creates a local
variable whose __close metamethod will be called when it goes out of scope.
Since Lua has lexical scope at the block level rather than the function level,
this works similarly to the way Python calls __exit__.

These snippets of Python and Lua are roughly equivalent. They both open a
file, read one line, and close it, or do nothing if there was some error
opening the file.

    
    
      try:
        with open("foo.txt") as f:
          print(f.readline())
        # f is now closed.
      except:
        pass
    
      local f,err = io.open("foo.txt")
      if not err then
        local f <close> = f
        print(f:read("*l"))
      end
      -- f is now closed.
    

C#'s `using`[1] seems much closer, except that it handles nulls by not trying
to close them and lua's <close> does not include any such handling.

[0]:
[https://docs.python.org/3/reference/compound_stmts.html#the-...](https://docs.python.org/3/reference/compound_stmts.html#the-
with-statement)

[1]: [https://docs.microsoft.com/en-us/dotnet/csharp/language-
refe...](https://docs.microsoft.com/en-us/dotnet/csharp/language-
reference/language-specification/statements#the-using-statement)

~~~
frabert
The reference manual
[https://www.lua.org/manual/5.4/manual.html#3.3.8](https://www.lua.org/manual/5.4/manual.html#3.3.8)
seems to specify that "nil" and "false" values are ignored, so it behaves
similarly to C#'s using.

~~~
anonymoushn
Oh, that's great!

------
pansa2
There are many lightweight, embeddable scripting languages available, yet Lua
remains dominant. This is despite the language being "quirky" at best (and
"dreaded" at worst [0]).

Is this just inertia, or is there a technical reason for new projects to embed
Lua rather than a different language?

[0]
[https://insights.stackoverflow.com/survey/2018#technology-_-...](https://insights.stackoverflow.com/survey/2018#technology-
_-most-loved-dreaded-and-wanted-languages)

~~~
TazeTSchnitzel
What better languages are there which are friendly to embedding? A significant
part of Lua's popularity, AIUI, comes from it being easy to embed.

~~~
bjoli
Guile scheme, if you have users that don't have the knee-jerk "ewww
parentheses!" reaction. The threading model is vastly different from lua, but
compared to regular lua you get a lot faster execution.

~~~
srean
Guile is great and I enjoy how active it has been. However, it is no more that
small embeddable lisp. Its embeddable of course, but not small by any means.

~~~
bjoli
One you start wanting to run complex things over multiple interpreters in Lua
(say, to use multiple threads) you might as well just use guile.

Sure, we are talking 2.5mb of library and about the same amount object code.
Quite a bit larger than Lua. But that also gives you object code for working
with texinfo (and quite a lot of completely unused, undocumented modules). I
wonder how much could be stripped without anyone actually boticing

------
anonymoushn
One interesting change in this release is the move to a stackful VM. This
creates a pretty small (edit: actually huge, 1978 stack frames in a lua 5.4
repl installed with luaver just now) limit on the depth of the Lua call stack.
I wonder what motivated this decision.

~~~
shakna
I believe, but am not certain, that the change was made to help speed up Lua.
As an embedded language, and a language that runs on embedded devices,
performance is one of the key criteria of design that goes into the language.
The compile-to-bytecode step may have seen some improvement with the change.

So instead of speculating on why the change happened, I've found a few bits of
information around the limits that make me feel like it often simply won't be
a problem.

There's some interesting discussion around the limits across various platforms
here [0].

Whilst I'm not entirely clear on the motivation, even Lua running inside a
browser under emcripten has a stack limit of over 4000, which is somewhat
decent even for recursive functions.

Whereas on Linux you seem to be able to tweak a config when building and
safely have it in the region of 20,000, and quite possibly more.

Even under a tight restriction, Lua seems to cope fairly well with it when
recursing (the example crashed at 1758 recursions for a ulimit of 400).

Whilst Python has a terrible limit of 1000 for the default recursion limit
which is somewhat comparable to this aggressive test, Python also has no tail
call elimination - but Lua does.

I have a somewhat large Lua project (6000 lines of C, 10,000 lines of Lua)
that is a recursive parser for a format that I passionately hate (think self-
modifying LaTeX). With the default 5.4 limits it never hits a point where it
crashes, though I had expected it to.

[0] [http://lua-users.org/lists/lua-l/2019-06/msg00083.html](http://lua-
users.org/lists/lua-l/2019-06/msg00083.html)

~~~
schwartzworld
Doesn't Lua have tail call recursion built in to the language? Stack overflows
shouldn't be an issue on properly written recursive functions.

~~~
shakna
Yes, it has tail call elimination (which I think I mentioned). But you can
still right badly behaved recursive functions that avoid tail call
elimination, and that's when you'll hit your stack overflow.

I have a few of those badly behaved recursive functions in the parser I
mentioned at the end of my comment, and expected them to overflow, but they
didn't hit the default limit.

------
loustak
I really like lua. Especially when it comes to game with Love2d beeing so
wonderful. I wish more projects adopted it as a scripting interface...

~~~
donquichotte
Love2D is indeed excellent, and integrates well with the Tiled map editor. It
makes making games almost as easy as Macromedia Flash did!

------
grillorafael
"const variables" makes me giggle a little

------
shakna
The new random library implementation should stop me dragging in a Mersenne
Twister for every other thing.

It was libc's random, which is... Trash.

Now it use's xoshiro256 [a], which I hadn't heard of. (Not my area). [0] It
seems somewhat similar to the MT, as it isn't cryptographically safe, but
should give you a fast and fairly random set of integers back. It's also
significantly faster than the MT.

[a] Followed by two asterisks. I give up trying to make them appear in HN's
formatting.

[0] [http://prng.di.unimi.it/](http://prng.di.unimi.it/)

~~~
JoshuaRLi
Yeah, I was happy to see the switch away from C random/srandom or rand/srand.

Not my area either, but a little surprised why a PCG-family non-CSPRNG
algorithm wasn’t used: [https://www.pcg-random.org/posts/a-quick-look-at-
xoshiro256....](https://www.pcg-random.org/posts/a-quick-look-at-
xoshiro256.html)

PCG XSL RR 128/64 (MCG) could be used for 64-bit values (albeit with period
2^126 compared to xoroshiro256 __’s 2^256 − 1) with 128 bits. Half the state
size than xoshiro256 __, and the performance might be comparable (would need
benchmarking). LCG version if you want to trade PRNG quality for even more
speed.

------
amirmasoudabdol
I just recently started using Lua through the sol2 library. I'm basically
providing some flexibility through my input file to my users. I'm very
surprised by both library and Lua itself. For my use, I don't feel a lot of
performance hit, and gets a lot of flexibility.

------
catalogia
[Unfortunately, IMHO] Lua "minor versions" tend to break compatibility with
older versions, making it problematic to upgrade to new versions without
breaking scripts written for older versions. I believe mpv is stuck on 5.2 due
to this. Here's the list of incompatibilities for 5.4:
[https://www.lua.org/manual/5.4/manual.html#8](https://www.lua.org/manual/5.4/manual.html#8)

~~~
topspin
Likewise OpenResty is "stuck" on 5.1.

~~~
bungle
"Stuck" might be a wrong word. OpenResty actually dropped support for Lua and
is now LuaJIT only. LuaJIT is a fork of Lua 5.1, but it has features from 5.2
[1] (and from later versions where it sees fit). Because of this, Lua 5.1 is a
golden standard of Lua world at the moment.

[1]
[http://luajit.org/extensions.html#lua52](http://luajit.org/extensions.html#lua52)

~~~
xvilka
There is more modern alternative to LuaJIT though - MoonJIT [1]. Unlike the
former it supports modern Lua standards.

[1] [https://github.com/moonjit/moonjit](https://github.com/moonjit/moonjit)

~~~
spacechild1
AFAICT, MoonJIT is just a fork of LuaJIT. Both support many 5.2 extensions.
Additionally, MoonJIT also has a few Lua 5.3 extensions, but it is _not_ fully
compatible with Lua 5.3 or even Lua 5.2. Quote:

"Note: this provides only partial compatibility with Lua 5.2 at the language
and Lua library level. moonjit is API+ABI-compatible with Lua 5.1, which
prevents implementing features that would otherwise break the Lua/C API and
ABI (e.g. _ENV)."

Some internal language changes in 5.3, like the new integer type, would
require deep surgery in the JIT. This also seems to be the main (but not only)
reason why Mike Pall doesn't have plans to ever support Lua 5.3.
([https://www.reddit.com/r/lua/comments/2zutj8/mike_pall_luaji...](https://www.reddit.com/r/lua/comments/2zutj8/mike_pall_luajit_dislikes_lua_53/))

~~~
blodorn
Am I wrong? I thought the author of LuaJIT has been pretty well over it for
the last 5 years and no one else picked it up.

LuaJIT had a beta release 3 years ago and hasn't been updated since.

~~~
spacechild1
Looks like Mike Pall is still actively working on the project:
[https://github.com/LuaJIT/LuaJIT/commits/v2.1](https://github.com/LuaJIT/LuaJIT/commits/v2.1)

The last commit is from 5 days ago.

------
ausjke
many projects I am aware of stuck with 5.1 and luajit still and so do I. I
still use lua5.1 myself these days.

