Hacker News new | past | comments | ask | show | jobs | submit login

One interesting change in this release is the move to a stackful VM. This creates a pretty small (edit: actually huge, 1978 stack frames in a lua 5.4 repl installed with luaver just now) limit on the depth of the Lua call stack. I wonder what motivated this decision.

I believe, but am not certain, that the change was made to help speed up Lua. As an embedded language, and a language that runs on embedded devices, performance is one of the key criteria of design that goes into the language. The compile-to-bytecode step may have seen some improvement with the change.

So instead of speculating on why the change happened, I've found a few bits of information around the limits that make me feel like it often simply won't be a problem.

There's some interesting discussion around the limits across various platforms here [0].

Whilst I'm not entirely clear on the motivation, even Lua running inside a browser under emcripten has a stack limit of over 4000, which is somewhat decent even for recursive functions.

Whereas on Linux you seem to be able to tweak a config when building and safely have it in the region of 20,000, and quite possibly more.

Even under a tight restriction, Lua seems to cope fairly well with it when recursing (the example crashed at 1758 recursions for a ulimit of 400).

Whilst Python has a terrible limit of 1000 for the default recursion limit which is somewhat comparable to this aggressive test, Python also has no tail call elimination - but Lua does.

I have a somewhat large Lua project (6000 lines of C, 10,000 lines of Lua) that is a recursive parser for a format that I passionately hate (think self-modifying LaTeX). With the default 5.4 limits it never hits a point where it crashes, though I had expected it to.

[0] http://lua-users.org/lists/lua-l/2019-06/msg00083.html

Doesn't Lua have tail call recursion built in to the language? Stack overflows shouldn't be an issue on properly written recursive functions.

Yes, it has tail call elimination (which I think I mentioned). But you can still right badly behaved recursive functions that avoid tail call elimination, and that's when you'll hit your stack overflow.

I have a few of those badly behaved recursive functions in the parser I mentioned at the end of my comment, and expected them to overflow, but they didn't hit the default limit.

Ah yeah, I checked just now and the number of allowed frames is quite large. I observed a smaller limit of around 200 with the default config of an earlier release candidate of 5.4.0 and just expected the final version to be similar.

Citation? I don't think anything significant changed in that regard. Coroutines are still "stackless" in terms of the C stack, but "stackful" in terms of the language semantics. The virtual data stack is still clearly on the heap.

Lua 5.4 adds a new function, lua_setcstacklimit, but this merely exposes what was a compile-time constant, LUAI_MAXCCALLS, in earlier versions. Lua can't avoid making some use of the C stack as it supports intermingling of lua_call's from C code with in-language VM calls, which necessarily will use some amount of C stack. Lua has lua_callk for yield/resuming across C code invocations, but not everybody makes use of this and in any event it's only for coroutine semantics, not for recursion.

> Citation? I don't think anything significant changed in that regard.

The limits for recursion have undertaken a significant change. (Though it might not be that concerning, if you see my sibling answer).

> As already discussed, Lua 5.4 is using a "non stackless" implementation. That means that it uses the C stack for recursion in Lua functions. Because of that, the maximum depth of recursion is much lower in 5.4 than it was in 5.3. - Roberto Ierusalimschy [0]

[0] http://lua-users.org/lists/lua-l/2019-06/msg00083.html

In CorsixTH we're hitting C Stack Overflow issues on 5.4. I'm having particular trouble with our serializer for saving games which includes recursive calls to lua_gettable.

Was it not a stack machine already?

The embedding API explicitly exposes a stack, so I assumed that it was implemented that way.

it had a stack, but that stack was on the heap. In lua 5.4 under normal circumstances, 20 frames of Lua call stack corresponds to 20ish frames of C call stack. In lua 5.3 the number of C stack frames was some smaller number unrelated to the depth of the Lua call stack.

Ah - so it uses the C native stack.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact