
A Lua 5.3 VM and compiler written in Go - erwan
https://github.com/milochristiansen/lua
======
jzelinskie
There are a couple different Lua implementations in Go. Has anyone researched
the difference between them?

[https://github.com/yuin/gopher-lua](https://github.com/yuin/gopher-lua)

[https://github.com/Shopify/go-lua](https://github.com/Shopify/go-lua)

[https://github.com/milochristiansen/lua](https://github.com/milochristiansen/lua)

[https://github.com/afitz/golua](https://github.com/afitz/golua)

~~~
wahern
Milo Christiansen's seems to be the only Lua 5.3 implementation.

Note that Lua's versioning scheme is not like most other languages. Lua 5.1,
5.2, and 5.3 are not the same language. Though most code will still work
across those versions, the Lua developers are more aggressive at adopting new
features and killing old ones, even for core functionality. 5.2 dropped
function environments in favor of lexical _ENV, which was a huge change for
people implementing sandboxing, reflection, module frameworks, etc. 5.3 added
an integer type whereas previously all numbers were floating point, and
because Lua is dynamically typed you can't move an existing body of Lua code
to 5.3 without some careful attention. Similarly, the APIs also have
incompatible changes that while mostly simpe to deal with require porting
attention.

This more aggressive style of stewardship is what has made Lua an amazingly
elegant and powerful language, but reduces it's attractiveness for large,
monolithic, long-lived frameworks unless they're prepared to periodically
upgrade languages and migrate their user base. Since Lua is more typically
used for extension and embedding (i.e. not monolithic, pure-Lua projects),
this is less of an issue.

I was explaining to a colleague the other day that the reason you don't hear
so much about Lua is because choosing Lua isn't a decision that dominates a
project. A project that heavily uses Perl is a Perl project; one that uses
Python is a project. For the most part, you don't build Lua applications; you
build C, C++, or w'ever applications that use Lua, even when 90% of the code
is in Lua. This is a reflection of the carefully crafted API and language
semantics. But IMO it's the biggest reason why Lua hasn't yet had a break-out
moment, despite Lua 5.1, 5.2, and 5.3 being (IMNSHO) far superior languages to
JavaScript, Perl, Python, Ruby, or other similar dynamic languages. Lua is a
better language because it aggressively improves itself. But that evolution is
too rapid to support many large, marquee projects.

~~~
fao_
In my experience the difference between 5.2 and 5.3 was more towards
"negligible" than you're making out. While the functional difference may have
been major, most programs in my experience operate exactly as you would expect
them. For the majority of programs I doubt that they were even felt, aside
from a few minor changes like "unpack" -> "table.unpack"

~~~
wahern
That's been my experience, too. And the consensus on the lua-l list seems to
be that the migration to mixed number types went relatively smoothly for most
people despite the anxiety.

But people were still anxious, even gung-ho Lua users. Whether or not
justified, people and companies contemplating large projects are especially
averse to that sort of environment.

My open-source project experiences notwithstanding, before I left my day job a
few months ago we still hadn't migrated our 10+ year-old project to 5.2, let
alone 5.3, even though LuaJIT wasn't involved, and even though lua_callk and
lua_pcallk would have significantly simplified the libevent async I/O
integration. In a corporate environment planned obsolescence is a hard to
sell, even when it's objectively the better choice.

------
ComputerGuru
A _partial_ implementation without IO, pattern matching, or coroutines.

Nothing to see here, really. Just remaps basic Lua syntax to Go without
actually _implementing_ what makes Lua useful.

~~~
Impossible
In an embedded context lua is definitely still useful without those things,
although coroutines are nice to have almost all of the lua work I've done
hasn't used any of those features. IO and pattern matching seem important if
you're using lua for text processing, which isn't the only useful reason to
use Lua.

------
bpizzi
Slightly related: I'm in the process of choosing the best way to give our Go
binaries some scripting capabilities (it's entreprise stuff, the idea is
"let's the customer script this behaviour").

I'm aware of Lua and JS interpreters, some fully native, some made of binding
against existing interpreters. Native JS interpreters have my preference right
now: easier to build/maintain, I can bear the loss of perfs, and JS seems
easier to sell than Lua (remember it's entreprise stuff).

Does anyone have some insight or experience in that area and care to give
feedback to a fellow HNer? And did I miss other scripting languages?

~~~
Derbasti
Lua is incredibly easy to embed. The whole Lua interpreter including the
standard library is about 150kb. That's Kilo, not mega! You can swap out the
memory allocator or the number type on one simple header file. Lua was _built
specifically for_ embedding! What is more, it is a joyously well designed
language that is well suited for building DSLs. And it is pretty fast and
memory efficient for a dynamic language.

Other options are TKL, but the syntax is a bit unorthodox. Or Python, but it
is much bigger, and harder to embed. JS is not a great fit, I would say,
mostly because it is comparatively HUGE and not particularly optimized for
embedding.

~~~
hnlmorg
Perl 5 is another option. There's a few embedded Perl libraries out there. But
Perl has a bad reputation (undeservingly in my personal opinion) so
potentially another hard sell for enterprise.

~~~
hnlmorg
I guess it's too much to ask for whoever downvoted my comment to explain why
they did? Otherwise it's going to look little more than a knee-jerk reaction
from some Perl hater.

~~~
wahern
I didn't downvote you, but IME Perl is a poor language for embedding. Perl XS
is itself pretty bad, whether embedding a C application with Perl, extending a
Perl application with C, and especially if you're doing C->Perl->C->Perl mixes
--the Perl XS interfaces quickly become leaky abstractions. And don't even get
me started on the build process....

I've written far more than my fair share of Perl XS code over the past nearly
15 years. I still use Perl and the last time I wrote Perl XS module was just
last month. I could go on for pages about the negative aspects, without having
much to say about the positive aspects.

But here's a very concrete reason why nobody should contemplate embedding
Perl: I filed this bug report (Thread race in dist/IO/IO.xs) over two years
ago:

    
    
      https://rt.perl.org/Public/Bug/Display.html?id=125685
    

It's still not fixed. The fix isn't easy, either. It was introduced in an
attempt to fix some other stuff. But maybe the biggest reason it hasn't yet
been fixed is because the bug is only likely to be encountered when creating
multiple Perl interpreter instances in separate threads--a very uncommon usage
for Perl.[1] Uncommon usages will not be (and are not) well supported.

Non-threaded Perl embedding may be more common, but ultimately you don't want
to be embedding any interpreter that has shared state, especially if the fact
of this shared state leaks into interfaces or breaks the interface contracts.
Python is also still problematic in terms of not being able to completely
erase vestigial assumptions of one interpreter per process. Neither Perl nor
Python were originally designed for embedding, and even their extension APIs
were bolted on. While they've both been improved over the years and work well
for straight-forward C bindings, the fact that neither the languages nor
implementations were originally designed for embedding eventually comes
through and cause headaches when you want to do anything sophisticated--
sometimes even when you're not doing anything sophisticated.

[1] The only significant use of Perl in the open source world that I'm
familiar with that embeds Perl this way (multiple instances, multiple threads)
is Apache mod_perl. But for other reasons mod_perl creates and initializes all
Perl instances from a single thread. This is why the bug went undiscovered for
so long after it was introduced, and why there's little impetus to fix it.

~~~
hnlmorg
I don't think it is fair to compare XS with other embeddable languages because
you wouldn't expect to use C with them under those circumstances either.
Granted performance would be subpar without XS but if that is your primary
concern then you'd write your pluggins I'm C and do away with the embedded
scripting interface entirely.

Threading is an interesting point. It's worth noting this issue exists with
quite a few scripting languages though and the way Apache gets around it (eg
with mod_perl, which is garbage by the way) is by forking rather than
threading (I mean you can obviously run Apache with threading if you can
guarantee all your XS code is thread safe, but it's safer to just prefork)

------
throwaway2016a
This is really interesting for me personally because I just completed writing
an interpreter in Golang and it's great to compare and contrast.

One thing I noticed is that the parser is hand written. Which is interesting.
I always preferred to write my parsers in Yacc and Go actually has Yacc as a
built in tool (although know Lex).

Edit: I went through the list someone else posted and only one of the
implementations actually has a Yacc grammar file in it.

~~~
villedepommes
> Go actually has Yacc as a built in tool (although know Lex).

Interesting. Did they combine both a lexer and a parser into one tool?

 __lex __, __flex __, and __re2c __are typically standalone tools (lexers)
that tokenize input, whereas __yacc __and __bison __are parsers that parse
those tokens into ASTs (usually).

~~~
throwaway2016a
> Did they combine both a lexer and a parser into one tool?

No. They give you an interface definition for the lexer and you can implement
the methods however you see fit as long as it matches the interface.

Which I found annoying as there are some tedious parts to writing a lexer as
soon as you have a language requiring multi-rune tokens with common
prefixes.[1]

Apparently the back story is that go needed a parser generator but they didn't
need a lexer so they only built just the one out of necessity.

I read some places that they would welcome pull requests for a lex/flex
equivalent.

[1] A `rune` in GoLang is like a `char` in C only it stores multi-byte
characters where as in C it is more analogous to a single `byte`

~~~
hnlmorg
For clarity (I know you know what a rune is but your description felt a little
misleading imo):

A rune is just an alias for int32. But it's context usually means unicode
chars which are indeed multi-byte.

You can still use byte arrays (or "slice" if you're writing idiomatic Go, but
that's another tangent again) too though.

Also "byte" in Go is another alias, but for int8. But that shouldn't come as a
surprise to anyone :)

~~~
throwaway2016a
Thank you for the clarification. Your explanation will be helpful to people. I
wasn't planning on going into that technical detail, I was focused more on
typical usage.

------
lasfter
What is the use in Lua without pattern matching or coroutines?

~~~
wruza
This project may have a value as interpreter/vm implementation, not as
substitution of any sort. Table iterators and weak references are also
missing. Name choice seems pretty arbitrary, so author surely could just
mention that this vm was inspired by Lua and not name it alike.

The statement that he could easily implement coroutines via go seems like
oversimplification or lack of problem domain understanding.

~~~
weberc2
> The statement that he could easily implement coroutines via go seems like
> oversimplification or lack of problem domain understanding.

For someone with a lack of problem domain understanding, could you elaborate
on this? :)

~~~
wruza
vm.go/l.call doesn't seem to be reentrant (though I'm not native go speaker).
This means that yielding across, e.g. for-loop [edit:]iterators, would leave
vm with broken invariants. There are also pcalls that must be dealt with. But
since this language is just similar and differs in too many things, explicit
restrictions may help solve that, like it was done for <= 5.1.

------
gaigepr
Is the security policy that is used as a reason to not implement a bunch of
stuff documented somewhere?

~~~
loeg
6th paragraph of README.md:

> Anything to do with the OS or file IO is not provided. Such things do not
> belong in the core libraries of an embedded scripting language (do you
> really want scripts to be able to read and write random files without
> restriction?).

------
alvil
Crippled Lua

~~~
hnlmorg
Honestly, comments like this help noone. The author wrote this because it
solved a particular problem s/he had and was excited enough by it to share.
S/he has also released it with a permissive licence so that you can contribute
to it or even fork and enhance if you so wish. Or if you really don't share
the author's enthusiasm for the project then you can simply move on and use
someone else's Lua library (or perhaps write your own and risk other internet
randoms criticising your work without offering any constructive pointers).

~~~
vfclists
Pedant note - The pronoun you want for the unspecified gender is 'they', not
's/he', and it was well established before the gender-fluidity brigade adopted
it.

