Hacker News new | comments | show | ask | jobs | submit login
Lua VM running in a WASM environment (github.com)
180 points by vvanders 69 days ago | hide | past | web | 57 comments | favorite

So, here's a fun example: a self-hosted Lisp.

Copy this: https://gist.github.com/anonymous/af755d7cf0f71a574f39f547d3...

And paste it into the VM: https://cdn.rawgit.com/vvanders/wasm_lua/d68f46a8/main.html

It's a modified version of Lumen (https://github.com/sctb/lumen) made to run on this VM. It defines a runtime, a reader, a compiler, then compiles the runtime from Lisp to Lua and prints out its own source code.

Lumen is also capable of compiling to JS, and I wanted to show it printing out its own source code as JavaScript instead of Lua. But unfortunately I was getting some strange WASM-related memory errors when compiling as JS, so I decided to settle for a neat demo that everybody can run.

(The demo could be a little less cryptic, and I could do a better job of explaining it, but it's late.)

That's really cool(and kinda meta!).

I love how multi-paradigm Lua is, we used to do component based composition with metatables but I've seen OO inheritance and a slew of other approaches as well.

Other interesting projects:

- https://github.com/giann/fengari - new project, most exciting IMO

- https://github.com/daurnimator/lua.vm.js - emscripten based, has best interactions with DOM (but is a dead end)

- http://www.moonshinejs.org/ - no longer maintained, interesting model though

- https://github.com/paulcuth/starlight - transpiler: you loose ability to use proper coroutines which are IMO most interesting feature for lua in browser

- https://github.com/creationix/brozula/ - unmaintained

- https://github.com/logiceditor-com/lua5.1.js

- https://github.com/mherkender/lua.js

To add some anecdotes to the above (plural of anecdote is of course data):

- fengari: Dunno, nice logo & name though :) Not done yet?

- lua.vm.js: Pretty good, but as you say in your README: "Next step is to iterate on the Lua <=> JS interoperability. Clever solutions to the lack of finalisers in Javascript are being searched for." (-> leak mem if passing stuff to JS in certain situations)

- Moonshinejs: paulcuth/gamesys' previous effort. It's a Lua VM written in JS, that accepts just Lua 5.1 bytecode. Works in IE6+. Very small. Nice 'DOMAPI' plugin; nice in the sense it mostly just works, and when it doesn't, the plugin is so small & simple you can hack it yourself.

One thing to note is when getting strings back from the DOM using DOMAPI, I've run into situations where the strings when accessed Lua-side are in fact 0-indexed not 1-indexed. (the_string .. "") is a working normalization technique.

Coroutines work very well here.

Also, do not overlook the in-browser debugger addon. It's a seriously nice piece of work, that more than makes up for the lack of source map support.

It deals with the lack of finaliers in Javascript by providing explicit .retain() and .release() mechanisms.

- Paulcuth/Starlight: As you say, transpiler, no coroutine support though.

How to get coroutines: ... I was looking into this, and you need something like using kripken's Relooper algorithm or a more general version of Facebook's Regenerator. Scary stuff.

Note about both moonshinejs & starlight, with respect to the string library: It implements Lua patterns by turning them into JS RegExp. The implementation isn't good enough as it stands; it's easy to get the wrong results, as the escaping logic is wrong. Damned if I know where though. I tried to fix it and just broke a different class of patterns...

- Brozula: Dunno

- Lua5.1.js: Dunno. Would be nice to have a better API for interaction than the Lua C API. No commits since 2013, like Brozula.

- Lua.js (lua2js): Dunno. Several open issues about e.g. '<' operator not working. No commits since 2013. And:

  _lua_coroutine["resume"] = _lua_coroutine["running"] = _lua_coroutine["status"] = _lua_coroutine["wrap"] = _lua_coroutine["yield"] = _lua_coroutine["create"] = function () { not_supported(); };

I had a project that was doing some WebGL stuff using Lua, I found the Emscripten compiled VM to be at least 2x faster than the Moonshine. Compatibility was great, I seem to recall a lack of direct access to the underlying TypedArray. It needed something akin to a table that was directly mapped to raw memory.

That's very good to know! My current project is pretty insensitive to performance, but that's definitely useful info for the future.

It is a little extra upfront work, but I found it valuable to support a diversity of Lua VMs in Javascript because little impedance mismatches can really derail a project. Being able to switch from one to the other can isolate bugs, compare perf, memory footprint, or different interop schemes. There is literally no way to know at the beginning of a project whether any one runtime will be sufficient.

This was just a little demo that I threw together. I didn't expect so much interest but I'm glad people like it.

Lua is a unique language and it's ease of embedding made it a perfect fit for WASM.

This could be be very useful for me.

Long term, I would love to see a Lua to WASM compiler/transpiler with additional Lua runtime functionality efficiently implemented in WASM as well. Since WASM already has its own stack-based VM, why not skipthe additional Lua VM bytecode execution layer?

I would love to work on this myself, but right now the time investment isn't practical for me.

I've got a Befunge JIT compiling directly to WASM: https://github.com/serprex/Befunge/tree/master/js (hosted: http://etg.dek.im/funge/funge.html )

Maybe I should move on to something a little more practical

Note that there seems a lot of misinterpretation of WASM as a good VM bytecode target. You're stuck with i32/i64/f32/f64. Strings have to be manually managed through byte indexing. There's no GC. Targeting WASM directly is really close to targeting an actual ISA directly. So efficient implementation to WASM is going to come off closer to something like LuaJIT. It also doesn't facillitate self modifying code so lazily compiling will imply either using a trampoline to jump between multiple WASM modules or recompiling everything

There's no jump instruction in WASM, so arbitrary goto has to be converted into loop-and-switch. The Befunge VM I linked breaks that down to only relooping between basic blocks. Browsers optimize that pattern. There's room to play around & see if they can optimize something like a CPS VM. Instead of dispatching based on the value of a memory cell, encode an instruction as a 'store-next-instruction to local, break to instruction implementation (if not short bytecode), have that instruction end by looping back, where we switch to next instruction which repeats the store-next-instruction to local'. Where next-instruction encodes this dance repeatedly. Hot areas of the VM may then be inlined by the browser's JIT

To summarize the above paragraph: instead of compiling to bytecode, compile to threaded gotos

Lua compiles to C which compiles to WASM, and the Lua forums already has some people demonstrating Lua code compiled to WASM like this. So you can do this already to some extent.

The WASM file seems to be 88K compressed, 241K uncompressed. Not bad!

At last.

(Lua is everything JS should have been.)

Probably true 5 years ago. I believe now it isn't, mainly because JS is moving much faster (not always good, but many goodies in JS simply do not exist in Lua).

There's some things intrinsic to Lua's design that makes it better for performance & embedding. Take a look at how LuaJIT destroys JS in most benchmarks.

Lua also has coroutines and a few other things that don't exist in JS. JS certainly has the market share and libraries but there's quite a few areas where Lua does much better.

I'm pretty sure that the original designers of Lua didn't envison that, but LuaJIT is quite nice. I like Lua coroutines but I think JS has practically caught up with generators and other external supports.

Huh, didn't know about generators, learn something new every day(although like everything with JS support looks a bit spotty).

Generators are also less general. You can't `yield` from a non-generator function. In Lua, coroutines are implemented as a library, not as a core language/syntax thing.

Pretty sure coroutines are a core part of the language since there's specific C APIs around them but I could be wrong.

The library requires low level access to the VM, but you can yield from anywhere, and pass `coroutine.yield` as a value to higher order functions.

However, there are (were?) restrictions when C functions are present on the call stack between `coroutine.resume` and coroutine.yield`.

Edit: not that low level, actually, here's the coroutine implementation, built on top of the public Lua C API:


(167 lines including white space and comments).

JS's core language remains the same. If you're of the opinion that Lua was designed better from the start, then adding features to JS doesn't change that, as nice as some of those features might be.

One, keep in mind that Lua has the version 5.x. It is an accumulation of early user experience just like JS. I haven't said Lua was designed better from the start. I have merely stated that Lua in 2012 is probably better than JavaScript in 2012 (and I was careful enough to add a word "probably" :-).

Two, core languages don't matter here because they are fundamentally similar to each other at the core level (dynamic and considerably weak typing, unified object-array, first-class value from missing indexing, semi-working lexical scopes and much more). The pain points of Lua tended to be same to those of JavaScript in my professional experience.

Three, the language is not a mere combination of syntax and semantics. Any evaluation should also account for user bases and ecosystem, and in my very humble opinion Lua spectacularly fails at both. I'm not going to assume the alternative reality---Lua has a sizable user base and its ecosystem is worse even for that user base.

> ... user bases and ecosystem, and in my very humble opinion Lua spectacularly fails at both.

I think that call depends on your use case. If you've ever tried embedding v8 or spidermonkey you might have a different opinion.

For instance we use to run the entire game state for multiple game titles I worked on in Lua. This was on the PSP where we only had a 400kb block of execution space and a 333MHz MIPS processor(system memory was 8mb w/ 24mb for video + audio).

Those types of environments are where Lua shines which lets you use a scripting language that scales from nothing up to full-blown systems.

> I think that call depends on your use case. If you've ever tried embedding v8 or spidermonkey you might have a different opinion.

I'm saying this in the embedded perspective (my professional involvement with Lua was primarily in that form). The lack of package manager might be acceptable as an embedded language---the lack of quality library isn't. The embedded story masks the weakness in the ecosystem because you can somehow make everything from scratch, but when you are programming at large the story becomes a disaster (combined with common symptom of dynamically and weakly typed languages).

I don't doubt Lua is a good language to embed. I doubt Lua is a good language to write a large software, especially after having worked with more than 300K lines of Lua code in a single code base.

I thing any dynamic language starts falling over at that scale. Lua, JS and the like make great glue and logic bridges but I won't want to use them across the whole stack.

I first mention that JS has been actually successful in non-glue areas :) Dynamic languages are hard to scale, but it is possible after some headaches.

The lack of quality library also means that you are even more risky when you are writing a small program (because you have less incentive to write it yourself). I have experienced multiple times that even the existing libraries (including the standard ones) had crucial flaws and no one seems to be bothered to fix that. Also in the embedded setting the use of snippets, rather than proper libraries, are more common as libraries can be harder to integrate, and unfortunately we are left with PHP-esque lua-users.org for Lua...

I was assuming alternative reality. The sole reason why JS got the popularity that it did is because it was the one and only browser scripting language. If Lua was one, then it'd get the same popularity boost instead (and all the great tooling, large community etc that came with it).

> If Lua was one, then it'd get the same popularity boost instead (and all the great tooling, large community etc that came with it).

And there are so many languages with no such popularity boost that nevertheless have much better tooling than Lua. In fact, JS is very exceptional (in that the growth drove the tooling) and in many cases the tooling tends to keep up with the growth---Lua didn't, even though it could have done better. The alternative reality is an easy way to excuse this situation but also meaningless here.

Lua in the browser? Truly, we live in the future.

You should check Sailor web framework which is implemented in Lua; it lets you do such thing


if you can do lua, you can prolly do any language.

Can't wait for GHC.

Can't wait for BEAM -- the Erlang VM in the browser!

you can already kinda get that with purescript

Never say never but Lua is pretty unique because of the vm size and language design.

It seems to me that Clojure is similar insofar as it was designed to be a hosted language. Although, it sounds like wasm might be too low level for now.

Its faster for sure... I benchmarked a dummy pi calculation code and got this results:

> JS: 0.11499999999068677ms

> Lua: 0.0019999999999527ms

BUT, they don't print the same results.

I'm running into an integer overflow error. Try this as input: https://gist.github.com/anonymous/f00581ecf4dba42501c9c56991...

Is there a way to debug it?

It should print the following string: "print(1 + 2)"

This is very cool.

A Lisp running on top of Lua on top of WASM? :)

Yup! I couldn't resist. https://github.com/sctb/lumen

I think there's some bug in indexing into Lua tables. print(str({1, 2})) should print "(1 2)", for example, but it triggers an integer overflow error.

It's actually capable of compiling to both Lua and JS, so I was going to show off a Lisp running on top of Lua on top of WASM which generates JavaScript, completing the circle of life.

I really look forward to being able to compile WASM to native machine code. With a default link to an SDL canvas library, it would become something like a universal runtime for most media applications. You run that in a sandbox and you pretty much have a native version of the modern browser minus the DOM.

WASM apps will still have access to the browser; you'll need more than SDL to replace that.

More likely, native apps will port to WASM.

Didn't work for me. Here's my Chrome OS X console output: http://i.imgur.com/RnA5mUb.png

Let me know if you need more info.

EDIT: Ah, it's firefox-only.

Is there some way I can help get it running on Chrome?

On Chrome you need to turn wasm on apparently. The emscripten GH (https://github.com/kripken/emscripten/wiki/WebAssembly) says you should also use Canary, but I see chrome://flags/#enable-webassembly in my Chrome flags, so I turned that on. Still doesn't work though; I'm getting this now:

    Uncaught TypeError: WebAssembly.instantiate is not a function
        at doNativeWasm (main.js:1)
        at Object.Module.asm (main.js:1)
        at main.js:1

That's implementing an older version of the WASM spec, hence why you need Canary.

Go to chrome://help/ and it should update to latest version (Version 57.0.2987.110 (64-bit)), after that it works like a charm.

This usually happens when you haven't restarted Chrome for a while.

Support for WebAssembly is enabled from version 57: https://www.chromestatus.com/feature/5453022515691520

Make sure you're on Chrome 57

Worked without any configuration changes for me (Chromium 59 on Stretch).

Worked without any configuration changes.

Chrome Version 57.0.2987.110 (64-bit)

Sweet, I've thought about doing this with emscripten in the past for a an LED panel i've been building/working on. It runs LUA on a microcontroller to drive the display and I wanted a way to simulate the whole thing on the fly.

Anyone else getting " Assertion failed: Cannot call unknown function run_lua (perhaps LLVM optimizations or closure removed it?)" ?

Yup, on Chrome 56.

WebAssembly is only on by default in Chrome 57+.

If I can get cqueues running, then I can run lapis on that and get a webserver happening.. in my browser!

Works on Chrome 58 out of the box. Very cool!

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | DMCA | Apply to YC | Contact