Hacker News new | past | comments | ask | show | jobs | submit login
Fennel – Lisp in Lua (fennel-lang.org)
182 points by tosh on Sept 18, 2018 | hide | past | favorite | 62 comments

While I can't say much on the language (well, it's a Lisp), I must congratulate the author for creating exemplary front page for it. The four key bullet points as elevator pitch, code sample above the fold to grab attention, embedded interpreter to play around, and the simple instructions how to run locally now that I'm sold, it has it all without being too heavy or noisy or anything. Clean, neat, beautiful.

Looks like fennel compiles to lua; does that mean this is a blazing fast lisp when using luajit? And you can call any lua library? A blazing fast lisp with a large library ecosystem (luarocks)? That's very interesting indeed!

Edit; not bytecode, thanks

Just to go ahead and "be that person." Pretty sure CommonLisp already fits the "blazing fast with a large library ecosystem."

The ecosystem can be somewhat confusing, because much of what would look like abandoned packages in most ecosystems are more like completed packages, but speed hasn't been a real failing of Common Lisp for a long time now. Talked about a ton, of course.

CL is a system in itself; it can be seen as an environment / OS, and has even been used in this role.

Fennel piggy-backs on a known and widely deployed language / VM.

This gives them different capabilities, and very different possible adoption curves.

I agree: CL is a wonderful language but you have to embrace the whole environment/philosophy if you want to adopt it. And the language is not so easy to embed which prevents many "enterprise"/application development use cases.

As a matter of fact the only game in town for embedding CL code is ECL which: 1) relies on the Bohem garbage collector 2) is LGPL licensed

Both points have cons which might discourage embedding of Common Lisp code in other applications. Matters would be different if there was something like a CL to C transplier adopting a generational GC distributed with a MIT/BSD like license.

SBCL on some benchmarks outperforms C, and what portions of the code are not completely public domain are MIT/BSD licensed.

My point was not about peformances but rather about embedding. CL performances are good and toghether with interactive development these are the main selling points. It's a compiled programming language that feels like a scripting language "done right". However try to develop something like an embeddable library in SBCL and then use it in native Android, Ios , Mac and Windows applications. This is something that can be done in Javascript, Go, Dart, C,C++, Lua and so on. As far as I know the only CL that can achieve something similar is ECL. In my opinion this limits considerably CL adoption because it forces to adopt the entire ecosystem as a whole.

True, but one very great advantage of Common Lisp is that it’s a (reasonably) well-designed system, borne of the combined experience of many people. It has many features which seem like hacks (e.g. optional dynamic scoping) or historical baggage (e.g. dynamic scoping) or like over-engineering (e.g. UPDATE-INSTANCE-FOR-REDEFINED-CLASS), but which are actually the product of sound engineering practice, and which lead to solid, well-engineered code.

In some of my own benchmarks, was indeed able to get better perf using SBCL for some graphics stuff vs. Love. I particularly want to run this stuff on mobile though, so Lua / Love is the way to go!

It "transpiles" to lua. So you can do anything lua does, run on any lua interpreter, use any lua library.

So yeah, blazing fast, with a decent macro system and fixing some of the odd corners of Lua at compile time. It's really a great little thing (it's 2k lines of lua, so super easy to embed/ship too).

Targeting Lua is a great choice, emitting byte-code ties one to a particular implementation.

There are at least two other Lisps in Lua

* https://github.com/sctb/lumen

* https://github.com/SquidDev/urn

I want to use one of these three in order to write Lisp code, while getting Lua's ecosystem and distribution, but I am not sure which is most stable, and the differences between them. Can anybody expound on what makes them distinct from each other? I see Fennel and Lumen as the same with a big exception that Lumen also transpiles to JavaScript?

[Edit] I recalled this older HN thread with some relevant information: https://news.ycombinator.com/item?id=16566825

The creators of Fennel and Urn both chimed in to describe the differences between the two here: https://news.ycombinator.com/item?id=16567763

tl;dr Urn is a much bigger language that happens to compile to Lua as an implementation detail. Fennel is much closer to Lua and is dramatically simpler, only diverging from Lua semantics when the changes can be implemented purely at compile time. (immutable locals, blocking accidental global references/setting, etc) The compiler output from Fennel tends to be pretty readable and looks a lot like the input.

If you need to write code that runs in the browser, Lumen is one choice, but you can also run Fennel in the browser using Fengari: https://fengari.io/ I know next to nothing about frontend development but was able to integrate Fengari into https://fennel-lang.org to get the live REPL working with ease.

Yes, I had a comment in that older HN thread. I think this has me moving towards Fennel when compared with Urn, since I want a smaller language, but the examples of Lumen in this HN thread:


are swaying me to spend some time with Lumen this weekend to understand how it compares to Fennel. The Lumen examples were pretty impressive and it works for JavaScript too.

I tried the live repl and input the fibonacci example:

  > (fn fib [n]
   (if (< n 2)
    (+ (fib (- n 1)) (fib (- n 2)))))
which seemed to take, but then upon trying

  (print (fib 10))
it spit out the following:

  [string "return print(fib(10))"]:1: attempt to call a nil value (global 'fib')
Do I need to initialize fib before in the repl?

It works if you run both expressions from file. There's a tutorial page that explains why this doesn't work in REPL. See "Locals are per-chunk" here: https://github.com/bakpakin/Fennel/blob/master/tutorial.md#g...

Thanks, that makes sense. I will also try making it a global value assuming global works in the online REPL.

I find your point about Lua's ecosystem to be interesting. I have been working with Nginx/Openresty for quite a while now and I've found the Lua library ecosystem to be quite a bit lacking.

I like Lua quite a bit and it is of course, extremely fast, but I wish there were good "batteries" along with other stuff that I've come to expect from Python/Ruby et all. Of course, I've coded up some of those libraries myself as well. :)

Just wondering what people's general thoughts on this are.

I may not be in need of as many libraries as you. I find Lua has what I need. I have even started playing with Torch vs. Tensorflow, since I am not a fan of Python which is purely subjective for me. Python has a ton of libraries, but it just doesn't feel as good as Lisp or Lua, or many other languages to me. But then again, I love J! [1]

  [1]  jsoftware.com

> It "transpiles" to lua. So you can do anything lua does, run on any lua interpreter, use any lua library.

Bit odd statement, considering that LuaJIT afaik targets Lua 5.1 compatibility with some 5.2 extensions, while main Lua is currently at 5.3. I guess the question is what version Fennel is targeting

Fennel is continually tested against mainline Lua 5.1, 5.2, and 5.3 as well as LuaJIT: https://github.com/bakpakin/Fennel/blob/master/Makefile#L6

It works great with Fengari for client-side scripting; in fact this is what we use on the web site for the live REPL. We have run some tests against other obscure implementations such as Luerl and Rembulan but have encountered some bugs in those compilers that need to get addressed before they get integrated into the test suit.

There's already a blazing fast lisp with a large library ecosystem... It's called Common Lisp.

> There's already a blazing fast lisp with a large library ecosystem... It's called Common Lisp.

sorry what ? where does this meme comes from ? Most CL implementations are at best multiple times slower than your average JITted language

Like I said in a different reply, it was previously possible to compare right on Debian's language shootout page. Apparently they took out that feature, so here's an old link to somebody comparing SBCL, Factor, V8, and LuaJIT.


All four languages have improved over the last 8 years, but you can see SBCL and LuaJIT are in the same ball park, each winning on some tests. My recollection is that SBCL usually "wins" in the number heavy tests like Mandelbrot, but I don't have time to dig up the links.

Interesting you should mention Mandlebrot:


> the inner loop of mandelbrot is now identical (at the ucode IR level) to the IR generated by a C compiler. All type checks have been eliminated and only the pure (statically typed) FP adds/multiplies are left (the remaining task is to transform the ucode to the best possible machine code -- working on it). Similar results can be obtained for other numeric loops.

^ This is from a ten-year-old post about LuaJIT optimizations.

Now I’m curious, so I’ll try to benchmark SBCL vs LuaJIT vs Fennel when I get home tonight and post the results.

Regardless, though, I think it’s clear that there are already fast Lisps.

I'm definitely interested in your results. I do want to chime in to say that I do not think this would, in any way, take away from any of these efforts. If anything, I would hope they would all help challenge each other. In a beneficial way.

I finally got around to doing this, and the results were nearly the same on the Mandelbrot benchmark. SBCL generates slightly faster code, but in this case it's using less efficient multi-threading.

I might try again with a different program.


Maybe most are so, but SBCL and CMUCL are reasonably fast. Also I am not sure what you mean with "average JITed language".

Where did the meme come from that the average JIT compiler is fast ?

Maybe you could post some links to benchmarks that demonstrate your claim.

When people only care about the open source ones, maybe.

The free, open source CL implementation that matters and pretty much everyone (incl Google) is using - SBCL - destroys luajit in performance, not to mention your average JITed language..

I seriously doubt SBCL "destroys" Luajit. Or maybe it depends on the type of code. I remember a few years ago doing some Project Euler problems in both Common Lisp and Lua, and my recollection is that for that sort of code (nested loops, array accesses, fixnum arithmetic), SBCL ran code that had no type annotations significantly slower than Luajit, and with full type annotations SBCL only slightly faster than Luajit (and Lua uses no type annotations!). Of course, that type of code is probably in the JIT's sweet spot.

(I recognize this is unscientific and anecdotal, but I'm just trying to point out that it would take significant evidence to convince me SBCL is faster than Luajit, because my experience is the opposite.)

Some benchmarks of graphics stuff was faster for me in SBCL vs. Love. Going with Love still, but yeah.

Out of interest do you have any links regarding performance comparison of SBCL and luajit.

Before they completely ruined the UI, it used to be possible to compare languages on Debian's language benchmark shootout page. But now I can only find Lua, and can't even compare it to Lisp.

If I recall correctly luajit was removed from those benchmarks, for reasons I'm not entirely sure why, but I think it was to do with including only the 'standard' interpreter for a language?

Lua is emerging as a "everything compiles to it" source target - you can have Lisp style syntax, you can have static types and JS style syntax with Haxe - it's designed to piggyback off C codebases well - and on its own, it makes a decent Lisp too. So there are a lot of potential benefits to building a stack centered around Lua.

The winner of the last Lisp Game Jam (https://technomancy.itch.io/exo-encounter-667) was written with it and love2d, seems pretty cool.

This was my game, and I've contributed significantly to newer features in the Fennel compiler; happy to oblige if you have any questions about either.

Wow, congrats! I just posted above about deciding on Fennel over Urn. Have you checked out Lumen? What are your thoughts on it compared with Fennel? I want to choose one to work on my own type of game dev, live coding toy experiments.

I've looked briefly at Lumen. I don't do any JS work, so cross-compiling to JS doesn't really appeal to me personally. I would expect that compiling to two targets would make it more difficult to implement clean, readable compiler output the way Fennel does, but I haven't checked.

The docs for Lumen are a bit sparse, but it looks like it doesn't make globals opt-in or make you flag locals as mutable the way Fennel does. IME calling these things out explicitly makes certain kinds of bugs easier to catch. Fennel also allows for optional arity checks on functions.

I would expect that Lumen's keyword arguments would cause some complication when interoperating with Lua, which doesn't have them. In addition, cross-compiling to both JS and Lua means that it can't seamlessly support multiple return values, which are used pervasively in 3rd-party Lua libraries.

As I mentioned elsewhere, Fennel can run in the browser with Fengari, but it's a very different approach because Fengari includes a full Lua VM. This means there's some overhead to load the VM (~220kb) but once it's loaded you get access to coroutines, which are a much nicer way to code non-blocking functions than callbacks.

But these are just my initial uninformed impressions.

I don't do much JS either.

I am going to try Fennel, specifically look over your work on Fennel and the game you did. I am sure it will speak tons to my questions. I can always try and duplicate efforts in Lumen if I really want to know for myself if it is a good fit for me and my needs. Thanks!

I'm currently looking for lisp-like language for Lua. I've looked into Urn, Lumen and Fennel, still haven't decided on one.

Maybe someone can answer - why does fennel description insist on macros as compile time feature? I'm interested in live coding and self-modifying code; is there a speed penalty or some other reason why Fennel would be a bad option?

> why does fennel description insist on macros as compile time feature?

This just means that it's possible to ship the compiled code on its own and have no dependencies on Fennel at runtime. It doesn't mean you can't ship the compiler if you want to; the choice is up to you. In fact, I strongly recommend in most cases shipping the compiler so you can write and refine your code interactively as the program runs. It's helpful to think of "compile time" as a subset of runtime; it's just that portion of runtime when the function that's running happens to be the compiler!

Fennel is very well-suited for live-coding; I used this heavily when writing my entry to the Lisp Game Jam this spring. I blogged a bit about how live-reloads interact with the Lua module system here: https://technomancy.us/189

What problem domain are you working in?

I'm building the 2D interactive graphics engine for mobile platforms. The focus is on authoring all of content on phone/tablet.

Instead of using bitmaps/meshes, the user starts with single primitive (y < 0 half-plane) and applies series of transformations (wrap, intersect, linear transform...) to sculpture it into desired shape. This scene description looks and behaves both like JSON tree and S-expressions.

For interactivity, I would like to have a lisp-like language that's visualized as tree. The user would execute 'macros' to modify existing code, for example to wrap a function into a loop or to make a mirror reflection of existing graphics. The host should make sure that code is always syntactically correct and should provide default primitives when constructing new code, that will be replaced or modified by user.

Here's some WiP screenshots: [0] https://raw.githubusercontent.com/jmiskovic/arbo/master/medi... [1] https://raw.githubusercontent.com/jmiskovic/arbo/master/medi... [2] https://i.imgur.com/qzb6cC0.gif

Lisp in Lua? It does not have Lisp syntax and semantics.

Why would one call it 'Lisp' (which stands for 'Lisp Processor), when for example it does not do list processing as in Lisp?

  > (list 1 2 3 4)
  [string "return list(1, 2, 3, 4)"]:1: attempt to call a nil value (global 'list')
  > (append '(1 2 3) '(3 4 5))
  Compile error in '3' unknown:3: cannot call literal value

So, since many mediawiki systems use lua, does this mean that you can code in Lisp (or Fennel), transpile to Lua, and then run the code as a module in a media wiki system?

What about Common Lisp libraries? would they be transpiled too?

Is there even a site with Common Lisp libraries written in common lisp to be the source to be transpiled into Lua ?

> So, since many mediawiki systems use lua, does this mean that you can code in Lisp (or Fennel), transpile to Lua, and then run the code as a module in a media wiki system?

Yes, easily.

With a little bit more work you could load the Fennel compiler up server-side so it could do the compilation for you and support Fennel "natively". I have done a similar process with TIC-80 despite knowing very little C: https://github.com/nesbox/TIC-80/pull/597

> What about Common Lisp libraries? would they be transpiled too?

No, Common Lisp is a completely different language with little in common with Fennel.

How much longer until every project is written in its own language? This doesn't actually bother me: there is so little difference between the proliferation of new languages that "mastery" is kind of a meaningless term. Previous languages are tweaked for a particular task from a base language, and the project / application is made a bit easier.

In other words: writing a custom language for each application in the name of efficiency or more elegant patterns seems to make perfect sense.

I admire projects like this.

I wonder if there's a master list of "this language X transpiles to language Y and imitates language Z".

There was a site, but it went offline years ago. https://github.com/tcr/langlangmatrix

Here's a link to the page that renders as HTML: https://rawgit.com/tcr/langlangmatrix/gh-pages/index.html

(there will be a warning about loading unsafe scripts as the original file loads jquery under http)

Wish there was one for Tcl/Tk.

Why? Lua is embedded in numerous programs, as such targeting Lua is useful the same way targeting JS for web is.

I kinda would be interested in the opposite of that. Running Tcl on LuaJIT…

Would this be a good first lisp to use for learning?

There are a few factors at play that make a language good to learn: simplicity, learning materials, and suitability for learning projects.

On the first point, Fennel is simpler than any other lisp I know, beyond "my first interpreter" type Schemes. But you'll find much better learning materials for some other lisps; in particular Racket is exemplary in this regard.

So I would say it comes down to the learning projects you might want to build; this is always the trickiest part of learning any language. If you want to learn by creating a game, Fennel is a great choice. Likewise if you want to extend an existing program that already has Lua support, such as AwesomeWM or Redis. But for most "industrial" type projects that access external APIs or write to existing file formats, you'll have a better time in Racket due to the better library ecosystem.

I don't find it particular simple or a good Lisp. I don't think it is a Lisp at all. It does not even know about QUOTE or basic list processing. If Fennel is supposed to be a Lisp, than the concept of 'being a Lisp' is meaningless.

Actually the best Lisps are those who are actually Lisp (actually implementing the particular syntax and semantics of Lisp) and not a transpiler to another language. This other language leaks on all places.

Fennel is neither a Lisp nor a good learning environment for Lisp. Fennel is actually some kind of Lisp/parentheses-inspired layer on top of Lua. That's nice but not Lisp.

There are a lot of small Lisps which make learning Lisp much more easier. Start with any small Scheme implementation and that's much nearer to Lisp than Fennel will ever be: s-expressions, list processing, evaluator, code as data, macros, data representation, tail/recursion, procedural abstraction, first class functions, lambda calculus, etc etc. There is a lot of excellent material in the Lisp world to teach that. There is even material how to do useful things with that. None of that applies to Fennel. It does not do list processing, its code as data is weak and Lua leaks everywhere.

If you used Lua before, then this seems like perfect lisp to learn. If you don't know any Lua then some features will confuse you (multiple return values, error handling, arrays as tables, pairs/ipairs) because all these design decisions make for seamless inter-operability between two languages.

Perhaps for learning purposes you would benefit from lisp that has tools with debugging capabilities, like Racket.

I dig it

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact