I took some Visual Basic programming courses in high school, some PASCAL courses in college, and then didn't program anything other than basic HTML/CSS for the next eight years. Last year I programmed an iPhone game in Lua (using the Corona SDK) and it was easy as Tau. Sure a better programmer would have finished the programming aspects in a quarter the time it took me, but I started by looking at Objective-C and I'm not sure I ever would have picked it up without some kind of formal training. Contrast with Corona/Lua, where I watched a few eight minute videos on YouTube and could turn around and make simple but complete programs, and more importantly completely understand what was going on within them. Reading the Lua documentation was a revelation, reading Apple's was a headache.
As long as you are not running code that is downloaded after app install (e.g. either embedded in the app itself or entered by the user from the keyboard) you are good to go. This mod to the previously ominous app store policies is used by quite a lot of applications.
The more I learn about Lua's design and implementation, the more impressed I am. It's very rare to see software that does so much with so little code. The design is extremely clean and the code is extremely fast.
The C API is easy to use and gives good performance, and yet encapsulates enough of the VM's implementation that C modules are source and binary compatible with both Lua and LuaJIT, two very different implementations of Lua. Note that the C API was designed before LuaJIT was even written, so this API was designed without the benefit of seeing how different implementations of Lua might vary.
The source is very readable for the most part; my only stumbling block is that their naming of types and variables is extremely terse at the cost of readability sometimes, and the extensive use of macros can sometimes can require a lot of jumping around. But Lua is an impressive achievement that I frequently refer to for design inspiration.
Lua has great C integration, but LuaJIT's is even better. You can natively interface with C code without a recompile, just by defining the C function prototype in Lua, and using it like you would any normal lua function. Better still, LuaJIT can frequently inline the calls to C code, so there's no cross-language marshaling penalty.
FFI is extremely cool and impressive, no doubt, but a major downside of it is that you give up memory safety. Once you import it your Lua program can SEGV the interpreter and read/write arbitrary memory in your process. Freedom from memory errors is a major motivation for using high-level languages, so this should not be given up lightly. Also, from a security standpoint it means your Lua is less sandboxed.
Yes, but you use it to interface with native code. Normally, you'd write Lua_Cfunctions to achieve the same behavior - and those can crash your app just as easily, since they're native code. Perhaps more so, since it's easy to screw up the stack manipulation.
Yes, but the point is that once you've loaded FFI you have to trust 100% of the Lua. With a Lua C extension you have to trust the extension, but not the Lua that loads it.
If FFI is loadable, how are you going to prevent your untrusted script from loading it and going to town? How are you going to avoid providing "direct access?" My whole point is that I don't think you can. It's all or nothing. If ffi is available to some Lua code, it is available to all of it.
OK, native code can crash. Not much of a problem. He have been running servers written in C/C++ since the ancient times for more critical tasks than your average Lua program. From Apache to Nginx and from Qmail to MySQL to MongoDB. So what if a Lua program has the "potential" to crash if you interface with C?
You do the interfacing because there is a need to do it, anyway. It's not like you go FFI for fun.
haberman's argument is making an assumption that perhaps you aren't. Namely, one of the benefits of Lua is that, barring behavior introduced by C extensions (and bugs in the Lua interpreter), it can't really harm your program. It can't crash or do anything else. It can suck up resources, but that's about it. As such, with whatever sandboxing you feel is appropriate, you can run arbitrary Lua code without worrying (e.g. World of Warcraft can run third-party addons). But with LuaJIT's FFI interface, it appears (note: I have never used LuaJIT or its FFI interface) that you can no longer trust arbitrary Lua because it can reach into your program and call whatever function it feels like, including the ability to crash you at random, or do other things that it's not supposed to be able to do. You can no longer just trust Lua code to not impact the rest of your program.
Why isn't Lua more widely used? One reason is a consequence of it being an embedded language. Lua has had 5 major versions which are very incompatible with each other. You're just supposed to stick with the previous version until you upgrade your code.
I read all the Lua papers, and they are quite fond of "getting it right" (which is refreshing). They will break things to get it right, whereas other languages stick with all their old design warts in the name of compatibility. I like this approach, but it comes at the expense of popularity.
Compare Lua with Python, which has had basically 1 major version for 20 years Python 1.x are all compatible with 2.x -- that was a license change; and Python 3 is having adoption troubles which proves the same point. Python 3 was the chance to "get things right", or fix old design warts, but you can see what a big impact it's having on adoption, fragmentation of libraries, etc.
The other reason is that embedding a language isn't as common a use case as a scripting language. I've tried to write stuff in Lua rather than Python, but really Python's "batteries included" philosophy just kills any advantage Lua has.
The main reason I would like Lua is so you can distribute stable (small) executables... but when you're starting out Python is a lot easier. You don't want to download LuaSocket or whatever and mess with the build system.
It's easier to break reverse compatibility when your whole language is a 200k DLL and MIT Licensed. If people need to stick with version 4.0, it's realistic to maintain an in-house fork.
This could definitely stand to be better. I only learned about LuaRocks by googling around when make failed to build LuaSocket and the README was useless. By then I had been using Lua for about 3 years.
Not only does it lack a standard library, it lacks a standard metaobject protocol. The fact that a MOP can be implemented in Lua is very nifty, but not having a standard one makes an ecosystem of shared code very difficult.
I don't know whether this is in fact true for the Lua community, but surely this is not necessarily true in general. MOP is not part of CLOS as defined by the standard, but that doesn't seem to create horrible problems (especially with Closer)
Is ability to break compatibility due to being embedded?
I recall that MS ships with each game the exact version of DirectX it need, so they are free to improve it without being hamstrung by back-compatibility.
Good article, but one nitpick: not everything in Lua is a table. Tables are a versatile data structure that can be used as arrays, dictionaries, objects, etc. However, Lua provides many types besides tables: numbers, booleans, closures (functions), strings (which are interned), coroutines, and something called userdata (which are typically pointers to something implemented in C).
Another cool thing about Lua, which was mentioned only briefly in the article, is proper tail-call optimization (TCO) like you'd find in Scheme. TCO makes expressing recursive algorithms nicer because you don't have to worry about blowing your stack.
Lua's design philosophy -- giving the programmer a handful of basic yet powerful features -- makes it somewhat Schemy. I suspect that Lua is about as close as you can get to a ball of mud before becoming Lisp.
To nitpick your nitpick: in Lua tables are the only composite data structure. Everything you listed is either atomic or opaque (you can hide composite datatypes in userdata, but you can't introspect the values).
If you think about it, this is very similar to C where the only composite datatype is a struct (arrays are just sugar on pointer arithmetic). In fact, I think if you wanted to make a scriptable dialect of C, you'd create Lua.
Of course, by being simple, add in dynamic typing and first-class functions, and Lua does feel a bit like Scheme's kid brother. Or, rather, Lua (in my mind) reveals how C and Scheme aren't so far apart after all!
Well, since we are nitpicking nitpicks, allow me to nitpick your nitpick of a nitpick: Lua closures are also composite data structures. You can get/set their upvalues via the debug API. Behold:
function create()
local a,b,c,d,e,f,g,h,i,j;
return function() print(a,b,c,d,e,f,g,h,i,j); end
end
function array_get(f, index)
local k,v = debug.getupvalue(f, index);
return v;
end
function array_set(f, index, value)
debug.setupvalue(f, index, value);
end
function map_get(f, key)
for i=1,math.huge do
local k,v = debug.getupvalue(f, i);
if k == key then return v; end
if k == nil then return nil; end
end
end
function map_set(f, key, value)
for i=1,math.huge do
local k,v = debug.getupvalue(f, i);
if k == nil then return; end
if k == key then
debug.setupvalue(f, i, value);
return;
end
end
end
function test()
local f = create();
array_set(f, 1, "one")
assert(array_get(f, 1) == "one");
map_set(f, "b", "something");
assert(map_get(f, "b") == "something");
print("Success!")
end
test();
If table creation was disallowed, you could use functions in place of tables. And you could repurpose an existing table as the function type's metatable, allowing syntax like func.x=func[y]
Oh, and coroutines probably also fall under composite data structures technically...
Very well played! If I were the nitpick-y type ;) I'd wonder whether use of the debug library is kosher (esp. in production code), but you are definitely correct. Though, actually, this again correlates very closely with C/Scheme -- You can use an offset from a stack frame pointer to access stack variables in a function scope in C; and upvalues are just closed-over variables a la Scheme closures.
Actually, that's another point in Lua's favor. The mechanism Lua uses for managing closed-over variables is WAY more elegant than, for example, Ruby's mechanism...
Yes, tables are the only composite data type in Lua. But it's not fair to say that everything is a table. Compare with Smalltalk or Ruby where everything is an object, even types that are not usually viewed as composites.
Lua has a lot of depth to explore (TCO, coroutines, prototypes, ...), but the language has been carefully designed to only really be like JSON, if that's all you need. The advanced parts stay out of the way.
Good point. I was thinking (as jballanc sort of mentioned) of this with regards to composite types. E.g., there is no difference between lists versus hashes versus tuples versus objects, etc. But duly noted for any revised edition :)
LuaJIT speeds can rival code written in C. And if you are still not getting the speed you want, see my 1st point above (Integration with C and C++ for that matter).
The simplicity of Lua is, I think, really underrated.
It is a good learning exercise to write some small benchmark / utility / tool in Lua so you can get an idea of how quickly you can develop, and how fast the resulting code will be.
Then read through the Lua source [0] to see just how little code is required to give you that platform for your ideas.
The simplicity of any code is really underrated. Like 4 orders of magnitude underrated: http://www.vpri.org/html/work/ifnct.htm (Long story short: a typical mainstream desktop weights more than 200 millions lines of code. The guys at the Viewpoint Research Institute can simplify it down to 20.000 lines.)
As far as I can tell, several things can explain how they fit a whole library in a single book (assuming 50 lines per page, 400 page books, 10.000 books in a library). First, the use of dedicated languages instead of C and C++ seems to explain 2 orders of magnitude. The 2 remaining ones are explained by feature creep and plain lack of reuse (in the Franck system for instance, they use a single library to draw everything, from characters in a word processor to the frames of the windows.)
They also claim specific achievements: a full self-implementing compilation stack in less than 2Kloc, on top of which implementing something like Javascript or Prolog takes less than 200 lines (see http://piumarta.com/software/maru for actual code). A Lex/Yacc equivalent in 400 lines or so (see http://www.tinlizzie.org/ometa/ for actual code). A Cairo equivalent that run with acceptable performance in about 500 lines. A TCP-IP stack in 160 lines, stable enough to run a web site.
If half of that is true, we can effectively talk about a silver bullet. That bullet won't kill the complexity werewolf of course, but it will seriously cripple it.
Looking at OMeta, it seems to use libcairo and X11. It's hardly fair to _not_ count those lines. Same goes for dedicated languages. If I don't have to count my toolchain, and I'm allowed specialized languages, I can do everything in one line...
DoAwesomeStuffJustLikeILikeIt();
Don't get me wrong, there's some cool stuff there - but the claims of 4 orders of magnitude seem exaggerated to me.
Wait, what? OMeta uses libcairo and X11?! Where did you see that? The only thing I saw it use is a host programming language (Javascript or Python or Ruby…). You may want to count those lines instead. Anyway, remember that they also claim a full language stack in less than 2000 lines. Including a variant of OMeta. Even Lua is 5 times as big. Not counting GCC.
I know it's unbelievable. But other personal experiences make me think they're probably right. I have written equivalent OCaml and C++ code where the C++ version were 5 times larger (both where optimized for clarity). In my day job, I routinely divide substantial portions of C++ code by 2 through light refactoring.
VPRI's miracle doesn't only come from the awesomeness of their ideas. It also comes from the awfulness of current systems. A full desktop in 20.000 lines may not be so small, if you consider that current ones are way too big.
Addendum: I omitted a rather important detail: while the STEPS project aims to build a full desktop system, along with networking, publishing, messaging, and programming capabilities, it makes no attempt be compatible with anything (except the hardware on which it has to run). It doesn't do HTTP nor HTML, for instance.
Based on a cursory glance at the sources. Maybe I'm misreading things, or maybe it's an intermediate version - certainly possible.
And, to be clear, I _do_ believe there are substantial savings in program size to be had. (Especially over C++. Chatty little monster ;)
It's the 4 orders of magnitude that make me doubtful. The good thing, AFAIK VPRI is a year or so from completion, so we'll see soon. And even if it's not 4 orders of magnitude, I'm sure there's a lot to learn from it.
Guile does it better. You can use shared memory threads in guile without any penalty. Atmost you have to allow for the garbage collector to run when inside FFI functions. But that is a small price to pay in case you need to use multiple parallel-concurrent threads with a single heap.
Guile was built with FFI in mind and has an impressive history. Just take a look at guile gnome bindings.
> Speed and Simplicity
Guile 2 is extremely fast. Not as fast as LuaJIT, but it no reason it won't get there. As for simplicity, take a look at the partial evaluator in trunk of guile 2.
> Education
Guile is good old scheme.
> Functional
Can't get more functional than scheme :)
> Everything is a Table
Well, almost everything is a pair in guile. Vectors and hash-tables are trivially available. Though I recommend to sticking to functional programming in scheme style.
> Consistent
As before, can't get more consistent than scheme.
> Portable
Guile is available on n900. So there.
To continue, guile has continuations (delimited or otherwise), and macros (hygienic or otherwise), both of which are effectively missing in lua.
And guile offers all of this while supporting native threading with a single heap. Sweeet.
Not sure if I'd agree that Guile's FFI bindings are better than LuaJIT's FFI. It's certainly easier to declare a function in LuaJIT - you can take the definition directly from the header file, rather than transcoding it into s-exps.
You also get the ability to add metamethods and finalizers to returned C objects (so you can use Lua's built-in GC to clean up after, for example, your FILE*).
As for the speed, there's no reason that a lot of dynamic languages couldn't be as fast as LuaJIT. But none of them are even close. I wish LuaJIT was still up in the computer language shootout. The LuaJIT interpreter (with JIT turned off) is 3x faster than V8 on the PC, and faster than Java on Android. And that's the interpreter - the JITed code is way faster.
Of course, macros are awesome, and a huge point in Guile's favor. On the plus side, Lua is very easy to understand, especially if you're coming from Javascript.
"As for the speed, there's no reason that a lot of dynamic languages couldn't be as fast as LuaJIT. But none of them are even close. I wish LuaJIT was still up in the computer language shootout. The LuaJIT interpreter (with JIT turned off) is 3x faster than V8 on the PC, and faster than Java on Android. And that's the interpreter - the JITed code is way faster."
[Citation Needed]
I know of old benchmarks (pre Crankshaft) that showed the interpreter being faster than V8, but my own tests using the same benchmarks from the computer language shootout show V8 to be about 1.5x faster than luaJIT now.
You can compare v8 to vanilla lua on the computer language shootout benchmarks page, and then compare luajit to lua on the luajit page. V8 has definitely made up some ground, and the luajit interpreter isn't generally faster than v8, but the jit is still way faster on almost all tests. The one area where luajit isn't great is on tests that stress garbage collection, since luajit still uses vanilla lua's GC. I think that's luajit's next area of focus.
My definition of an FFI was simply about fully supporting native threads. Simplifying FFI definitions has not been a big deal for me. It is either a few hours of work, or if more than that, then I simply use Swig.
I agree on the speed part. But guile 2 is getting better. The reason guile cannot be as blazing fast as say gambitc is their need to inter-operate with all sorts of C code. Native threading included.
Rich Hickey: "Functional has been applied, you know, for a whole spectrum of things. I think there’s the haskell meaning of functional, which is sort of, the pure notion of function applied to absolutely everything in the language. And then there’s another notion of functional... lisps claimed to be functional early on, only for the simple reason that functions were first class objects or values that you can pass around. Those are the two ends. Somewhere in the middle, i think, is a style of programming that emphasizes programming with pure functions, in other words functions that are free of side effects, that operate only upon their arguments, which are values, and that only produce values as results. So that’s a pure function. Clojure emphasizes the style of programming using pure functions, where lisps, traditionally, have not. You could do it, but it was a choice. The key thing in forcing you to make that choice, is to have the core data structures be immutable, so they now can be treated as values, in a way that mutable things cannot. So Clojure emphasizes functional programming, by providing a set of immutable core data structures, including those ones that you use to represent code. They’re also persistent, which is a special characteristic we could talk about. Because of that, it encourages the functional style of programming, because you cannot change [the core data structures], so you have to write a function that produces a new value as a result. but there’s no enforcement. if you write a function and that does IO, well that’s not a pure function anymore, and I don’t preclude it, and there’s no type system to preclude it, its a dynamically typed language, which just respects the programmer’s right to do whatever they please." [1]
> And guile offers all of this while supporting native threading with a single heap.
What is Guile's synchronization model for shared-state multithreading?
> Guile is available on n900. So there.
This appears to be a Linux-based phone, so that's not very surprising or impressive. Lua is straight ANSI C and can compile on much more limited systems than a Linux-based smartphone.
One other data point: Guile is 5x the size of Lua, in both source and binary forms.
> To continue, guile has continuations (delimited or otherwise), and macros (hygienic or otherwise), both of which are effectively missing in lua.
One of the authors of Lua made an interesting point in his slides describing Lua 5.2: if you think "goto" is evil, continuations are much worse. And yet it's considered "cool" to support continuations. http://www.inf.puc-rio.br/~roberto/talks/novelties-5.2.pdf
Scheme community frowns upon mutating code. R6RS even forced that by default, though the rest of the specification was a mess.
So, scheme style these days is all about pure functions, tail calls and continuations. To think about it, that really is setting it apart from common lisp.
Th Scheme language allows, and the Scheme community encourages, functional style, but it's still up to the programmer to write their programs that way. In Haskell, there is no other way -- even imperative programs in Haskell are functional!
* It lacks a standard way for people coming from OO/Java to define objects and interfaces.
* It lacks a GUI toolkit.
* It has a great set of manuals. It lacks an O'Reilly book with a woodcut animal on the cover.
* Arrays indexes start at 1.
Except for the last item, these are all relatively small things that are simply waiting for someone to come along and do them. This is a testament to the great design of the language.
And many of these things do exist, it's just the Lua style seems to be to understate the achievements to the point that folks can't tell the big new stuff from the abandoned projects.
This sounds dangerously similar to "I can't use Python because it has significant whitespaces". Developers might like or dislike it, but if you can't get over _that_, you either lack important cognitive abilities, or you're suffering from a very serious case of Asperger's/OCD. I can't imagine that someone seriously hampered by _that_ could ever digest any large API.
However, I'll admit it's a bit irritating when you're frantically switching between C and Lua.
> the Lua style seems to be to understate the achievements to the point that folks can't tell the big new stuff from the abandoned projects.
You're right, organizing the efforts of the Lua "community" would probably be trickier than herding cats, and the Lua core team doesn't seem very interested in helping that happen. I sometimes suspect that even Lispers form a tighter community than Lua hackers...
> However, I'll admit it's a bit irritating when you're frantically switching between C and Lua.
I would imagine it can be confusing when swapping between C and Lua, but when working in a Lua-only environment (e.g. writing addons for World of Warcraft) it very quickly becomes a non-issue.
This looks excellent, even for people who don't like Eclipse. Lua sorely lacks an interactive debugger, and your DBGp-based solution is portable beyond Eclipse.
I also think that Lua has a perception of being an embedded-only language (which it's very good at). The biggest code bases are World of Warcraft interface modifications and other plugin architectures. The features Lua would need to be really competitive as a stand-alone language (ex: good library management[1]) are still developing. I think it has the potential to be very powerful, though.
If you want to have some fun with Lua, check out LÖVE[2].
Additionally, it is explicitly a batteries-not-included system. There are lots of libraries for Lua all over the net, but many people stop at "Python's standard library has hundreds of modules including the four I want. Lua's doesn't have the four I want."
Lua's goal is to be no-fat and customizable to only what you need for your unique situation. It's even common practice for systems to cut out stuff like the standard library file I/O module if it's not appropriate for your situation.
Lua's original focus as an embedded language has been a huge handicap here. There's almost no standardization, because each developer builds their own set of libraries based on their needs. It's sort of what server-side javascript looked like before node.js.
Lua is in dire need of an opinionated killer platform. Unfortunately, unfamiliarity with the language (vis-a-vis javascript) means that the road to mass adoption is a tougher one.
> Unfortunately, unfamiliarity with the language (vis-a-vis javascript)
Unfortunate especially given that Lua and JavaScript are extremely similar semantically. The difference between {} vs begin/end really does blind people to the all similarities.
That is an interesting claim. What are the first rate IDEs for Javascript, Python, Ruby or C++? That isn't a rhetorical question I mean vim, emacs, textmate all seem equally good for Javascript and Lua. I know people use Eclipse for Javascript but it hardly seems like a killer integration.
lua (the language and core libraries) is stable and small. many developers are attracted to node.js or rails due to the community ("look, everyone else is doing it!"), ecosystem (abundance of modules/gems to choose from), and rapid release cycles.
lua seems to have more of a relaxed, niche community among game developers and scripting language embedders.
i've been using lua for about a week so far, for a music player i'm building (http://cloudplay.fm/) and i'm writing the song search/ranking system in lua. it's wonderful to be able to prototype it outside the application and not have to deal with the integration until you need to.
i would consider using lua on the server-side (see http://luvit.io/ for a node.js-style interpretation, although i'd prefer coroutines instead of callbacks) but there needs to be more work on the library front.
to help lua grow, i'd fix these things:
- add unicode support
- bring the CLI up to par with node.js (i use node as a calculator, too)
- build a modern distribution that includes frequently used luarocks
Lua strings aren't C-style: they're prefixed with the string length. Lua's strings are suitable enough to store UTF-8 encoded data, but none of Lua's built-in functions are equipped to process it. Unicode libraries don't need to provide a special Unicode string type, they simply need to provide Unicode-aware string processing functions.
Right. I meant "C-style" in the sense of Lua being oriented around the kind of single/multibyte encodings used with ANSI C (as contrasted with kind of the UCS-2 and UTF-16 stuff you get with Win32 and Java).
I work at Zipline Games and we're building out a server side hosted lua system. We're targeting it at game developers, but there is nothing game specific about the lua handlers you can write. You can find out more about it at http://getmoai.com
I just want to say: thank you for MOAI. It is an awesome way to get an app done for Android/iOS platforms simultaneously, and I really look forward to the future of this framework. Without question, it has been a very enjoyable experience - develop on Mac OSX, simultaneously deploy on both Android and iOS, and then ship. Really, a pure joy to use.
As well as Mongrel and Tir, there is LuaNode https://github.com/ignacio/LuaNode/ which is modelled on nodejs. I think there is another node variant too. Also, with a slightly different feel is the Lua Nginx module which is also async with coroutines, but using nginx to provide all the IO. https://github.com/chaoslawful/lua-nginx-module
Yet another language with significant whitespace and otherwise python-ish syntax. No thanks. I would have used Python if I wanted to. Plus, any source->source translator makes debugging a pain.
>>> In Python, you might __import__('...')' some module and then the variables of the return value would be accessed like vars(module)
it makes me wonder if the author is just unfamiliar with Python (and by extension, any language other than Lua) or if they're deliberately misrepresenting other languages to make Lua look good (which it definitely doesn't need).
Sorry, I would actually consider myself quite proficient in Python. What I was trying to [unsuccessfully] show was that the underlying mechanics of Python's import mechanism are different then, say, Python's dict or list implementations. Whereas in Lua everything really is just a table. I ran into this (in Python) when I was trying to dynamically import code based on command line arguments and access the functions in said script as strings. So in Lua, you just `x = require"script" ; x["fn"]` whereas in Python you sort of need to know this information up front unless you go through the machinations I outlined in the article.
But this was obviously a poorly written part of the article that could use improvement.
Hey, thanks for the reply, I do see your point now--I thought you were referring to the usual case of importing modules. It's true that Python modules aren't implemented in terms of dicts or lists and that Python is a larger language than Lua, so I think I completely agree with the point you were making. :)
For future reference, I'd implement your dynamic module importer like this:
def dynamic_import(name):
import imp
return imp.load_module(name, *import.find_module(name))
os = dynamic_import("os")
I've used Lua for a couple of personal projects. My main objection to it is that accessing an undefined variable or member returns nil rather than throwing an exception. If you make a typo, you don't find out until you try to call or perform arithmetic on your nil value. Since inserting nil into a table just deletes that key of the table, the use site might be several steps away from the typo.
Other than that, I agree with the other posters here. It's an impressively lightweight and elegant language. It's especially good for embedding: its C integration is next to none, it's easy to sandbox if you want to run untrusted code, and the interpreter doesn't use any global variables.
function nilguard(tbl)
tbl = tbl or {}
local mt = {
__index = function(t,k)
error("Invalid key: " .. tostring(tbl) .. "[" .. k .. "]")
end
}
setmetatable(tbl, mt)
return tbl
end
local myObj = nilguard({foo=10})
myObj.crap -- should raise
For undefined global variables, just require("strict")
Does anyone know how good is Lua's support for async IO? Especially the handling of large amount of connections and the memory footprint for each connection?
Luasocket does async IO. It uses poll or select on Unix (calling it 'select'); I'm not familiar with what it uses on Windows. It's not "web scale", but is very easy to use, and works well enough for a couple hundred simultaneous connections.
Lua's convenient C API means that wrapping libev, libuv, or libevent is really not hard, if you want to go that route. (I have a libev wrapper on github, FWIW.)
The thing to keep in mind is that if you're writing high-throughput servers, you probably want to avoid parsing the network IO via Lua. While Lua strings are fine with arbitrary binary data (i.e., \0s are fine), it interns all strings; you'll have the overhead of converting every read into an atom/symbol, and it will spend a lot of time garbage collecting. (Lua makes this daring trade-off because it's usually a net win, and when it isn't, you still have the option of doing things in C instead.) Lua's great for handling all the control logic, though. Coroutines are particularly applicable there.
In a nutshell: If you're good with C, and understand the issues involved with performant async IO, Lua can work very well.
Thanks for the info. Very informative reply. Sounds like the network handling stuff should be in the C layer. And pass the parsed message/objects back to Lua is the way to go.
I'll spare you the exact results of my recent web search, but my impression is that it seems to be a rapidly developing area. There are definitely some good libraries out there of varying scope and maturity.
Lua seems to be such a good fit for this paradigm that I would bet the community is going to produce some outstanding frameworks. (Was kicking around the idea of implementing something myself.)
The article seemed a little short for me, but then I am actively trying to select between Lua and TCL for some personal scripting projects. There are many fine features with each language and few downsides, so the selection process is hard. Of course, that's a nice problem to have.
TCL is ahead by a nose at this point with Unicode support baked in (vs. using a library) and file system handling built in (again vs. using a library).
Tcl is an interesting little language as well, and the niche it was created for is similar to the one Lua was created for. The biggest advantages that Lua has over Tcl, off the top of my head, are speed and lambdas. Tcl's semantics make it very hard to optimize because it is stringly typed (with some bytecode and value specialization in the background, but with conversions between strings and other values as needed).
Tcl's file system interface is definitely an advantage if you're trying to avoid third party libraries. However, depending on what you need to do with Unicode strings, Lua's strings are "8-bit clean" and have no difficulty storing Unicode characters; Lua's string functions (the pattern-matching functions in particular) aren't Unicode aware.
Tcl has lambdas (anonymous functions): http://wiki.tcl.tk/4884. As of 8.6 (currently in beta), it also has coroutines, and has a non-recursive engine (stackless) among other niceties.
This works fine with `apply`, but if you treat it like a normal function, it does some strange things. Conversely, functions that are defined with `proc` aren't compatible with `apply` unless you wrap them up.
I'm not saying it's inadequate, just that the anonymous functions don't quite work how you expect lambdas to work.
I can't speak to what most people expect from lambdas; I force myself use Tcl's [apply] occasionally just to try slightly different paradigms, so I appreciate your commentary.
One thing to note though, your final [inc 1] has nothing to do w/ $inc. In the interactive REPL (which I'm assuming you used), Tcl will (by default) essentially autocomplete commands if it can, and [inc] completed to [incr], which is "increment". [incr] adds (by default) 1 to the named variable and returns its result. In this case, the variable name happens to be "1", later accessible via "$1", or [set 1], for example.
[edit: I erroneously initially described the "1" in [incr 1] as the integer constant 1; @groovy2shoes reply below reflects my original error. This does go to show another neat feature of Tcl: no keywords.]
I didn't know that tclsh autocompletes commands; that's pretty neat. In this case, it looks like [incr 1] is incrementing a variable called 1, and appears to start from zero if the variable isn't initialized.
I find Tcl's command model interesting because it leads to a very tiny semantics. A few years ago I spent a lot of time thinking about how it could be extended to have lambdas, and I couldn't come up with a way that wouldn't break the command model in some way or another. They aren't something Tcl needs to have, I just thought it'd be convenient. (Tcl's evaluation model allows you to simulate higher-order procs, which, along with `apply`, covers many use cases of lambda).
Ya -- interactive tclsh has a few creature-features. The autocomplete we're discussing, automatic fall-through to exec as if via sh(1), command history, history editing, and perhaps more I'm forgetting.
kamloops$ tclsh8.6
% info commands up*
update uplevel upvar
% uptime
7:17PM up 1 hr, 7 users, load averages: 0.06, 0.12, 0.17
% ls fu
ls: fu: No such file or directory
child process exited abnormally
% ls -ld fu
ls: fu: No such file or directory
child process exited abnormally
% ^fu^foo
ls -ld foo
-rwxr-xr-x 1 joe users 7300 Mar 23 2011 foo
% history
1 info commands up*
2 uptime
3 ls fu
4 ls -ld fu
5 ls -ld foo
6 history
% !2
uptime
7:20PM up 1:03, 7 users, load averages: 0.13, 0.12, 0.16
@groovy2shoes (and anybody else who's interested in modern (or historic, for that matter) Tcl), you should drop by #tcl on irc.freenode.net. We'd love to have you visit.
To my point of no keywords, and touching on the interesting model @groovy2shoes is talking about here[1]:
$ tclsh
% set a set
set
% puts $a
set
% $a b 9
9
% set b
9
% set set set
set
% $set set
set
% rename set foo
% foo set
set
% foo a
set
% foo a 9
9
% puts $a
9
%
More good points for both sides. This is not making it easier.
I don't need speed as I'm doing offline processing and not worried about lambdas at this point. The unicode is important as I am looking to take Greek and Hebrew text and process it.
It's not so much that I don't want to use a library, but more that I would be unable to use much of the core language for the processing and would have to do everything in/with the library and store my text in Unicode-aware holders rather than just strings.
If that's how it ends up, then so be it, but I'll try TCL first and know that I have Lua as a reasonable fall back plan.
If you're considering TCL, the Jim implementation might also be worth a look. It has most of the features of the core language, some extensions to enable even more and a very small footprint. If I recall correctly, it's even faster for some things and does lambdas etc. a bit better.
The bad thing about Tcl is probably its library support. It has an extension mechanism, but for a lot of "contemporary" projects (web etc.), you're actually more likely to find working Lua "Rocks" than tcl modules.
I suggest you to use Tcl, and I think I'm not biased since for instance the Redis unit test is written in Tcl but I'm using Lua as scripting language for Redis.
But as a language, Tcl will allow you to experiment a completely different way of thinking. Lua is neat but not so different from what you already know.
not often mentioned: tcl's slave interpreter, and command redirections from a slave interpreter to its master. a master interpreter can hold many slaves interps, and any slave can be a safe interpreter not execing anything that might be dangerous. redirecting, by renaming, a slave/safe interps commands the master will know. tcl's threading model (if needed): sending messages, one thread per interpreter, and one thread can have multiple interpreters. tcl-8.6 has a killer object model. tcl's vfs, and starkit/metakit makes distribution simpler.
what i do not use tcl for is: web. unfortunate lack of a framework.
it has good support for downloading stuff, ftp, http, bignums and algorithms if needed.
Disapointed because the author's link to the benchmark website lead me to the wrong impression that Lua would be almost 30 times slower than Java 7. Then I googled around and discovered the awesome LuaJit which seems to be able to compete even with C++ in performance.
What I really like is Lua's code density (see samples in the Shootout's benchmarks). Very impressive!
As an old LISPer and Schemer I would like to know if Lua Macros are really as expressive and powerful as Lisp macros. Some people claim this but I am not convinced (I would like to be convinced). The expressive power may be theoretically equivalent but this is also true for C and Assembler :-) The question is: Are Lua macros as easy to handle as Lisp and Scheme macros?
The reason its not so popular? IMHO because we live in a society that favors 1sec landing pages, customized per individual options, complexity that rivals air skyscrapers and shrinked wrapped ready-to-go solutions. This is not LUA.
LUA is just one tool in the tool box - its not the tool box.
On the integration front, one thing I will also add is that in .NET integrating Lua with your program feels pretty close to being a first-class citizen. Very nice.
When my main language was C, I used to feel that dynamically mutating or self-modifying data structures were somewhere between black magic and heresy.
I have found that being able to place arbitrary constraints on the way my programs store things in (seemingly simple) data structures has been very, very powerful.
metatables don't involve dynamically mutating / self-modifying data structures. They're more equivalent to javascript's prototype, except slightly more general-purpose. You can easily use them to build an OO system, among other things.
Yes, I agree that Lua metatables / metamethods look and work a lot like JavaScript's prototype property.
I should have been more specific when mentioning self-modifying data structures and Lua metatables in the same post.
Consider: it is possible for a table to be its own metatable, and for that table to contain (pointers to) some functions that govern storing data in the table. For the non-Lua reader, these tied-to-an-event-on-a-table functions are called table-access metamethods [0], and they allow for some dangerous shenanigans.
If these particular metamethods knew how to add to, rearrange, or delete from the set of callable functions already stored in the table, the subsequent layout and contents of the table could change in a non-deterministic way.
Something that looks like a simple assignment/update (__newindex/__index, insert/update, or your favorite pair of terms)
t[0] = "command phrase"
x = t[0]
could trigger wholesale reorganization of the table, including deletion or addition of new metamethods.
[0] "Programming in Lua 1ed (Chapter 13. Metatables and Metamethods, 13.4 – Table-Access Metamethods)" http://www.lua.org/pil/13.4.html
Could you explain what you can accomplish with Lua metatables that you can't do with python __eg__ methods? I've only seen them used as a way to implement operator overloading.
In addition to everything a python class can do, you can also use setmetatable() to modify the metatable of a table at runtime. A metatable is also just a table so you don't have to assign it to designated "metatables". You can for example even set the metatable of a table to itself.
The metatable is more general and conceptually simpler. It doesn't have to be the "class" of an "object", it can implement other patterns to OO without feeling weird, because the assumption that metatable is "Object Oriented" just isn't there.
Do you have python beginners touching metaclasses? I doubt it. Do you have lua beginners touching metatables? Certainly yes.
__index and __newindex are the key ones - effectively equivalent to method_missing in Ruby. __index gets called if you try to read a non-existent key, and __newindex gets called if you try to write to a non-existing key. You can use these methods to implement inheritance, proxies, getters/setters, all sorts of stuff.
Yes, but in Python, you set them on every object. In Lua, you can set them once (in a prototype) and have each table share them (and possibly other methods, or a reference to another when lookup fails there). This makes certain behaviors (e.g. implementing your on object system) cheap and straightforward that would be impractical in Python.
Imagine if each Python object had one field that stored all the __blah hooks, and objects could share those. It's actually more powerful than that, but it's a good start.
I'm not that familiar with Python, but can you use __getattribute__ to provide methods as well? Using __index is the way you typically implement inheritance in Lua.
If you want fast Lua I'd recommend checking out Havok Script (http://havok.com/products/script). Technically it's not Lua, but rather an extremely fast virtual machine that is compatible with Lua. I worked with it when it was Kore prior to Havok's acquisition, but it was substantially faster than the base Lua. It has some excellent debugging functionality as well.
No idea what it costs these days, but if you're writing Lua code and it's need to be fast I'd recommend checking it out.
The worst problem with Lua is that arrays start at 1. That is way too bad for me to use it. I only use it to configure Awesome WM, but it sucks for everything else IMO because of "tables" starting at 1.
I've done a lot of work in Lua and haven't really found it to be an issue. As long as you stick to doing things the "lua way" (i.e. pairs / ipairs), the specific indexes matter very little. Every once in a while I'll have an, "oh yah," moment where I use the wrong index and something doesn't work, but I don't see how it's a deal breaker for anyone.
There are many features of Lua that have caused me more problems (the somewhat inconsistent string conversion, for instance).
It is a minor mind shift. The kind I find alternately annoying, or a fun challenge, unpredicatbly.
When I was doing Lua stuff I ended up writing some FP-style helpers and other functions for iteration. This cut down on the number of times I had to think about the table indexes (and was helpful in general).
It's annoying, especially if you switch between languages frequently.
My first task in Lua was to build a solid collections library (modeled after Ruby's), so this issue comes up far more infrequently for me now. (e.g. I use :first(), :last(), :push(), etc instead of [1], [#length], [#length + 1])
Why Lua? http://itunes.apple.com/us/app/beat-boxes/id460798042