
From Python to Lua: Why We Switched - chase202
https://www.distelli.com/blog/using-lua-for-our-most-critical-production-code
======
orf
I usually don't comment on these posts, but I felt the need to in this one. A
couple of things stood out:

\- Python 2.4? Really? In 2014?

\- Yes, writing compatible code with a 10+ year old release of Python is going
to be hard. Who uses 2.4 still? You have to use the lowest common denominator,
I mean I don't think Python 2.4 even supports context managers!

\- Yes, in 2014 you should be supporting Python 3, or at least have it on your
roadmap.

\- The ethernet address (MAC?) via uuid.get_node(). Or for something a bit
more advanced you can use netifaces[1] (or even better
psutil.net_if_addrs[3]). One of Pythons core strengths is the number and
variety of the packages available. Took 2 seconds of googling to find a cross
platform solution.

\- Calling C from Python is really really simple. Use the built in ctypes
library, or something better like cffi[2]

I'm obviously firmly in the Python camp so perhaps I'm a bit biased, but I
don't see any clearcut reason to switch other than "the current code is not
optimal, let's re-write it in X", where X could be anything. I would actually
say Go would be a much better fit than Lua in this situation. I guess the
memory reduction/CPU usage is a point, but really your program seems to be
network orientated. What's wrong with asyncio? How is replacing everything
with a C library _actually_ better?

1\.
[https://pypi.python.org/pypi/netifaces](https://pypi.python.org/pypi/netifaces)

2\.
[https://cffi.readthedocs.org/en/latest/overview.html#simple-...](https://cffi.readthedocs.org/en/latest/overview.html#simple-
example-abi-level-in-line)

3\.
[https://pythonhosted.org/psutil/#psutil.net_if_addrs](https://pythonhosted.org/psutil/#psutil.net_if_addrs)

~~~
kt9
Hi, I'm the Founder at Distelli and I wrote the original version of the
Distelli agent in Python that Brian rebuilt using Lua. The main reason I
picked Python 2.4 is because we have large enterprise customers that are /
were running Python 2.4 on servers and I had to support Python 2.4 to serve
their use case.

~~~
batbomb
They were probably running RHEL5, but even in 2014 they would have been in the
Production 3 phase which is really should only be used for legacy purposes,
and that's definitely not the sort of system you push new software to. Red Hat
even pushes those customers towards virtualization because a) hardware rarely
lasts 7 years and b) they won't bother backporting drivers for new hardware to
the old kernel at that point.

I still use a few RHEL5 machines but nobody would ever dream of pushing new
software to them.

~~~
hueving
>not the sort of system you push new software to.

Telling customers that they are doing things wrong isn't really a great way to
win customers.

~~~
takeda
This is something I don't understand. If someone is using Java, they'll
install the version of Java they need. If they use Ruby, they'll install the
version of Ruby they need. If they need Lua, they'll do the same thing.

Why everyone is relying on python that's installed there for RedHat's needs
(e.g. yum).

Unlike other languages, Python is probably the best one that can have multiple
versions coexist together.

~~~
iofj
As a consultant I have very different experiences. Even in Java development
you find shops that refuse to upgrade their JVM for various reasons.

RHEL 5 is really common.

~~~
ghshephard
A lot (all that I'm familiar with) of the enterprise software packages that
I've worked with ship with their own, very specific (read: regression tested
against) version of java. Enterprise software usually requires a ton of
servers, so it's not like anything else on those servers will be using the
jvm.

------
forgettableuser
The overall tone of the responses here are surprisingly negative and doing a
lot of Monday morning quarterbacking, as if this was some kind of failure.
There are wild assumptions about their clientele and the desires of the
company, completely disregarding the fact that the company did this to meet
their own stated objectives.

They wanted something easy to deploy (single executable), easily portable,
very small CPU footprint, and very small memory footprint.

Lua is well regarded for all these things, and the company succeeded in
meeting all these objectives.

Congratulations Distelli. Nice job. And thanks for sharing.

~~~
lmm
If you rewrite your app you end up with something better, in any language.
It's an uncontrolled experiment, and so the headline is very likely misleading
- this isn't about Python and Lua. And many of us have been majorly
inconvenienced when a manager read a similar headline and declared that we
would rewrite our app in language y, regardless of whether that language is
appropriate to our circumstances.

Something like [http://roscidus.com/blog/blog/2013/06/09/choosing-a-
python-r...](http://roscidus.com/blog/blog/2013/06/09/choosing-a-python-
replacement-for-0install/) where the author actually considers multiple
candidates and performs a controlled comparison would be far more valuable.
Given that the goals sound very similar I'd be very interested to see a
comparison between Lua and OCaml.

~~~
_pmf_
> If you rewrite your app you end up with something better, in any language.

You end up with something that is superficially better, but misses a lot of
edge cases that the ugly legacy application covered. Fortunately, you have
usually moved on to save other projects, leaving lowly maintenance developers
to fix the problems, thereby creating job security. It's a win win situation,
really.

------
kt9
I'm the founder at Distelli and I just want to add a quick bit of background
around the original python decision (2.6 vs 2.4) as well as the decision to
"rewrite in Lua".

1\. I started the company as a single founder and wrote the first version of
the agent. See this HN post that gave me the encouragement to keep going
(thanks HN!) -
[https://news.ycombinator.com/item?id=6059481](https://news.ycombinator.com/item?id=6059481).

2\. I wrote the original agent in python 2.6. It was great because I could use
things like python-requests (which was awesome!). However my first encounter
with an enterprise customer showed me that was not feasible because they were
running only python 2.4 and would not be upgrading for another 2 years.

3\. I had the choice of either not getting the customer or back porting the
code to Python 2.4. I chose to back port the code to python 2.4 and get the
customer. This was a lot of work but worth it because I was able to then raise
a round with a16z. - [https://www.distelli.com/blog/distelli-series-a-
funding](https://www.distelli.com/blog/distelli-series-a-funding)

4\. When Brian joined Distelli, he actually worked on the Python code for many
months and made many improvements and added many features. When it was time to
add the streaming logs feature he built the prototype in Lua just to prove
that the idea would work and then we talked extensively to figure the
direction we should go in.

5\. We decided to move to Lua because Brian was very familiar with it and it
was a lightweight language and we could ship a single download with zero
dependencies. We also looked at using Go. At the end of the day the decision
was Brian's to make and own as the senior engineer on the project. The passage
of time has convinced me that was the right decision.

I'm sure we could have stayed with Python with all the suggestions on this
page, but we decided to go with Lua. It was a good fit for what we needed and
honestly I felt that my original code was not in great shape and would need a
lot of work to refactor and a rewrite felt like the right decision at the
time. Looking back I still think it was the right decision and I'm very happy
with the end result.

edit: grammar

~~~
meric
I have been programming Python by day and Lua by night my entire career for
the past 7 years, and I find the main difference between the two is, Python
has the overwhelming advantage in its package manager providing ease of access
to thousands of packages[1], and Lua is faster and lighter by a constant
factor. In your case, Python's package system is a disadvantage - you want to
have as few dependencies as possible.

I think your company made a good decision. The only reason not to have done it
was because your existing code was valuable, but as you've said it wasn't in
great shape and required tons of work to refactor and rewriting anyway.

[1] Lua has LuaRocks, but it's just not the same.

~~~
stuffinstuff
I looked at Lua briefly, but the immediate impression I got was that Python
gives you more help in avoiding errors. In Lua, you might be bitten by:

* variable name typos (undeclared vars are nil),

* accidentally confusing array-style tables and hash-style tables (or vice versa),

* tables with "holes" (nils) in them,

* multiple assignment with wrong number of vals or vars (mismatch silently ignored),

* variables are global by default.

Have you found this to be the case at all? Is it more difficult to avoid bugs
when writing in Lua?

~~~
meric
_Have you found this to be the case at all? Is it more difficult to avoid bugs
when writing in Lua?_

Good questions.

 _\- variable name typos (undeclared vars are nil),_

There's a linter to catch undeclared variables.
[https://github.com/mpeterv/luacheck](https://github.com/mpeterv/luacheck)
There's also some runtime checking techniques you can use.
[http://www.lua.org/pil/14.2.html](http://www.lua.org/pil/14.2.html)

 _\- accidentally confusing array-style tables and hash-style tables (or vice
versa),_

I've had that happen about as often as confusing Python dictionaries returned
by different types of functions. (Which is - not often enough for me to
remember the last time that happened)

 _\- tables with "holes" (nils) in them,_

When I'm building a vector and I want holes, I use false to fill the holes.

 _\- multiple assignment with wrong number of vals or vars (mismatch silently
ignored),_

For me, when I'm thinking about multiple assignment to function returns, it
does mean having to read documentation for a library function more carefully
than if I was reading the same for another language. If it's as simple as
"local a, b, c = 1, 2, 3", then it's no issue.

 _\- variables are global by default._

If I'm using linting, when I can't have global variables implicitly, then it's
like C or Java where I must declared all my variables and by default they are
local to the scope.

Otherwise it's just like javascript where it's best practice to always declare
variables using "var".

~~~
stuffinstuff
Thanks for the feedback! Will have a look at the linter.

------
tyingq
The described issues that prompted the switch don't really have much to do
with Python. The issues map back to running on whatever installation of Python
the customer might already have.

Lua, in and of itself, didn't solve the problems. Shipping their own, known,
Lua interpreter + extensions solved them. (which they could have done with
Python)

~~~
btilly
The reason why people don't want you to ship your version of Python is that it
takes a lot of space. Lua makes that hurdle much easier.

For example my local Python venv is about 130 MB. It symlinks in another 52 MB
of standard Python libraries. And I know it won't run without some unknown
amount of additional dependencies in C-libraries. If I wanted to send that to
someone I'm starting at 200 MB. After compression that might be 40 MB or so.
You're going to have to work to strip that down.

By comparison their download including Lua, code, and libraries is under 6 MB.

I have a minimal running Python application that was started recently and has
done nothing. It takes close to 90 MB of RAM just to start.

Theirs is generally under 10 MB while it is running.

So at every step Python requires 5-10 times as much data. If you're trying to
get other people to put you in their containers, this can matter a lot.

~~~
lmm
Or it can not matter at all. To the kind of enterprise that is still running
RHEL 6, maybe shipping a 200mb install on a CD will show them that you're a
serious business.

There are use cases where 200mb of disk space matters, but for the
overwhelming majority of applications, a 5% (say) increase in developer
productivity would be well worth adding 200mb to the install.

~~~
brimworks
The point of the Distelli Agent is to deploy software. Think of it as the
"bootstrapper". Serious customers would rather use that 200MB for their own
software, not for the "bootstrapper" :). Here at Distelli we optimize for the
customer, not for our own internal developer productivity.

~~~
lmm
Good soundbite, but if using 200MB of extra space would let you lower your
prices while offering the same service (by increasing developer productivity)
then I bet many customers would take that.

And did you try spending a person-week or so of dev time trying to minimize
the size of the Python version? That would be a much smaller one-off cost than
the costs of migrating everything to Lua, and I could easily imagine you'd get
it down by one or two orders of magnitude. (You could do the same for Lua too
I'm sure, but if we're talking about say 5mb vs 200k then that's "who cares?"
size for most customers).

------
cbetz
It isn't linked to in the article, but if you're interested in this please do
check out luvit by tim caswell ([https://luvit.io/](https://luvit.io/)). It's
a very impressive system.

At a very high level luvit looks like node.js for Lua, with some extra goodies
thrown in like being able to bake your apps into single binaries. This is
awesome in itself and the article discusses these things.

That said, I think the _major_ innovation over using javascript is that it
allows the support for coroutine-based I/O. This means you can get the
performance benefits of non-blocking node.js without the headache of
callbacks/promises. I think the linked article only touches on coroutines
briefly but they are a major piece of Lua's awesomeness that you shouldn't
miss.

Though you aren't required to use coroutines, the package management server
which hosts packages for luvit is called "lit" and it seems to demonstrate
this ability nicely:

"Lit is written in lua and uses the same system platform as the luvit project,
but is has a very different I/O style. Instead of using callbacks and event
emitters, it uses coroutines so that code can be written with simple blocking
I/O, but still maintain the abilities of a non-blocking event loop." (from the
README on [https://github.com/luvit/lit](https://github.com/luvit/lit))

~~~
etiene
I also suggest looking openresty based frameworks, it is non blocking I/O, but
you can write procedural code normally and Nginx will handle events
internally. That means _no callback hell_

There's Lapis, by leafo [http://leafo.net](http://leafo.net) And I'm also
improving Nginx experience on the framework I develop, Sailor, because it was
originally developed for Apache
[http://sailorproject.org](http://sailorproject.org)

------
tetha
On the python side, this matches my experiences with python. Python is a great
language, and I like it very, very much. However, packaging and deployment in
python remains such an amazing mess, even after 20 years.

And outside of scientific computing, python doesn't offer any real edge, to me
at least. Ruby has better tooling libraries, java/go/c++ have better
performance, JS has more powerful website frameworks.

Why should I bother with that packaging mess after having it done once
already?

~~~
ynezz
> packaging and deployment in python remains such an amazing mess, even after
> 20 years.

I'm Lua fanboy, but I've to admit, that I'm missing clean deployments and
upgrades with virtualenv and pypi available in Pyhton. I've never had any
issue so far, am I being just lucky bastard? It has been like that for many
years already.

> Ruby has better tooling libraries

On the other hand, I've honest to maintain one Redmine instance (which is
written in Ruby) and this is real nightmare for me. Each time I've to upgrade,
I've to drink a lot day or two in advance :-)

~~~
x0x0
those handle python deps ok, but not lib dependencies, at least last time I
was forced to use it. If your company has a mix of older and newer projects,
you'll end up with eg conflicting dependencies on libxml.

My solution was to docker all the things. It made my life so much better; you
could simultaneously run multiple services on the same box without awful
fragile hacks.

~~~
brimworks
Docker is a great tool, but not all customers will have that available to
them, and the docker experience isn't great for customers on Windows/Mac since
they have to run Linux in a VM, which means the agent would have the footprint
of an entire OS on those platforms.

------
qwertyuiop924
Yeah, the only real advantage of lua is that you can link your code into the
binary, and that it's super lightweight. But you can still deploy a python
binary with your code, so the only real reason for switching is, "I like lua,
and I felt the need to make my team switch."

Not that I don't lua. It's like the best parts of javascript, mixed with some
AWK and scheme, with all the crap removed. OTOH, from-1 indexing is evil.

~~~
cbetz
I find that the only reasonable complaint that can be made about Lua is the
from-1 indexing. Like: "I found this really awesome language and it's ONLY
wart is that it doesn't index arrays like every other programming language in
the world".

Yet, I still admire the Lua creators for making this bold choice. Why should
we use from-0 indexing for all eternity? Just because it's always been that
way? This is a real question if you have the answer.

~~~
tzs
In addition to Lua, these languages are 1-based: ALGOL, AWK, COBOL, Fortran,
Julia, Mathematica, MATLAB, R, PL/I, and RPG.

~~~
ufo
BTW, FORTRAN is the big reason Lua has 1-based indexing. Lua' s first users
were engineers, which were more familiar with FORTRAN.

------
viraptor
Here are some ideas that could work just as well without a full rewrite:

\-
[http://nuitka.net/pages/overview.html](http://nuitka.net/pages/overview.html)

\-
[https://code.google.com/archive/p/shedskin/](https://code.google.com/archive/p/shedskin/)

\- [https://micropython.org/](https://micropython.org/)

It would result in a similar binary, but save the full code rewrite.

------
binarymax
As a crazy exercise to learn both Python and Lua at the same time, I've been
porting a small Numpy project to Torch. I will say that both the languages are
equally nice, but Numpy is much cleaner compared to some of the seemingly
hacky things I need to do in torch.

For example. In numpy:

    
    
         x = numpy.array([1,2,3,4])
         y = 1/x
    

In torch:

    
    
         x = torch.Tensor({1,2,3,4})
         y = x:apply(function(n) return 1/n end)
    

In any case, its been a fun and interesting exercise!

~~~
omaranto
As ufo said, Lua could also support the 1/x syntax since it does have operator
overloading. Additionally, there is nothing hacky about having an apply method
taking a function.

------
BuckRogers
Any rewrite is going to be superior to the old code (in any language).

It sounds like the author just wanted to move it to Lua to me. I would only go
with Lua over Python if you needed something more embeddable. Like the game
scripting usecase it's known for. The lost libraries and language features may
make some things harder. I package up Python and my code, deploy it to other
platforms all the time with under 5MB footprint in a single executable using
PyInstaller.

That said, it'll work out fine with Lua. My question would be who will
maintain something years down the road. There's far more Python devs than Lua.
The blog said he joined in 2014 so it may end up being someone else.

------
dingdingdang
Did you ever contemplate Golang as a possible solution? Seems to me like it
would have provided a cross platform build environment superior to both Python
and Lua in terms of standardization and ease.

~~~
kt9
yes we did consider Golang but Brian' familiarity with Lua was a major factor
in the decision. Most importantly however, we foster a culture of ownership
and it was Brian's decision to own and he owned it and it was the right
decision.

------
techie128
Couldn't they have used virtualenv with Python? Also, Python packages have a
way of declaring dependencies including specifying the Python version that
your package requires. This should've guarded against any incompatibilities. I
don't think any of this was specific to Lua.

~~~
rifung
I don't think virtualenv would solve their problems here; they are installing
their agent on a customer's machine. I imagine they didn't want to ship the
entire Python interpreter or alternatively install whatever version they want
to use on the customer's machine.

But you're right, I don't think Lua is unique in being able to fix this issue.
I think it's more that the author was already very good with it and liked it.

------
bitmadness
Lua is a beautiful language, I'm glad the switch worked out for you guys!

~~~
Latty
I really don't see the 'beautiful language' part. I tried LUA for the first
time making a custom game mode for DOTA2 a while back (so admittedly not a
huge amount of experience), and the language just seemed terrible to work
with. Its focus on keeping the language simple meant that the standard library
and language features are so bare-bones doing the most basic things was a huge
chore.

~~~
brimworks
Here are some things that I think are "beautiful" about Lua:

[a] Here is the complete syntax of Lua:

[http://www.lua.org/manual/5.2/manual.html#9](http://www.lua.org/manual/5.2/manual.html#9)

...compare that to Perl where there is no "complete syntax" of Perl. This
shows how simple it is and thus it is easier to write tools for and teach
others how to use it.

[b] The complete default API consists of ~200 functions:

[http://www.lua.org/manual/5.2/contents.html#index](http://www.lua.org/manual/5.2/contents.html#index)

[c] The defacto book on learning the complete language (including the C
interface) is 366 pages:

[http://www.amazon.com/dp/859037985X](http://www.amazon.com/dp/859037985X)

Simplicity is beauty IMHO. I would expect any competent developer to get up to
speed on the Lua language within a week, it would then take them another
couple of weeks to get up to speed on the libraries we use with Lua.

The real power in Lua comes when you couple it with competent APIs that do the
"real" work. The Distelli Agent uses libuv for the OS interface, libcrypto
(part of openssl) for the crypto primitives, zlib for compression, LPeg for
parsing, sqlite for an embedded DB, etc.

The job of the Lua code is simply to "tie" these core components together.

------
vancan1ty
From the article: "The Lua standard library uses the POSIX C library, making
it portable; but that also means that you can’t use sockets, it doesn’t
support signals or timers, and all OS calls block the thread. Using the POSIX
C library is like going backwards in time."

I can't speak to the "Lua standard library" but I can say that I believe every
single statement above about the POSIX C standard is incorrect as written.

POSIX Sockets: [http://man7.org/linux/man-
pages/man2/socket.2.html](http://man7.org/linux/man-pages/man2/socket.2.html)

POSIX Signals: [http://man7.org/linux/man-
pages/man7/signal.7.html](http://man7.org/linux/man-pages/man7/signal.7.html),
[http://pubs.opengroup.org/onlinepubs/009695399/functions/sig...](http://pubs.opengroup.org/onlinepubs/009695399/functions/sigaction.html)

POSIX Timers:
[http://www.2net.co.uk/tutorial/periodic_threads](http://www.2net.co.uk/tutorial/periodic_threads),
[http://pubs.opengroup.org/onlinepubs/009695399/functions/tim...](http://pubs.opengroup.org/onlinepubs/009695399/functions/timer_create.html)

Nonblocking IO: [http://man7.org/linux/man-
pages/man7/aio.7.html](http://man7.org/linux/man-pages/man7/aio.7.html)

That doesn't mean that the POSIX features are best in class or easy to use --
rather just that the features exist.

~~~
brimworks
That was a mistake in the article. I should have said "ANSI C". I'll see about
getting it corrected.

Thanks!

------
reality_czech
Wow, the comments here are amazingly bad. A bunch of people sneering at the
author for having to support older versions of Python that his customers use.
Suggesting that he bundle hundreds of megabytes of dependencies and force the
users to install them. Can you give this guy a break?

------
solidsnack9000
It's great to see people discussing the use of Lua for systems applications.
It has a lot of advantages over many dynamic languages, as far as footprint,
performance and deliverability, while offering a comparable development
experience.

------
aeturnum
My current job is heavy in Python and a previous job was heavy in Lua. It's
hard to tell from the article, but it seems like this is an environment that
is embedded in another executable. In that context, Lua is certainly lighter
weight and luvi sounds like a fantastic extension to the language (lua scripts
are generally single threaded).

That said, anyone who has written Lua knows that scaling is difficult and
relies on really high quality code. I'd be wary about replacing python scripts
with Lua, especially if the problem with python was users wanted more advanced
features. Still, my use case isn't yours and I'm glad the migration went well.

------
scardine
It is a bit off-topic, but last time I checked the unicode support in Lua was
less than stellar.

Since that was a long time ago, can anyone chime in about the state of unicode
support in Lua?

~~~
brimworks
Lua 5.3 has a utf8 library standard now:

[http://www.lua.org/manual/5.3/manual.html#6.5](http://www.lua.org/manual/5.3/manual.html#6.5)

Full unicode support is rarely necessary in my experience. Lua "strings" are
raw byte arrays. It is the job of the programmer to validate the byte arrays
are in the character set of your choosing and doing appropriate conversions
and validation when necessary.

~~~
z92
I would rather like my language natively supports binary string, rather than
unicode string.

One can always use library functions to process the binary string as unicode.

------
lifthrasiir
Lua is great when you will write at most a few thousand lines of code, you can
ship or control your interpreter binary _and_ the number of developers who
will ever deal with the code is single-digit. Beyond that it is miserable.
Hope that Distelli Agent remains within those constraints.

------
m3mnoch
i suspect it's more about lua being a really nice, embeddable scripting
language. there's a reason game companies all* use it for content developer
scripting languages.

language preferences aside, sometimes it's all about the right tool for the
job.

* not literally _all_ of them, but lots and lots and lots.

------
dtoma
My company also uses RHEL5 quite a bit, so in order not to stay stuck on 2.4 I
use anaconda:
[https://www.continuum.io/downloads](https://www.continuum.io/downloads)

It's a life saver.

------
groovy2shoes
Minor correction: with the exception of libdl (and its equivalent on various
platforms), Lua uses only the C standard library, not even POSIX.

