

Should Apache replace its current httpd.conf with Lua? - compay
http://thread.gmane.org/gmane.comp.apache.devel/37627

======
silentbicycle
While this may sound like it's coming out of nowhere, Lua was designed
primarily for use in extension/configuration, and has been used in production
this way for about fifteen years. It's a scripting engine, config file format
parser, JSON-like data format library, etc., all in a library a bit smaller
than SQLite.

The lua interpreter is essentially just a "code = read(); f = compile(code);
print f()" loop written in the C API, with better error messages. The C API is
primary, not the standalone language, and using it with existing C projects is
quite easy. (As a language, it's very much like a twin of Javascript that was
able to mature for another decade, rather than suddenly getting standardized.
Most of the flaws Crockford describes in "Javascript: The Good Parts" were
fixed in long ago in Lua.)

More specifically relevant to apache, running code in a sandbox is trivial.
You can use setfenv() to control the environment, global packages, etc.
available to any code. If code shouldn't access the compiler, OS or debug
packages, etc., just pass it an environment without them.

There's also a chapter in _Lua Programming Gems_ (<http://www.lua.org/gems/>)
(it's "Lua as a Protocol Language" by Patrick Rapin, no sample PDF though)
that shows how to set up a simple filter at the bytecode level to block loops,
guaranteeing that code will always terminate. With a few minor changes, it's
completely safe to use as a network protocol language.

------
garnet7
Surprised to see so much discussion on that. Instead, would expect someone to
simply implement it, throw it on the table, and provide instructions for
building it into your own apache.

If it catches on, then you have a replacement to the current config syntax.

That said, in my experience, admins are _slow_ to change. There are still
sites running Apache 1.3 out there.

~~~
fsniper
Admins are not slow to change. Operations and Corporations are slow. Admins
generally have more than one version of any software running. More than you
can imagen.

Generally they do have development versions daily updated and built and tested
under the hood. Generally they do provide support and feedback to these
software by bug reports or even with changesets.

But they are not slow.. most of the time they do work out of your eye sight.

I can bet some admins are trying to see if mod_lua can be a good replacement
for the current configuration.

For myself I do not like it to happen. Because most of the deployments of
apache are huge and this will break too many things.

And yes, if it's working do not change it :)

(edit: gramer and typo corrections)

~~~
randallsquared
As someone who has been an admin (both with and without that job title), I'll
say that _I_ was slow to change, and often still am. Why? Because every change
is an opportunity for something to go wrong, and in my experience, something
goes mildly wrong about half the time, and catastrophically wrong way more
often than it should. This is _after_ doing all the testing that can usually
be done. Either there's no testing server for this client, or you can't
replicate the exact conditions of the change because you don't have another
Windows 2003 SBS handy to test interaction with, or you run into lots of
issues in the test and can't easily roll back to make sure that you can do it
flawlessly this time...

Unless you've got an employer or client that can afford you for an additional
20 hours for each major change, everyone involved just crosses their fingers
and holds their breath. I've never been an admin in an environment where there
was adequate funding, though, so maybe it's just me.

Anyway, I typically put off upgrades and new installs that affected old
servers (new apache modules or whatever) as long as possible.

Edit: this is why I just code, now. It's easier to write stuff that works than
to find the magic this-THEN-that incantation to make someone else's stuff
work. ;)

~~~
garnet7
Admin work is weird because when everything's working, no one says a peep. No,
"hey, great job with that update last night which we were largely unaware of
because it all happened seemlessly!". It's only when something goes wrong that
anyone says anything.

------
pilif
I wonder what is wrong with the current way of configuring apache. I might be
biased, but I have experience with both lighttpd and apache and I always found
the apache way of doing things perfectly fine and actually prefer it to the
kind-of procedural style of lighty.

Let's just hope they don't switch to an XML based configuratino file. We don't
need even more filler-code in our configuration files.

XML just isn't human readable (and writable! and, yes, you DO write
configuration files here and then) enough to be convenient. IMHO at least.

~~~
compay
As far as I can tell XML config files (at least those intended to be hand-
edited) have largely fallen out of favor in non-Java open source projects.

I also don't mind the current config file, but I've been using Apache for more
than 10 years so I've probably lost all perspective at this point. Having the
config file in Lua would certainly make it eaiser to automate, validate and
document though.

~~~
pilif
well... the XML approach _was_ in fact proposed in the linked thread.

mod_perl already allows for a totally dynamic configuration which should be
sufficient in terms of automation.

The documentation is quite perfect already and would IMHO lose quality if the
syntax changes this drastically.

The only point I'm willing to hand you is the validation argument, although
the current parser seems to be doing a decent enough job already.

No. Discussions like these feel to be motivated by artistic considerations
instead of practical ones. Think of all the external documentation such a step
would invalidate. Books, Tutorials, Blogs. And think of the external addon
modules that would probably break because of this.

Remember the 1.3 => 2.0 switch and how long that took and consider that 2.0
actually had some real advantages compared to 1.3.

~~~
compay
"well... the XML approach was in fact proposed in the linked thread."

Yes, but, my point was that I see it as very unlikely to be adopted.

"The documentation is quite perfect already"

Sure it's well-written but "perfect" may be a stretch. Having a more
consistent syntax would allow the documentation to be much more terse.

The current documentation reads like a manual, which is great. What's being
proposed would mean that the documentation could be presented as a manual, AND
as an API.

"The only point I'm willing to hand you is the validation argument"

I think you should hand me "automation" as well. :-) Writing a tool that
interprets a Lua config file would be trivial. Writing a tool that interprets
the current Apache config is not.

Anyway I'm not really advocating this change. I simply find the proposal
interesting because I do a lot of programming with Lua and it's nice to see it
get some high-level recognition.

------
growt
I prefer the nginx config style over apache2. Why the hell would anyone need a
turing complete configuration file syntax?

~~~
barrkel
Turing completeness is a red herring and not an argument against using a
language. Turing completeness is usually practically useful only to prove that
certain analyses are impossible.

As to why, sometimes you want the flexibility of inferring settings from
third-party data sources, or implicitly from the layout of the directory
structure, or naming conventions, or any other thing you choose to mention.
IMO the power of a programmable configuration language is that the
configuration need not actually be written in the language, but stored
somewhere else, and just retrieved using the language.

------
tvon
I like this:

[http://thread.gmane.org/gmane.comp.apache.devel/37627/focus=...](http://thread.gmane.org/gmane.comp.apache.devel/37627/focus=37644)

> I would like to see the _output_ of configuration be a struct (of structs),
> and the actual config files are never looked at again (until reload). At
> that point we don't need to care how it is built, and we can try using lua,
> vcl, xml, jelly, groovy, windows ini files, or plists :)

I wonder what's going on with this, I just noticed the whole discussion is
from early June...

------
jrconlin
Wait, what?

key=value is really so hard that we need to replace m4 with lua? Honestly?
Before you feel the need to flame, imagine if this proposal had been for
implementing the config file in COBOL (a language designed to be code readable
by non-technical people). Would you still believe it to be a good idea?

What ever happened to wanting to keep things simple?

~~~
gjm11
m4? key=value? Have you ever either read or written an Apache httpd config
file?

~~~
jrconlin
yep, on a daily basis. I also handle php.ini, java properties files and a slew
of others. (Including the ever loving satan spawn of sendmail.cf).

All of which are simple key value pairs. Apache (and a few others) introduce
scoping with psuedo XML, but again, it's not complex logic.

I brought up m4 because that was the last time someone tried to solve the
configuration problem with a "simple and easy" macro language. Yeah, that
worked great.

Layering Lua (or any language) as your configuration option will not simplify
things. In fact, I'm willing to say that it will make it much, much harder as
folks suddenly become increasingly "clever" about how they construct those
config files.

I want my config files stupid. I want them heavily documented so that when I
(or someone else) next has to manage them two years from now after working on
multiple other projects in various languages and systems, they don't have to
spend a week trying to remember what the hell this is doing, deciphering
cryptic compile time errors, or wondering what the complex state machine
construct someone layered on top of port allocations is supposed to be doing.

~~~
chad3814
clearly you've never ran any big site (yahoo, netflix, unitedheroes, etc)
where complex configs are _required_.

~~~
thras
I don't see how that's clear at all. And which of those three do you work at,
btw?

~~~
jrconlin
heh. I used to work at Yahoo, I currently work at Netflix, and my personal
site is unitedheroes.net.

I also know chad and he's being a dork. (hey, i appreciate a well crafted
troll.)

------
davidw
Tcl's nice for this kind of thing too, as it's really easy to make your own
"DSL"'s (or is that passe`?)

    
    
        directory "/usr/local/foo/bar" {
            options bee bop doo wah
            if { $someotheroption } {
                order allow deny
            }
        }
    

Lua might or might not be better for processing requests: it's sure fast.
Config files don't really need to be fast though.

------
mgrouchy
Its a neat idea, but I would echo the sentiment brought up in the thread
linked that casual users would have a much harder time with a lua config in
comparison to the current one.

Its probably a good idea to move in this direction in the long run, but there
is always a risk of displeasing your current users.

~~~
scott_s
I wonder if the people who made those comments are in denial about the
complexity of Apache configuration files. Having never seen one, I Googled
"apache sample config file" and I came up on this page:
<http://www.pantz.org/software/apache/apache13config.html>

This is representative of the more complex entries:

    
    
      <IfModule mod_alias.c>
    
        #
        # Note that if you include a trailing / on fakename then the server will
        # require it to be present in the URL.  So "/icons" isn't aliased in this
        # example, only "/icons/".  If the fakename is slash-terminated, then the
        # realname must also be slash terminated, and if the fakename omits the
        # trailing slash, the realname must also omit it.
        #
        Alias /icons/ "/usr/local/apache/icons/"
        Alias /icons/ "/var/www/icons/"
    
        <Directory "/usr/local/apache/icons">
            Options Indexes MultiViews
            AllowOverride None
            Order allow,deny
            Allow from all
        </Directory>
    
        #
        # ScriptAlias: This controls which directories contain server scripts.
        # ScriptAliases are essentially the same as Aliases, except that
        # documents in the realname directory are treated as applications and
        # run by the server when requested rather than as documents sent to the client.
        # The same rules about trailing "/" apply to ScriptAlias directives as to
        # Alias.
        #
        # ScriptAlias /cgi-bin/ "/usr/local/apache/cgi-bin/"
        ScriptAlias /cgi-bin/ "/var/www/cgi-bin/"
    
        #
        # "/var/www/cgi-bin" should be changed to whatever your ScriptAliased
        # CGI directory exists, if you have that configured.
        #
        <Directory "/var/www/cgi-bin">
            AllowOverride None
            Options ExecCGI
            Order allow,deny
            Allow from all
        </Directory>
    
        #
        # "/usr/local/apache/cgi-bin" should be changed to whatever your ScriptAliased
        # CGI directory exists, if you have that configured.
        #
        # <Directory "/usr/local/apache/cgi-bin">
        #   AllowOverride None
        #   Options None
        #   Order allow,deny
        #   Allow from all
        # </Directory>
    
      </IfModule>
    
    

That _is_ a language, and it's one that exists solely for Apache config files.
I think using an actual programming language would be easier for most people.

~~~
silentbicycle
[http://people.apache.org/~rbowen/presentations/apacheconEU20...](http://people.apache.org/~rbowen/presentations/apacheconEU2005/hate_apache.pdf)

Right, and it's a language with a brittle, inconsistently documented, largely
ad hoc syntax. While Lua's syntax has its quirks (the syntax for lambdas is a
little cumbersome, arrays are indexed from 1), that's because it is designed
to be straightforward for simple use by non-programmers. The advanced
features* don't get in the way if you're only using it as a log/config/etc.
file format.

* Tail-call optimization, closures, coroutines, etc.

~~~
compay
Thank you, I was looking for a link to that presentation this morning and
couldn't remember who had given it.

------
termie
<Perl>

#!perl

use DBI;

use Blah;

...

push @{ $VirtualHost{'*'} }, {

    
    
            	ServerAdmin     => "webmaster\@$domain",
    	        ServerName      => "$domain",
                    ServerSignature => "Off",
    	        ServerAlias     =>  \@aliases,
    	        AddHandler      =>  [qw(cgi-script .cgi)],
    	
    	        DocumentRoot    =>  "/home/$domain/www",
    
    	        Alias           =>  [
    			["/cgi-bin/", "/home/$domain/cgi-bin/"],
    			["/stats/", "/home/$domain/stats/"],
    		],

}, ...

------
antirez
I think that config files should be simple to write DSLs, and not programming
languages. For automation what is really needed is an API exported by Apache
via some kind of networking (a TCP socket could do the work) that allows to
configure and change the behavior at run-time via a scripting language. This
way you have the best of both the words.

~~~
ciupicri
Have you looked at Augeas <http://augeas.net> ?

Augeas is:

\- An API provided by a C library

\- A command line tool to manipulate configuration from the shell (and shell
scripts)

\- Language bindings to do the same from your favorite scripting language

\- Canonical tree representations of common configuration files

\- A domain-specific language to describe configuration file formats

------
est
It would be cool if we can use runtime variables in Apache's config

------
hypermatt
Haha I remember Brian from CNN he was talking about this years ago, I guess it
takes a lot of momentum to change something as big as apache.

------
SwellJoe
OK, for anyone who hasn't seen it before, this is what "second system effect"
looks like just before it gets really ugly.

------
stcredzero
Should [X] replace its [Y] configuration with [programming language here]?

Maybe. Depends. Is there a way you can try it out?

Reminds me of Zed Shaw's "The ACL is Dead." <http://vimeo.com/2723800>

I can also see that turning in to a quagmire.

