
Libucl: Universal configuration library parser - pykello
https://github.com/vstakhov/libucl
======
IgorPartola
Allow me to shamelessly throw my project into the ring:
[https://github.com/ipartola/groper](https://github.com/ipartola/groper). It
is Python-specific, but for me it fulfills a very real need: to have a single
library that I can use to both read a config file _and_ command line
parameters. getopt/argparse/optparse do not do config files, while
ConfigParser and friends are at least less than friendly to use and definitely
don't to command line arguments.

My colleagues and I have used this library in at least a dozen projects and it
worked great. I am actually wondering if this is worth writing a PEP about to
have something like this included in Python's standard library.

Edit: I forgot to mention my favorite feature of this library. As you develop
your program, you define the parameters that each module will use, optionally
with defaults. When you are done, you can get groper to generate a sample
config file for you that includes all the options you defined in your program.

Edit 2: It looks like there are a few people downloading this package from
PyPI which makes me very excited. Please feel free to let me know how your
experience was with groper and what features you'd like to see added.

------
barrkel
Before trying to create yet another configuration library, consider whether
your efforts wouldn't be better spent on adding another language binding to an
existing configuration language.

I'm rather partial to TypeSafe Config myself - similar to the extended JSON
described herein.

------
ay
[https://github.com/LuaDist/lua](https://github.com/LuaDist/lua)

The repository size is exactly the same as the submission, but you get a
fully-fledged language - which allows you to make parametrized configs, etc,
etc, etc.

And the syntax is quite convenient. Look at it for your next project.

~~~
mercurial
Turing-complete, non-easily parseable by external tools configuration? No
thanks.

~~~
andrewflnr
What would make it easier to parse by external tools than a portable library
that's insanely easy to embed? I mean, there's not that many now, but that's
how every idea starts, good or bad.

Whether Turing-completeness is a good idea depends on how much you trust the
configurers. In cases where they're writing code anyway, you may as well have
configuration be code too, among other cases.

~~~
mercurial
> What would make it easier to parse by external tools than a portable library
> that's insanely easy to embed?

A small, documented spec? Something you can easily validate, even if it
doesn't feature a schema?

> Whether Turing-completeness is a good idea depends on how much you trust the
> configurers. In cases where they're writing code anyway, you may as well
> have configuration be code too, among other cases.

It makes it much easier to misconfigure the application and, depending on how
much api surface you're going to expose, can prove to be an absolutely
terrible idea in terms of maintenance (I'm looking at you, Awesome window
manager). I'll take a "dumb", predictable configuration language for which I'm
extremely likely to already have bindings in whatever language I'm working
with than having to embed a Lua parser. Look at Ansible for how you can make a
DSL with YAML for a good example of how it can be done.

~~~
andrewflnr
I actually find YAML the most annoying part of Ansible, but that's probably
specific to YAML. The syntax drives me crazy.

~~~
mercurial
Really? I find YAML pretty nifty myself, though it has a few gotchas. Many
softwares can get away with nothing more than INI-style configuration, but for
more complex cases, YAML is much nicer to edit than, say, XML or JSON.

------
specialist
Very nice.

I wrote ARON (a righteous object notation)

[https://code.google.com/p/aron/](https://code.google.com/p/aron/)

Whereas your Libucl comes from the config management world, ARON is derived
from my prior VRML work. Though ARON is a general purpose object graph
serializer, I currently mostly use it for config, mostly as sane alternative
to the DI and IoC nonsense in the Java world (Spring).

Both Libucl and ARON have a minimalist, typed grammar. Personally, I very much
dislike goo like JSON. So you have my enthusiastic support.

I applaud the syntactic sugar you added, like units suffixes, eg 1k. I added
UTC date time. I'm still pondering how to add URIs, both relative and
absolute.

For reuse, ARON uses labels and references (my take on VRML's DEF/USE) vs
Libucl's macros. Your strategy hadn't occurred to me.

Libucl's try/catch is VERY interesting. ARON's include is modeled after Java's
imports, where declarations are at the top of the source file. Libucl's
strategy is far more robust.

Okay. Enough stream of conscious replying.

Libucl is great.

Ignore the haters.

------
mercurial
Damn. I thought "universal" meant "can parse multiple configuration formats
(JSON, XML, INI, YAML...)" but it just looks like yet another configuration
format.

------
nly
Unfortunately external configuration is much more than just serialisation. For
me, an ideal universal configuraion library will provide schemas, type
checking, full structural validation, and externalised defaults so I can tell
the difference between a value _explicitly_ set (or reset) to the default by
the user, and a value that has never actually been changed. Hardcoded defaults
are evil, configuration is actually a stack of developer-set defaults,
packager/distro set defaults, sysadmin-set defaults and user preferences.

Imho these are much more tiresome problems than serialisation imho. If I were
going to build a configuration library I'd probably use Apache Avro[1] as a
base. It has a lot of these ingredients already.

For parsing atm, since I write mostly C++, I'm prone to just using Boost
Property Tree[2], which exploits the fact that most of these formats (even
XML) are just trees of key-value pairs. There are parsers for XML, JSON, INI
and INFO formats[3]

With regard to this project, it'd be nice if the author would write a complete
grammar for his UCL format. The advantage of JSON is it's really well
documented[1] and I can always go and grab another implementation. There's
nothing in this repo except the hand-written C code. Other than that, the
cleanups to the syntax look nice and friendly.

[0]
[https://avro.apache.org/docs/current/](https://avro.apache.org/docs/current/)

[1] [http://www.json.org/](http://www.json.org/)

[2]
[http://www.boost.org/doc/libs/1_55_0/doc/html/property_tree....](http://www.boost.org/doc/libs/1_55_0/doc/html/property_tree.html)

[3]
[http://www.boost.org/doc/libs/1_55_0/doc/html/boost_property...](http://www.boost.org/doc/libs/1_55_0/doc/html/boost_propertytree/parsers.html)

~~~
specialist
_"...much more than just serialisation. ...provide schemas, type checking,
full structural validation, and externalized defaults..."_

I agree with your goals. But I'm confused by the serialization concern.

If one has classes, modeling the configuration schema, and you serialize those
config object graphs, isn't that what we're talking about here?

------
lifeisstillgood
For me the problem with configuration is not the config files - but getting
the config into the running application. There is never a perfect way - it is
a chicken and egg situation - pass it in from the command line? Ok now do that
for 100 servers, I know let's have an app - but that needs to know which 100,
for which we write a config file ...

it's all doable, but I always think I could make a better choice.

~~~
hugofirth
Interesting - so would you be looking for some kind of centralised
configuration deployment system?

~~~
emmelaich
Augeas plus ssh (ansible, whatever) plus some special sauce will do it for you
mostly.

~~~
icebraining
As an amateur sysadmin, I'm curious - why use Augeas instead of having a
template of each config file be filled in and copied to each node?

~~~
phaer
Augeas provides some abstraction over different config formats so you don't
even have to know the syntax and are less likely to make typos.

------
pawelo
Project idea: a tool to compile yaml(/xml/json) files to apache, nginx, php or
any other format.

~~~
thu
Maybe [http://augeas.net/](http://augeas.net/).

~~~
pawelo
I was thinking about converting from a "well known format" to application
specific format. File formats like yaml or xml are more popular then augeas
tree. Or maybe you are trying to say that augeas supports this scenario?

