

Nix 1.0 released: purely functional linux package manager - moonboots
http://hydra.nixos.org/build/2609700/download/1/manual/#ssec-relnotes-1.0

======
moonboots
The functional approach seems to work great for sandboxing binary artifacts of
packages, but configuration files and user data are more difficult to handle.
Daniel Burrows, a debian apt developer, wrote an analysis of Nix a while
back[1], and he brought up the problematic example of upgrading database
configuration file formats. It seems difficult to seamlessly upgrade and
downgrade these types of files whose semantics and syntax may change between
versions. When these conflicts occur in Debian/Ubuntu, the upgrade process
blocks and the user is prompted to resolve the 3-way merge.

I think packakge managers should only handle the storage of static files like
executables and shared libraries. Forcing the user or another process to
handle configuration files and startup/shutdown hooks has additional benefits
such as making these files easier to store in version control.

[1] <http://lists.debian.org/debian-devel/2008/12/msg01027.html>

~~~
edolstra
In NixOS (the Linux distribution based on Nix [1]) we use the functional
approach for managing all the "static" parts of the system, i.e. packages, but
also most configuration files in /etc - these get build by Nix functions in
much the same way as packages. This is good since it means that (say) a
rollback of the system will cause it to revert to a consistent combination of
packages and configuration files. (So you don't get an Apache httpd.conf that
refers to some feature that doesn't exist in the Apache you just rolled back
to.)

However, when it comes to stateful stuff such as user data or databases, the
funtional approach indeed doesn't really apply very well. NixOS manages state
in an imperative manner in the same way as every other distribution. (For
instance, NixOS uses Upstart jobs to create/update state.) That means that a
rollback of your system won't rollback (say) your PostgreSQL database, unless
somebody made the Upstart job of PostgreSQL do that. So when it comes to
state, NixOS isn't better than other distributions, but it's not worse either
;-)

[1] <http://nixos.org/>, [http://www.st.ewi.tudelft.nl/~dolstra/pubs/nixos-
icfp2008-fi...](http://www.st.ewi.tudelft.nl/~dolstra/pubs/nixos-
icfp2008-final.pdf)

~~~
moonboots
Thanks for the response. I agree that some functionality currently handled by
package managers is hairy, even for other package managers. To some extent, I
don't think this functionality should be managed by package managers. It would
hurt usability because the user would need to set up configuration files, hook
the package into the init/upstart system, and fix breakages across updates. On
the other hand, I've never been a fan of running 'sudo apt-get install
apache2' on a development machine and exposing a public port and running an
application I don't always need every time I boot up.

------
hristov
Ok maybe i am missing something, but it seems to me you may want some
mutability in a package manager. For example, updates that are simply bug
fixes and security fixes but are backward compatible should be substituted for
older versions.

Suppose you have a networking library, which is used by several applications.
All of a sudden you find a security hole in the networking library. And then
you fix it. But the fix does not affect any of the published APIs or
functionality and is thus completely backwards compatible for all legal uses.

When you fix the library you release a new version of it. Shouldn't that new
version be automatically used for all applications that use the library? If
you have an immutable packaging system, all applications will still use the
old version of the library and they would have to have their packages
explicitly modified to use the new version.

Now this may work for super high value and super secure systems, where someone
has the time to individually test the new version of the library with each
application that uses it and then individually update each application.

But for the usual desktop system this sounds like a recipe for having a bunch
of out-dated hole ridden software, where multiple badly maintained
applications still carry known security holes from many years ago; or, in
other words, this sounds very much like Windows.

~~~
moonboots
This is a potential problem, but I think in practice backwards compatible
updates and patches for Nix could work very similarly compared to a distro
like Debian. For example, when openssl releases a security fix, Debian uses
its large, mostly automated test suite [1] to verify that this new update does
not break compile and (to a lesser extent) runtime compatibility and
correctness.

Instead of an implicit promise of compatibility, a Nix continuous build and
testing system[2] could explicitly mark openssl-dependent packages as
compatible by bumping their versions. The end user would need to update a lot
more packages, but these cost could be reduced by binary diffing old and new
packages, which should be almost identical with the exception of dependency
metadata.

[1] <http://www.youtube.com/watch?v=3aoDjW1bjxo> [2] <http://nixos.org/hydra/>

------
ef4
I think that this is a stretch of the term "purely functional".

The relevant property is actually _immutability_ , which looks nice.

While immutability and functional programming often go together, they aren't
the same thing.

~~~
bumeye
There's also referential transparency.

For programming languages, that means the result of a function is dependent on
it's inputs ONLY. The function can not use any unspecified inputs like system
time or an internal state.

In Nix, that means a package can only depend on dependencies which are
specified beforehand. It's nearly impossible to use libraries or binaries
which you did not specify.

That behavior makes it easier to reason about your whole system, like
determining which packages are unused. This is similar to the advantages of
referential transparency in functional languages

~~~
emillon
> that means the result of a function is dependent on it's inputs ONLY

Not quite, it means that a function call can be replaced by its value (A
function can depend only on its inputs, yet write to a global variable, for
example).

------
shurane
The about-nix page is rather helpful on why you would want to use nix:

[http://hydra.nixos.org/build/2609700/download/1/manual/#id49...](http://hydra.nixos.org/build/2609700/download/1/manual/#id495822)

From my cursory glance over nix's advantages, it sort of looks like the
vagrant deployment system, but for package management. Or maybe something akin
to Chef and Puppet (of which I have no familiarity with whatsoever).

------
darklajid
So what is the difference to GNU stow [1], if we ignore the 'works a bit like
Haskell' description?

I like these kind of snapshots, the ability to install multiple variants. But
it's an old hat, no?

1: <http://www.gnu.org/software/stow/>

~~~
rcthompson
Nix is a package manager in the same sense as apt and yum. It resolves
dependencies, it doesn't just install software.

------
mjs
I don't understand this. In particular I don't understand the sense in which
it is "purely functional". Surely a package manager needs to mutate the state
of a system, which seems to be an inherently non-functional operation--? If I
have a system S1, and then I do S2 = install(S1, "firefox"), does S1 still
exist?

This bit is also interesting: "Runtime dependencies are found by scanning
binaries for the hash parts of Nix store paths (such as r8vvq9kq…). This
sounds risky, but it works extremely well."

~~~
jerf
Purely functional does not mean "has no state at all". This meme needs a stake
stuck through it, though beats me how to do it. Functional languages exist and
do real work.

In this case it means that packages are handled as functional programming
languages handle data values. New ones are created from old ones, and
unreferenced values are cleaned up via garbage collection.

~~~
thurn
It doesn't seem unreasonable to think that "purely functional" means
"consisting of pure functions". Why not just say "functional" if that's what
is meant?

~~~
9mit3t2m9h9a
The aim of Nix is to make function from dependencies to resulting packages a
pure function. Nix seems to have achieved it with respect to set of packages
installed on a system; the most problematic thing now is that some packages
manage to detect CPU version at build time... Fortunately, most of them allow
to disable this feature.

------
_seininn
I may be naive in asking this, but WHY? Why is being "purely functional" an
advantage for package managers?

~~~
stephenjudkins
<http://nixos.org/nix/> summarizes some of the advantages of this approach. To
summarize, it promises to be fundamentally safer and more flexible than
traditional package managers. Package management, IMO, is an area where a
"purely functional" approach could really shine, since correctness and
replicability are so important.

------
shasta
To everyone who "doesn't understand", go read the "about Nix" section of the
link.

------
agumonkey
Someone on reddit showed us this Gnome project : OsTree
<https://live.gnome.org/OSTree/NixOSComparison>

------
daivd
Since Nix keeps multiple versions of almost the same binaries, it seems to me
that something like ZFS, where several files can share the same physical block
on disk, is the perfect match. Is that something you recommend when using
NixOS?

------
rcthompson
Does anyone have experience using Nix (or NixOS) for practical purposes
(instead of just testing it out) that they'd like to share?

~~~
torkins
At LogicBlox (<http://www.logicblox.com/>), we use it to deploy our customers'
applications both for testing and production. We also use hydra
(<http://hydra.nixos.org/>) to run our buildfarm. We're soon going to use
NixOS for our deployments. If it wasn't clear from that, we're fans... :)

------
snprbob86
I'd really like to see an example of the "Service deployment" approach
discussed in the overview.

------
Cieplak
Can anyone explain advantages this might have over, say, Synaptic?

~~~
jerf
I've been eying this as a replacement for Gentoo. I like it well enough for
the configurability, but I tire of the way I'm rolling the dice every time I
update a package. (Binary distros like Ubuntu have this same problem, just
less so.) The primary use case for me would be enabling me to stay cutting
edge on software, while making distro upgrades reversible. If my new kernel
blew up my system, I don't go scrounging around for boot disks, I pick the
previous image on next boot.

I don't know of any other distro that can pull that use case off right now. Or
any other OS, really.

~~~
mappu

        If my new kernel blew up my system, I don't go scrounging around for boot disks, I pick the previous image on next boot.
    

Debian and Fedora's package managers both keep the last three kernels
operating and in grub.

(This is a hassle for me since Fedora has had a long string of updates since
2.6.36 which was the last one that gave me working r/w HFS+ support, and i
have to keep removing the second-to-last kernel manually with the package
manager.)

~~~
tene
FYI, there are a few yum options you can use to tweak this behaviour to how
you want it to be, including 'protected_packages' and 'installonly_limit',
described in yum.conf(5).

~~~
mappu
Oh neat, thank you.

------
strictfp
First yum - "a package manager written in Python" and now Nix - "a purely
functional package manager". Well, I for one do not care the least which
language or paradigm my package manager is using. What I do care about is
compatibility, reliability, speed and correctness. How is Nix improving the
situation for the user?

~~~
jamii
"Nix is a purely functional package manager. This means that it treats
packages like values in purely functional programming languages such as
Haskell — they are built by functions that don’t have side-effects, and they
never change after they have been built. "

"You can have multiple versions or variants of a package installed at the same
time. This is especially important when different applications have
dependencies on different versions of the same package — it prevents the “DLL
hell”."

"Nix helps you make sure that package dependency specifications are complete."

"Nix has multi-user support. This means that non-privileged users can securely
install software. "

"Since package management operations never overwrite packages in the Nix store
but just add new versions in different paths, they are atomic ... And since
package aren’t overwritten, the old versions are still there after an upgrade.
This means that you can roll back to the old version"

"In addition to downloading binaries automatically if they’re available, Nix
can download binary deltas that patch an existing package in the Nix store
into a new version."

"Nix can be used not only for rolling out packages, but also complete
configurations of services. This is done by treating all the static bits of a
service (such as software packages, configuration files, control scripts,
static web pages, etc.) as “packages” that can be built by Nix expressions. As
a result, all the features above apply to services as well: for instance, you
can roll back a web server configuration if a configuration change turns out
to be undesirable, you can easily have multiple instances of a service (e.g.,
a test and production server), and because the whole service is built in a
purely functional way from a Nix expression, it is repeatable so you can
easily reproduce the service on another machine."

~~~
fffggg
"You can have multiple versions or variants of a package installed at the same
time. This is especially important when different applications have
dependencies on different versions of the same package — it prevents the “DLL
hell”."

This sounds nice, and it's a direct quote from the product page, but I can
guarantee that Nix does nothing of the sort. This would need to be addressed
in ld.so. It's simply not possible for a packaging system to solve this
problem on a typical unix system.

~~~
edolstra
Feel free to try it out to see that it does, in fact, do this. No special
magic is needed (most of the time): for instance, for dynamic library
dependencies you can just set the RPATH at link time to indicate precisely in
which directory an application should look for its dependencies.

~~~
fffggg
No, it won't work. This is my area of expertise and I can unequivocally state
that it is not possible to solve this problem on a unix system. The linker
infrastructure does not exist, and here's why:

RPATH (or better RUNPATH) has intractable conflicts in library dependencies.
For example, this cannot work for any application which uses dlopen() and
packages its loadable modules independently of the invoking binary. This is a
problem for very high profile software such as:

* Apache, and just about any other application server with a plugin interface * Perl, Python, Ruby, Java, and just about every other interpreted/VM oriented language

Furthermore, there exists other intractable conflicts in complex library
dependency interactions. For example:

bin/A needs SONAME=B.so.1 needs SONAME=C.so.1 (v1)

bin/B needs SONAME=D.so.1 which needs both SONAME=C.so.1 (v2, incompatible)
and SONAME=B.so.1

The dependency relationship of every shared object in this chain are not
static, and you cannot modify the invoking binaries (where RPATH must be set)
every time an intermediate shared object changes its linking requirements. You
also cannot guarantee ordering when a single node depends on multiple
libraries -- the order of evaluation is non-deterministic. You might say this
is a corner case, but you would be surprised when you try at scale. This is
not a corner case.

Seriously, I understand everything about this. Can't be done. _You_ feel free
to try it out and see that it _doesn't_.

