
NixOps – Declarative cloud provisioning and deployment with NixOS - mrkgnao
https://nixos.org/nixops/
======
mrkgnao
In view of some replies downthread: yes, application-level management is
entirely possible with Nix(OS).

Currently, this works best for Haskell, for which there is a sizeable amount
(relatively speaking) of Nix tooling available. This is largely because a lot
of Nix(OS) users also use Haskell, perhaps because lend themselves to very
similar modes of "declarative" thinking.

This is a thorough, excellent guide on how to develop Haskell projects with
Nix, including managing non-Haskell dependencies (zlib, SDL, ncurses, and so
on):

[https://github.com/Gabriel439/haskell-
nix/](https://github.com/Gabriel439/haskell-nix/)

I only started using NixOS about a month ago, and it's fantastic. The best
part is that binary caches are available for almost everything, so the
hitherto-painful step where one compiles Haskell dependencies locally has been
completely eliminated. The best way I can describe how I work now is "Python
virtualenvs, but for everything at once and just so much better".

PS. Gabriel Gonzalez is, in general, someone who makes great documenation.
Check out the tutorial for his stream-processing library Pipes: the ASCII art
alone is awesome :)

[https://hackage.haskell.org/package/pipes-4.3.4/docs/Pipes-T...](https://hackage.haskell.org/package/pipes-4.3.4/docs/Pipes-
Tutorial.html)

~~~
Ixiaus
We use NixOS in production in our product, for our cloud services, and for
some development environments.

We build everything with nix from sbt projects, go projects, haskell projects,
purescript projects, docker images, and npm projects.

It's an incredible tool.

NixOps is also a good tool but there are a lot of issues with it we haven't
had time to contribute fixes for. We also wanted it to be lighter weight and
to decouple system specification from system deployment. So I wrote a tool
that handles performing the same system deployment operations that NixOps does
for you and you can simply pipe the nix path result from nix-build to it. We
use terraform to specify and manage the cloud resources.

~~~
tryckbarpingvin
Are you able to share your tool?

~~~
Ixiaus
We will definitely be publishing it. Time is very short and the backlog of
projects to publish is growing though so it might take time to polish and
write a blogpost and tutorial.

I work for Awake Security so keep an eye out on the company's GitHub org as
we've been publicly releasing a lot of projects even though they aren't
completely polished or announced yet. I'll see if I can do this for our deploy
tool soon.

~~~
iElectric2
Nice, I'd love to see it released.

------
zbentley
Other than integration with the rest of the Nix/NixOS ecosystem, how is NixOps
different from Puppet?

Reading briefly through the examples, it seems very oriented towards
provisioning machine instances rather than provisioning behavior across
networks or multi-machine services, which is something that Puppet really
excels at. What features does this offer in addition to ones that Puppet
provides?

That said, I've been getting tired of Puppet recently for some unrelated
reasons, so I'll definitely give this a try.

~~~
solatic
Linux wasn't designed to be an immutably-configured operating system. When
systems like Puppet try to apply configuration to a target server, they're
operating on top of a layer which is inherently stateful, which has a
propensity to cause all sorts of problems. What if somebody manually entered
this server and screwed with the configuration, outside of Puppet? What if
poorly packaged applications don't cleanly uninstall or upgrade properly?
Systems like Puppet tend to just throw an error at the sysadmin and walk away.

NixOS treats immutable configuration as a first-class citizen at the OS level.
Dependenices are not installed into /usr or /etc, rather specific versions of
packages (identified by their hashes) are installed into /nix/store and then
symlinked back to /usr or /etc. Different applications that require different
versions of the same dependency, when packaged according to standard,
harmoniously co-exist on the same server because the Nix packaging system
symlinks the dependencies between each other in /nix/store.

You don't realize the full power of this setup until you make a mistake, make
your system unbootable, and calmly tell GRUB to just use the last valid
configuration - a few minutes later and you're back at a desktop. Conventional
configuration management tool like Puppet tend to be worthless in those kinds
of failure modes, and most sysadmins would be implementing backup restoration
procedures.

~~~
aseipp
Here's another example that is simple, but one I still appreciate: I wanted a
slightly newer kernel. How do I get it, and ensure all the "downstream"
dependencies are rebuilt (e.g. kernel modules)?[1]

I write this into my configuration.nix:

    
    
        boot.kernelPackages = pkgs.linuxPackages_latest;
    

I run `nixos-rebuild switch && reboot` and I'm done. I can switch back to the
last one if I wanted, but this new one takes place immediately. I just did
this on a server 20 minutes ago.

There are other small things I can do with this. What if I didn't want to boot
this kernel to boot, just test it?

    
    
        $ nixos-rebuild build-vm
        $ ./result/bin/run-*-vm
    

This will instantly drop you into a QEMU-KVM instance that is using NixOS with
your `configuration.nix`, but if you rebooted nothing would change. (You can
also specify a different configuration.nix to test _other_ machines![2]) Or I
_could_ boot it, but just once and switch back pretty easily if something was
wrong.

\---

Another: What if I wanted Tarsnap backups of something? (This may be out of
date, but it's my old config, and I wrote the Tarsnap module)

    
    
          services.tarsnap.enable = true;
          services.tarsnap.archives =
            { nixos =
              { checkpointBytes = "5GB";
                directories =
                [ "/root" "/etc/nixos"
                ];
              };
    
              home =
              { excludes = [ "*.o" "*.hi" ];
                checkpointBytes = "5GB";
                directories =
                [ "/home/a"
                ];
              };
            };
    

Now I can do things like this:

    
    
        $ systemctl start tarsnap@home
        $ systemctl start tarsnap@nixos
    

To start on demand. Or I can configure timers/cron to do this manually.

Lots of other small things. Binary caches, your own Hydra, remote/multi-
architecture builds, etc. One time I "configured" my NixOS laptop install
before getting my laptop: on a remote testing server, over remote KVM. Pushed
it to git. I cloned my configuration.nix during NixOS installation on my
laptop a few days later, and I replicated my laptop in 10 minutes. :) Combine
this with Hydra (CI) server: You can even write tests and have _your laptop
configuration continuously integrated_ all the time?!?!?!?

That said NixOS isn't always the easiest thing to use, but it _does_ come with
some big advantages!

[1] I'm aware of DKMS (and have even worked on modules using it where it was
fine) but I've still had it break once or twice. Maybe that was VMWare's fault
:)

[2] If you share code between all your configuration.nix files, this becomes a
lot easier.

------
zeisss
Is there good explanation anywhere of the syntax of those files? I tried
getting into nix multiple times and was always put off by the lack of
understandability of this.

~~~
zimbatm
Nix is a bit like JSON + functions.

An object (called attrset in nix):

    
    
        { a = 3; b = 4; }
    

A list:

    
    
        [ 1 2 3 4 ];
    

A string:

    
    
         "hello"
    

A multi-line string:

    
    
        ''
          multine
          string
        ''
    

This defines a lambda with two arguments:

    
    
        a: b: a + b;
    

It's possible to pattern-match arguments. This accepts an attrset with keys a
and b, and maps them to local variables:

    
    
        { a, b }: a + b;
    

There are some variations around that like `{ a ? 3 }:` `{ a }@attrs:` and `{
a, ... }`

One special type is the path, which has it's own literal that starts with `./`

To load a file, the `import` keyword is being used. In most cases the whole
file is a big lambda function, which gets returned by the import.

Once you know that you can start reading nixpkgs and learn about the building
blocks to constructs packages.

~~~
FrozenCow
Also, for convenience, nested objects/attrsets can be shortened. For instance:

    
    
        {
            a = {
                b = {
                    c = 3;
                };
            };
        }
    

can be shortened to:

    
    
        {
            a.b.c = 3;
        }
    

This is used quite often in NixOS system configuration files. There you'll
find lines like:

    
    
        services.openssh.enable = true;
    

For a list of all system configuration options that NixOS supports by default
see (NixOS
options)[[https://nixos.org/nixos/options.html](https://nixos.org/nixos/options.html)].

------
thomastjeffery
NixOS makes a great OS for personal/workstation use, too.

With NixOS, my system is _always_ 100% clean, and _never_ broken.

I can install packages in my home directory without root.

Best of all, whenever I want to write any code, I can use nix-shell to make
the toolchain and dependencies _temporarily_ available to a self-contained
shell. No more cluttering my system with libfoobar-dev for a quick build, and
forgetting why.

------
pierrebeaucamp
I would love if someone could compare NixOps with BOSH.

~~~
jacques_chester
I haven't done much Nix, but I do peek over the fence now and then because I
(and others) wonder about applying it to buildpacks and/or OCI layer
construction. I do a lot more BOSH, since I work at Pivotal. So these remarks
are probably wrong.

However, from my ignorant viewpoint:

The deep agreement is that machine-level statefulness is evil. Trying to
repair it in-situ is pointless when you can simply reset to known-good states.

BOSH does this at the whole-machine level. You get a stemcell (fixed target),
you get releases (fixed configuration of software) and you get deployments
(fixed mappings from releases into running systems).

When you make a change, you redeploy. BOSH recompiles any changed release
packages and then rolls them out across the fleet, starting with canaries.

BOSH lets you define the number of VMs, networks and whatnot; I'm not sure if
NixOps does.

Treating components as replaceable is the main point of departure from
previous generations of configuration management, which largely evolved out of
a world in which the mission was "make this single machine look right,
starting from an unknown state".

NixOps appears to do something similar. I'm unclear as to whether it operates
at the whole-machine level (à la BOSH) or whether it can perform
transformations within an existing machine. The latter would, obviously, be a
_lot_ faster than replacing VMs wholesale.

That said, whole-machine replacement does have one accidental benefit: it's
ruthless. You very quickly encounter the one-in-a-million edge cases for
running services because BOSH will quite unmercifully steamroll VMs during a
deploy. Any stateful distributed services that aren't ready to have running
machines yanked from under them at short notice will quickly reveal
themselves.

You'd be surprised how many systems that advertise themselves as tolerant to
random machine loss ... er ... aren't. Not sure much how I can go into this,
but some distributed systems do much more poorly than the stuff on the tin
suggests, even after substantial engineering on our part to try and prop them
up.

Disclosure: I work for Pivotal, so I use BOSH sometimes in my dayjob. But I'm
by no means a capable operator or BOSH expert.

~~~
wmertens
* NixOps lets you define the complete layout of your deployment (in Nix, obviously): how many VMs, what they should be running, where they should be running.

* NixOps works on whole machine + transformation of machine level: NixOps builds all system configurations of the whole network, then makes sure the required target hosts are available, then copies the builds to the hosts, and then atomically switches all hosts to the new configuration at once. NixOS is stateless, and a new config only means that the new service configurations are applied.

Rollback is a matter of switching all VMs back to their previous
configurations, takes less than a minute in all.

~~~
jacques_chester
Thankyou for the additional detail. I'd say that the two arrived at similar
outcomes, which isn't surprising given NixOps's roots.

------
djsumdog
Seems neat, better than Terraform, although limited to just nix and a few
providers. Hopefully that will grow. I've been working on a provisioning tool
myself. Provisioning is hard, and supporting multiple APIs can be really
difficult.

~~~
mbrock
I don't really see Nix as a limitation. It has a good claim to being a
universal system configuration tool. Or how do you mean it's limited?

~~~
philipov
Can I use it to deploy either client or server applications on windows? All
configuration and deployment tools I've looked at seem like they assume the
only deployment target is linux servers running web applications. I don't care
about the web. I need a deployment tool that works for local research on my
clients' analysts' desktops, not just devops on remote somewhere.

~~~
detaro
Have you looked into the Windows features of Ansible or SaltStack? Would be
interested to hear about experiences with them.

~~~
reificator
I would like to adopt something for configuration of my personal machines.
Desktop and tablet running Windows 10, laptop running macOS.

If possible I'd prefer to use one tool for both OSes.

------
j_s
Can anyone comment on whether or not Nix is 100% reproducible? To the best of
my limited understanding, it is not.

~~~
catern
Nix doesn't guarantee bit-for-bit reproducibility: That is, two different
builds of a Nix package may not produce bit-identical output. (They probably
will, because of all the sandboxing, but nothing prevents an arbitrary package
from generating random numbers at build time.)

Certainly Nix makes it a lot easier to get reproducibility in practice,
though.

