I've tried again and again to like Nix, but at this point I have to throw in the towel.
I have 2 systems running Nix, and I'm afraid to touch them. I've already broken both of them enough that I had to reinstall from scratch in the past (yes yes - it's supposed to be impossible I know), and now I've forgotten most of it. In theory, Nix is idempotent and deterministic, but the problem is "deterministic in what way?" Unless you intimately understand what every dependent part is doing, you're going to get strange results and absolutely bizarre and unhelpful errors (or far more likely: nothing at all, with no feedback). Nix feels more like alchemy than science. Like trying to get random Lisp packages to play nice together.
Documentation is just plain AWFUL (as in: complete and technically accurate, but maddeningly obtuse), and tutorials only get you part of the way. The moment you step off the 80% path, you're in for a world of hurt, because the underlying components are just not built to support anything else. Sure, you can always "build your own", but this requires years of experiential knowledge and layers upon layers of frustration that I just don't want to deal with anymore (which is also why I left Gentoo all those years ago). And woe unto you if you want to use a more modern version than the distribution supports!
The strength of Docker is the chaos itself. You can easily build pretty much anything, without needing much more than a cursory understanding of the shell and your distro's package manager. Or you can mix and match whatever the hell you want! When things break, it's MUCH easier to diagnose and fix the problems because all of the tooling has been around for decades, which makes it mature enough to handle edge cases (and breakage is almost ALWAYS about edge cases).
Nix is more like Emacs: It can do absolutely anything if you have the patience for it and the deep, arcane knowledge to keep it from exploding in a brilliant flash of octarine. You either go full-in and drink the kool aid, or you keep it at arm's length - smiling and nodding as you back slowly towards the door whenever an enthusiast speaks.
I've gone down the same path. I love deterministic builds, and I think Docker's biggest fault is that to the average developer a Dockerfile _looks_ deterministic - and it even is for a while (build a container twice in a row on the same machine => same output), but then packages get updated in the package manager, base images get updated w/ the same tag, and when you rebuild a month later you get something completely different. Do that times 40 (the number of containers my team manages) and now fixing containers is a significant part of your job.
So in theory Nix would be perfect. But it's not, because it's so different. Get a tool from a vendor => won't work on Nix. Get an error => impossible to quickly find a solution on the web.
Anyway, out of that frustration I've funded https://www.stablebuild.com. Deterministic builds w/ Docker, but with containers built on Ubuntu, Debian or Alpine. Currently consists of an immutable Docker Hub pull-through cache, full daily copies of the Ubuntu/Debian/Alpine package registries, full daily copies of most popular PPAs, daily copies of the PyPI index (we do a lot of ML), and arbitrary immutable file/URL cache.
So far it's been the best of both worlds in my day job: easy to write, easy to debug, wide software compatibility, and we have seen 0 issues due to non-determinism in containers that we moved over to StableBuild in my day job.
I've work many years on bare metal. We did (by requirement) acceptance tests, so we did need deterministic builds, before such thing had even a name, or at least before it was mentioned as much as nowadays.
Redhat has a lot of tooling around versioning of mirrors, channels, releases, updates, etc. But I'm so old that even foreman and spacewalk didn't exist, redhat satellite was out of the budget, and the project was migrating from the first versions of CentOS to Debian.
What I did was simply use DNS + Vhosts (dev, stage, prod + versions) for our own package mirrors, and bash+rsync (and of course, raid+backups), with both, CentOS and Debian (and our project packages).
So we had repos like prod/v1.1.0, stage/v1.1.0, dev/v1.1.0, dev/v2.0.0, dev/2.0.1, etc Allowing us to rebuild things without praying, backport bug fixings with confidence, etc
Feels old and simple, however I think it was the same problem/issue that people gets now (re)building containers.
If you need to be able to produce the same output from the same input, you need the same input.
But also Nix solves more problems than Docker. For example if you need to use different versions of software for different projects. Nix lets you pick and choose the software that is visible in your current environment without having to build a new Docker image for every combination, which leads to a combinatorial explosion of images and is not practical.
But I also agree with all the flaws of Nix people are pointing out here.
I don't have any experience with Nix but regarding stable builds of Docker: we provide Java application, have all dependencies as fixed versions so when doing a release, if someone is not doing anything fishy (re-releasing particular version, which is bad-bad-bad) you will get exactly same binaries on top of the same image (again, considering you are not using `:latest` or somesuch)...
Until someone overwrites or deletes the Docker base image (regularly happens), or when you depend on some packages installed through apt - as you'll get the latest version (impossible to pin those).
I am convinced that any sort of free public service is fundamentally incomapatible with long term reproducible builds. It is simply unfair to expect free service to maintain archives forever and never clean them up, rename itself, or go out of business.
If you want reproducibility, the first step is to copy everything to a storage you control. Luckily, this is pretty cheap nowdays
> Until someone overwrites or deletes the Docker base image (regularly happens)
Any source of that claim?
> or when you depend on some packages installed through apt - as you'll get the latest version (impossible to pin those).
Well... please re-read my previous comment - we do Java thing so we use any JDK base image and then we slap our distribution on top of it (which are mostly fixed-version jars).
Of course if you are after perfection and require additional packages then you can install it via dpgk or somesuch but... do you really need that? What about security implications?
You gave example of nvidia and not ubuntu itself. What's more, you are referring to devel(opment) version, i.e. "1.0-devel-ubuntu20.04" which seems like a nightly so it's expected to be overriden (akin to "-SNAPSHOT" for java/maven)?
Besides, if you really need utmost stability you can use image digest instead of tag and you will always get exactly the same image...
Do you have an example that isn't Nvidia? They're infamous for terrible Linux support, so an egregious disregard for tag etiquette is entirely unsurprising.
> Anyway, out of that frustration I've funded https://www.stablebuild.com. Deterministic builds w/ Docker, but with containers built on Ubuntu, Debian or Alpine.
Another option for reproducible container images is https://github.com/reproducible-containers although you may need to cache package downloads yourself, depending on the distro you choose.
For Debian, Ubuntu, and Arch Linux there are official snapshots available so you don't need to cache package downloads yourself. For example, https://snapshot.debian.org/.
Yes, fantastic work. Downside is that snapshot.debian.org is extremely slow, times out / errors out regularly - very annoying. See also e.g. https://github.com/spesmilo/electrum/issues/8496 for complaints (but it's pretty apparent once you integrate this in your builds).
Yeah, but it's impossible to properly pin w/o running your own mirrors. Anything you install via apt is unpinnable, as old versions get removed when a new version is released; pinning multi-arch Docker base images is impossible because you can only pin on a tag which is not immutable (pinning on hashes is architecture dependent); Docker base images might get deleted (e.g. nvidia-cuda base images); pinning Python dependencies, even with a tool like Poetry is impossible, because people delete packages / versions from PyPI (e.g. jaxlib 0.4.1 this week); GitHub repos get deleted; the list goes on. So you need to mirror every dependency.
> Anything you install via apt is unpinnable, as old versions get removed when a new version is released
Huh, I have never had this issue with apt (Debian/Ubuntu) but frequently with apk/Alpine: The package's latest version this week gets deleted next week.
>Documentation is just plain AWFUL (as in: complete and technically accurate, but maddeningly obtuse)
Documentation is often just plain erroneous, especially for the new CLI and flakes, not even edge cases. I remember spending some time trying to understand why nix develop doesn't work like described and how to make it work like it should. I feel like nobody ever actually used it for its intended purpose. Turns out that by default it doesn't just drop you into the build-time environment like the docs claim (hermetically sealed with stdenv scripts available), it's not sealed by default and the commandline options have confusing naming, you need to fish out the knowledge from the sources to make it work. Plenty of little things like this.
>In theory, Nix is idempotent and deterministic
I surely wish they talked more about edge cases that break reproducibility. Things like floating point code being sensitive to the order of operations with state potentially leaking from OS preemption, and all that. Which might be obvious, but not saying obvious things explicitly is how you get people shoot themselves in the foot.
> Things like floating point code being sensitive to the order of operations with state potentially leaking from OS preemption, and all that.
That’s profoundly cursed and also something that doesn’t happen, to my knowledge. Unless the kernel programmer screwed up, an x86-64 FPU is perfectly virtualizable (and I expect an AArch64 FPU too, I just haven’t tried). So it doesn’t matter where preemtion happens.
(What did happen with x87 is that it likes to compute things in more precision than you requested, depending on how it’s configured—normally determined by the OS ABI. Yet variable spills usually happened in the declared precision, so you got different results depending on the particulars of the compiler’s register allocator. But that’s still a far cry from depending on preemption of all things, and anyway don’t use x87.
Floating-point computation does depend on associativity, in that nearestfp(nearestfp(a+b)+c) is not the same as nearestfp(a+nearestfp(b+c)), but the sane default state is that the compiler will reproduce the source code as written, without reassociating things behind your back.)
That's doesn't happen in a single thread, but e.g. asynchronous multithreaded code can spit values in arbitrary order, and depending on what you do you can end up with a different result (floating point is just an example). Generally, you can't guarantee 100% reproducibility for uncooperative code because there's too much hardware state that can't be isolated even in a VM. Sure, 99% software doesn't depend on it or do cursed stuff like microarchitecture probing during building, and you won't care until you try to package some automated tests for a game physics engine or something like that. What can happen, inevitably happens.
We don't need to be looking for such contrived examples actually, nixpkgs track the packages that fail to reproduce for much more trivial reasons. There aren't many of them, but they exist:
> We don't need to be looking for such contrived examples actually, nixpkgs track the packages that fail to reproduce for much more trivial reasons. There aren't many of them, but they exist
Less than a couple of thousand packages are reproduced. Nobody has even attempted to rebuild the entirety of the nixpkgs repository and I'd make a decent wager on it being close to impossible.
It’s really not that bad. However, with a standard NixOS setup, you still have a tremendous amount of non-reproducible state, both inside user accounts and in the system. I’m running a “Erase your darlings” setup, it mostly gets rid of non-reproducible state outside my user account. It’s a bit of a pain, but then what isn’t on NixOS.
That setup uses Home Manager, so maybe it's not for you, but worth mentioning if we're talking about making all state declarative and reproducible. You have to use the Impermanence module and set up some soft links to permanent home folders on different drive or partition. But for making all state on the system reproducible and declarative, this is the best way afaik.
True, I think it's more a more elegant setup than the ZFS version. Why actively rollback to a snapshot when ephemeral memory will do that automatically on reboot.
That said I'll just mention that ZFS support on NixOS is like nothing else I've seen in Linux. ZFS is like a first-class citizen on NixOS, painless to configure and usually just works like any other filesystem.
I use both Docker and NixOs at work. I've never had any of the problems you seemed to have above. Docker is fine, performance wise it's not great on Macs. I love nix because it's trivial to get something to install and behave the same across different machines.
Nix Doc are horrible but I've found that ChatGPT4 is awesome at troubleshooting Nix issues.
I feel like 90% of the time I run into Nix issues, it's because I decided to do something "Not the Nix way."
Give a try to Fedora Atomic (immutable). At this point I have pretty much played around and used every distro package maneger there is and I have broken all of them in one way or another even without doing something exotic (pacman I am looking at you). My Fedora Kinoite is still going strong even with adding/removing different layers, daily updates, and a rebase from Silverblue. Imho rpm-ostree will obsolete Nix.
You have to restart to boot into a new image. You use containers for stuff you don't need into your base distro, like cli tools, and flatpak for any desktop applications.
> Documentation is just plain AWFUL (as in: complete and technically accurate, but maddeningly obtuse)
That has been the case for as long as I can remember. I gave up on Nix about 5 years ago because of it, and apparently not much has changed on that front since then..
I never tried going all in on Nix, but I don't think it's an all or nothing proposition. In my case, I use Ubuntu for my personal notebook and I wanted to prototype something with Elixir. The distro package is versions behind latest so I can't use Phoenix 1.7 with it. The solution was simple: there's a Nix package for the latest version, so I simply used nix-shell. Bonus points for having VSCode so I didn't have to install it on my personal machine. So for the price of running <nix-shell -p vscode erlang elixir> I got all I needed with very minimal fuss.
I've been a nixos user for years and I generally had the opposite problem: the latest of the package you want is not available but hey here's a version from months ago - or just build it yourself (which is not hard, oftentimes updates work fine with no build change, you just point at a different version).
Also rebuilding everything at every update take forever (I had a few nix-shells with ai dependencies that would take hours to upgrade).
I love the concept of nix but I'm back to Arch, binary bleeding edge packages and AUR for less supported stuff.
I recently faced a similar hurdle with Nix, particularly when trying to run a .NET 8 AOT application. What initially seemed like it would be a simple setup spiraled into a plethora of issues, ultimately forcing me to back down. I found myself having to abandon the AOT method in favor of a more straightforward solution. To give credit where it's due, .NET AOT is relatively new and, as far as I know, still has several kinks that need ironing out. Nonetheless, I agree that, at least based on my experience, you need a solid understanding of the ins and outs before you can be reasonably productive using Nix.
.NET AOT really is not designed for deployment, in my experience - for example, the compilation is very hard to do in Nix-land, because a critical part of the compilation is to download a compiler from NuGet at build-time. It's archetypical of the thousand ways that .NET drives me nuts in general.
It's intended for 'cloud-native' deployments, as I understand it, so I concur that it's quite disappointing. The concept of downloading compilers via NuGet doesn't sit well with me either. However, I've observed performance enhancements in applications compiled AOT, and I remain optimistic that future versions of .NET will bring further improvements.
I’m not sure exactly why this is being downvoted. It seems pretty fair to want your container builds to not fail because of the “chaos” with docker images and how they change quite a lot. This isn’t about the freedom to build how you want, it’s about securing your build pipelines so that they don’t break at 4am because docker only builds 99% of the time.
I’ll use docker, I like docker, but I can see the point of how it’s not necessarily advantageous if stability is your main goal.
It's more complicated than that. Reproducible builds help build confidence that your build process isn't compromised.
Sure, your compiler, your hardware, or your distro might be compromised, but if you follow the chain all the way through you does indeed validate version X does result in SHA y, there's now less things were blindly trusting.
It also helps with things like rolling back to earlier versions when you don't still have the binary kicking around without having to revalidate the binary.
If you're not getting the same SHA on different hardware, weeks apart, even if it's good enough for you, it's not reproducible
You complain about the documentation, and the first thing I wonder is if you’ve tried using one of the prominent chatbots like chatgpt or claude to help fill in the gaps of said documentation? Maybe an obvious thing to do around here, but I’ve found they help fill in documentation gaps really well. At the same time Nix is so niche there might not have been enough information out there to feed into even chatgpt’s model…
>I've already broken both of them enough that I had to reinstall from scratch in the past (yes yes - it's supposed to be impossible I know)
Could you mention a bit about how they broke? I'm curious to see how that state looks, as from my perspective switching to a previous configuration seems to cover everything.
Nix and NixOS are in something like the state git was in before GitHub: the fundamental idea is based on more serious computer science than the status quo (SVN, Docker), the plumbing still has some issues but isn’t worse, and the porcelain and docs are just not there for mainstream adoption.
I think that might have changed with the release of flox: https://flox.dev, it’s basically seamless (and that’s not surprising coming from DE Shaw).
Nix doesn’t really make sense without flakes and nix-command, those things are documented as experimental and defaulted off. The documentation story is getting better, but it’s not there. nixlang is pretty cool once you learn it well, but it’s never going to be an acceptable barrier to entry at the mainstream. It’s not really the package manager it’s advertised as, nix-env -iA foo is basically never what you want. It’s completely unsurprising that it’s still a secret weapon of companies with an appetite for bleeding-edge shit that requires in-house expertise.
flox addresses all of this for the “try it out and immediately have a better time” barrier.
Nix/NixOS or something like it is going to send Docker to the same dustbin Subversion is in now, but that’s not going to happen until it has the GitHub moment, and then it’ll happen all at once.
Most of the complaints with Nix in this thread are technically false, but eminently understandable and more importantly (repeat after me Nix folks): it’s never the users fault.
I’m aware that I’m part of a shrinking cohort who ever knew a world without git/GitHub, so I know this probably sounds crazy to a large part of the readership, but listen to Linus explaining to a room full of people who passed the early Google hiring bar why they should care about a tool they feel is too complicated for them:
Ron from flox.dev here, the note brought a lot of smiles across the team. We've been working on this for a while now and would love to hear if there is anything we can prioritize or do to keep making it better.
I’m glad to hear it! I’ve been grappling with how to package something I’m calling “HYPER // MODERN” (which I can talk about if you’re curious) and we’re pretty locked-in on flox at this point, it had been a combination of flakes and Homebrew and flox is just a better time.
If you drop me an email at b7r6@b7r6.net (I also just joined your slack) I’d love to give my feedback on this or that nitpick.
But overall, well done friends, very very nice stuff.
I believe for a developer tool to success, for the most common thing to do there has to be at least three ways an engineer may misuse your tool and still get it "done" (by leaving non-obvious tech debts behind).
This is true for git, but not so true yet for Nix, so I'm not sure a GitHub-like moment helps.
In a full-metal-jacket NixOS setting it’s bloody hard to bash (no pun intended) your way through to the next screen by leaving behind tech debt (Python comes to mind, I made the mistake of trying to use Nix to manage Python once, never again).
But anywhere else you just brew/apt/rpm install whatever and nix develop —impure, which is easier than most intermediate git stuff and plenty of beginner stuff. git and Nix are almost the same data structure if you start poking around in .git or /nix/store, I might not understand what you mean without examples.
But all my guesses about what you might mean are addressed well by flox.
This blog post is missing the reasoning on why shared docker layers are useful. It is because of caching. The more images are sharing the same layers the better, as it allows you to cache more stuff. Better caching means faster startup of containers.
Why is docker bad at this? In order to enjoy the caching benefit, each time you build a docker image you want it to output as much existing layers as possible. So running apt-get install python3 today should result in the exact same layer as yesterday, if there are no new updates. But this requires the all the files to be exactly the same, including the metadata like creation time. As docker layers are cached by hashing the files.
Now, Nix already does storing dependencies by hash. So the layers will always be the same with the same version and same configuration.
The Dockerfile format imposes a hierarchical relationship between layers. This quickly becomes very annoying, since dependencies usually form dependency graphs, not dependency trees.
Alternative tools, like nix (probably bazel too), are not bound in the same way. They can achieve fine grained caching by mapping their dependency graph to docker layers, which is something that can not be expressed with a Dockerfile.
> The Dockerfile format imposes a hierarchical relationship between layers. This quickly becomes very annoying, since dependencies usually form dependency graphs, not dependency trees.
Isn't a Dockerfile just a sequence of dependencies, rather than a tree?
You're right, though it becomes hierarchical once you have multiple Dockerfiles inheriting from some base image (which I did not articulate in my original comment).
The final result need not be.
You can build a bunch of things then merge the results in a final stage without any hierarchy (this is "COPY --link" in a Dockerfile).
With nix, re-usability is very high. It's a function that is very baked in at very low levels of it's design. This comes with up front complexity but getting to these reusable layers is basically forced.
Docker is very simple and often touts reusable layers, but in practice is not. Unless you tackle that complexity.
Making reproducible and reusable content takes effort. Other tools are not designed for that. As a result the getting to the same state requires a similar amount of complexity. Worse, with docker you can never be sure that you actually succeeded in your goal of reproducibility.
An analogy could be rust. Rust has up front complexity, but tackling that complexity gives confidence that memory safety and concurrency primitives are done correctly. It's not that C _can't_ achieve the same runtime safety, it's just requires a lot more skill to do correctly; and even then memory exploits are reported on a near daily basses for very popular and widely used libraries.
Complex problems are complex. And sooner or later you'll need to face that complexity.
This is not how docker works.
Docker, exactly like nix, is based on a graph of content addressable dependencies.
What you are describing is chaining a bunch of commands together.
Yes, this forms a dependency chain stored in separate layers and is part of the cache chain.
Nix suffers the exact same problems with reproducibility.
The thing it provides is the toolchain of dependencies that are reproducible.
Docker does not provide your dependencies.
If the inputs change then so does the output.
If the output itself is not reproducible (like, say an artifact with a build-time embedded in it) then you have something that is inherently not reproducible and two people trying to build the same exact nix package will have different results.
EDIT: Fixed a sentence I apparently got distracted while writing and didn't complete (about layer caching).
Nix is not content addressable though, the hashes is based off on the derivation files which are equal to the lock files you would find in other package managers.
> The thing it provides is the toolchain of dependencies that are reproducible. [...] If the inputs change then so does the output. If the output itself is not reproducible (like, say an artifact with a build-time embedded in it) then you have something that is inherently not reproducible and two people trying to build the same exact nix package will have different results.
There are no guarantees they are reproducible. The only guarantees Nix gives you is that the build environment is the same which allows you to make some claims about the system behaving the same way. But they are certainly no guarantees about artifacts being bit-by-bit identical.
But doing this is going to give you a slight headace as most of the package repository in Nix is not checked for reproducible builds and there are no way to guarantee the hashes are actually static.
Right, all builds are dependent on their inputs.
Your inputs determine your outputs.
If your input(s) change, then so does your output.
We are saying the same thing here, I'm just trying to point out this is exactly how docker build works, but rather it is more about what you are willing to put into your docker build.
I think we are talking past each other. I'm just trying to clear up a misconception on how nix works, not anything about the docker portion of what you have written.
It would seem you don't understand how either work. They are basically opposites in how they actually work.
Docker layers are completely independent from each other. A docker layer is a sha256 sum of that layer. Separately there is an image manifest, which is also fetched with a sha256 of that manifest, states the order of the layers and at runtime those layers are stacked up on each other.
With docker, there is no explicit dependency chain. A layer is just a tarball or some JSON. Some tooling can take advantage of this fact. How nix builds docker images takes advantage of this.
Nix on the other hand, the output hash is not tied to the output hash, because the output hash is irrelevant to how it was produced. You also cannot know in advance the output hash of something. IE. If I do say "echo foo > bar.txt", I cannot know the sha256 sum of bar.txt until the code runs. But before the code runs I can know the hash all the inputs that will create bar.txt.
This fundamental difference means two builds, executing the same code can share the outputs. Provided that the build environment is trust worthy.
You are describing the makeup of an OCI image, which is the _output_ of a typical docker build (and also the output of the nix image builder).
While docker build can/does output OCI images, that is only an output.
How that output comes to be is not the output itself, same as the nix side the article is talking about.
> How that output comes to be is not the output itself, same as the nix side the article is talking about.
I see the confusion now. The nix image builders OCI layer's contents are _only_ nix store paths which _do_ include the input.
Nix store paths are are guaranteed to never overlap each other, and such the order of layering them in the docker manifest does not matter. But the docker layers are just tar balls of nix store paths. Each layer in the image has no dependence on previous or future layers at all. It is just one or more nix store paths.
I'm not talking about OCI images (again, that's the _output_).
I'm talking about how they are built.
OCI images are OCI images, they get extracted the same way no matter if there's conflicting paths or not.
What I'm saying here through multiple different threads is, buildkit and nix build things the same way.
`docker build` is not just a Dockerfile builder, its actually a grpc service (with services running on both the docker CLI and in the daemon).
This service is actually very generic.
It includes builtin support for Dockerfiles, which just converts the Dockerfile format into what buildkit calls "LLB", which is analogous to LLVM IR.
What I'm also saying is, people are comparing "docker build" with a Dockerfile that's using a package manager that's not even provided by docker to nix.
This is not an apples to apples comparison, and in fact you can implement nix packaging using buildkit (https://github.com/reproducible-containers/buildkit-nix).
I'm also saying that `Dockerfile` does actually support merging dependencies without being order dependent (this is `COPY --link`).
But also, you can drive buildkit operations without going through Dockerfile. You can also plug in your own format with the `syntax=<some/image>` at the top of your file.
This isn't "convert to dockerfile", its "convert to LLB", which is all the Dockerfile frontend does.
Finally, I'm saying nix isn't in and of itself some magic tool to have a reproducible build.
You still have to account for all the same things.
What it does do, at a package management level, is make it easier to not have dependencies that change automatically over time (which has its own plusses and minuses).
I am thinking more about pipelines that run daily. Also, this ‘docker cache’ effectively means not running the step. So you might miss important security updates. Via Nix you can ensure that your dependencies are updated. And no updates means same hash.
When said caching, I meant on the nodes that run the containers. With Nix you can also update only one layer, while keeping the other layers the same.
Won't get invalidated even if what "apt-get install python3" does changes - the cache is only based on the syntax of the RUN string plus the previous layer hash, IIRC. (COPY actually invalidates if the file being copied changes, so maybe there's a way to fetch a hash of the repo and stash it where copy will notice, or something, but then it seems you need external tooling to do that bit?)
I didn't claim there is a problem. The original comment made it sound as if docker will expire a cached layer because the (potential) result of apt-get is different, which isn't the case.
The big problem is how docker was designed. It is essentially a jail that is supposed to contain a Linux binary.
Things are straight forward on Linux. You build your binary, place in a docker container and you are done. The nix code will also be straight forward. If you can build your code, then creating a container is just one more operation away.
Unfortunately docker requires Linux binary and you are on Mac. So the docker desktop actually runs a Linux VM and performs all operations on it, abstracting this away from you.
Nix doesn't do that and you have two options:
1. Do cross compilation, the problem is that for this to work you need to be able to cross compile down to glibc, the problem is that while this will work for most community used dependencies you might get some package where the author didn't put effort making sure it cross compile. To make things worse the Hydra that populates standard caches that nix uses, doesn't do cross compile builds, so you will run into lengthy processes that might potentially end with a failure.
2. You can have a Linux builder, that you add to your Mac and configure to send build jobs for x86_64-linux to that builder. Now you could have a physical box, create a VM, or even have a NixOS docker container (after all docker ion Mac runs inside of the VM).
The #1 seems like the proper way, while #2 is more of a practical way.
I think you are running into issues, because you're likely trying #1, and that requires a lot of experience not only with Nix, but also with cross compiling. I wish Nix's Hydra would also build Darwin to Linux cross compilation as that would not only provide caches, but also help making sure the cross compilation doesn't break, but that would also increase costs for them.
Hydra not populating with cross compile builds is the bane of my existence.
I'm using `clang` from `pkgs.pkgsCross.musl64.llvmPackages_latest.stdenv` to cross-compile Rust binaries from ARM macos to `x86_64-unknown-linux-musl`. It _works_, but every time I update my `flake.nix` it rebuilds *the entire LLVM toolchain*. On an M2 air, that takes something like 4 hours. It's incredibly frustrating and makes me wary of updating my dependencies or my flake file.
The alternative is to switch to dockerized builds but:
1) That adds a fairly heavyweight requirement to the build process
2) All the headache of writing dockerfiles with careful cache layering
Not sure if this applies to your situation, but I believe you can avoid a full rebuild by modularizing the flake.nix derivations into stages (calling a separate *.nix for each stage in my case). That is how it appears to be working for me on a project (I am building a cc toolchain without pkgscross).
I pass the output of each stage of a toolchain as a dependency to the next stage. By chaining the stages, changes made to a single stage only require a rebuild of each succeeding stage. The final stage is the default of the flake, so you can easy get the complete package.
In addition, I can debug along the toolchain by entering a single stage env with nix develop <stage>
Not sure if this is the most optimal way, but it appears to work in modularizing the rebuild.(using 23.11)
I use Orbstack and it works flawlessly to do this. Really good tool. I use Docker to cross-compile for {aarch64,amd64} x {linux,darwin} since not all the cross-compiling is super robust across our stacks (I'm using a specific glibc for one Linux part, etc.). Just a bunch of docker on my Darwin aarch64 and it compiles everything. Good experience.
I installed Orbstack and found that I didn't really need it, so I removed the directory in /Applications. Wow, for weeks and weeks I found remnants of it in a lot of places. Very disappointing that it left so much cruft around. They should have an uninstaller. It left a really bad taste and I'm unlikely to try it again.
Before someone asks. I've been using macOS for a long time. I've never seen remnants like this from a program. Sure, there are often directories left in ~/Library/Application Support/, but this was more than that. Unfortunately, I didn't write down the details, but I ran across the bits in at least 3-4 places.
Dev here — I've been meaning to update the Homebrew cask to be more complete on zap, but there's a good reason that all of these are needed:
- ~/.orbstack
- Docker context that points to OrbStack (for CLI)
- "source ~/.orbstack/shell/init.zsh" in .zprofile/bash_profile (to add CLI tools to PATH)
- ~/.ssh/config (for convenient SSH to OrbStack's Linux machines)
- Symlinks to CLI tools in ~/.local/bin, ~/bin, or /usr/local/bin depending on what's available (to add CLI tools to existing shells on first install — only one of these is used, not all)
- ~/OrbStack (empty dir for mounting shared files)
- /Library/PrivilegedHelperTools (to create symlinks for compatibility)
Not sure what the best solution is for people who don't use Homebrew to uninstall it. I've never liked separate uninstaller apps, and it's not possible to detect removal from /Applications when the app isn't running.
IMO documenting this (and uninstall section in GUI with link) would be enough for me. Used that and never felt neglected by devs.
And cough since we’re at with - did you consider Nixpkgs distribution?
I’m slowly moving deeper and deeper into ecosystem and use Home Manager for utilities that I use often (and use nix shell/nix run for one offs). Some packages are strictly GUI and while they aren’t handled flawlessly (self-updaters) it’s nice to have them on a single list.
Yet based on your list it’s a definitely a nixventure…
And yet, you are the only(?) one with that knowledge, so the alternative seems to be replying to HN threads with a curated list of things that a user must now open iTerm2 and handle by themselves. Something, unless I'm mistaken, that computers are really good at doing (Gatekeeper and privilege elevation nonsense aside)
Even just linking to the zap portion of your brew cask could go a long way since it would be the most succinct manifest if I correctly understand what it does
I agree this should be documented, but I still appreciate uninstallers.
Also, I'm a little confused about your statement:
> Not sure what the best solution is for people who don't use Homebrew to uninstall it.
You said at the start you've "been meaning to update the Homebrew cask to be more complete on zap" ... does that mean Homebrew uninstall will not do a complete job?
But unfortunately cross compiling quickly broke when I started doing mild customization (and one reasons I’m doing this is a complex setup that’s very sensitive to version changes).
In the end solution was to “simply” get darwin.linux-builder up but that pulled a lot of weight behind it.
It works, but it’s not the first time I spent my time on nix-ventures.
On the flip side, once you have fixed the problem it has a very strong tendency to stay fixed. More importantly, the fix does not typically require me to remember that fix months later.
If something does break, rollbacks are free and an integral part of Nix.
macOS is a definitely rougher. I use colima there, and it does alright. There are one or two bugs with it, but I think those are primarily around volumes. But it does alright with building Docker images.
The rougher part is the speed of it; it's a one-two punch between the hardware & the fact that Docker has to emulate a Linux VM.
The way I've set this up for our macos devs at work was a script that runs nix builds inside docker-for-desktop using the official nixos upstream docker image (and some tricks to get ssh forwarding, filesystem mounts, ...) working. Works quite alright. Benefit is you don't need some weird Linux remote builder vm with ssh running.
My experience with building Docker images for Java applications using Nix wasn't very pleasant though. After the deprecation of gradle2nix, there doesn't seem to be a clear alternative method for building Docker images for Gradle-based Java applications. I challenged a friend to create the smallest possible Docker image for a simple Spring Boot application some time ago. While I was using Nix, the resulting image was twice the size of the image built without Nix. You can check out the code for yourself here: https://github.com/jossephus/Docker_challenge/blob/main/flak... .
That's because you're including two JDKs, zulu and the one that gradle includes via its jdk argument. Look for gradleGen in nixpkgs to see what I mean.
And sorry for gradle2nix, I'm working on an improvement that's less of a hack.
Thanks tadfisher, I will check it out. This is by no means meant to be a dunk on gradle2nix. Love your work on android-nixpkgs and I will be looking for the alternative. Thanks.
I haven't used java in over a decade so won't be able to help much with that, but for example I was able to get my application to fit in just 70MB container including python and all dependencies + busybox and tini
- compile python and disable all packages that I didn't use in my application
- trim boto3/botocore, to remove all stuff I did not use, that sucker on it's own is over 100MB
The thing is what you need to understand is that the packages are primarily targeting the NixOS operating system, where in normal situation you have plenty of disk space, and you rather want all features to be available (because why not?). So you end up with bunch of dependencies, that you don't need. Alpine image for example was designed to be for docker, so the goal with all packages is to disable extra bells and whistles.
This is why your result is bigger.
To build a small image you will need to use override and disable all that unnecessary shit. Look at zulu for example:
you add alsa, fontconfig (probably comes with entire X11), freetype, xorg (oh, nvm fontconfig, it's added explicitly), cups, gtk, cairo and ffmpeg)
Notice how your friend carefully extracts and places only needed files in the container, while you just bundle the entire zulu package with all of its dependencies in your project.
Edit: tadfisher seems to be more familiar with it than me, so I would start with that advice and modify code so it only includes a single jdk. Then things that I mentioned could cut the size of jdk further.
Edit2: noticed another comment from tadfisher about openjdk_headless, so things might be even simpler than I thought.
It might be still confusing to you at first, as you're used to list of incremental steps how to get to the final result, while this description instead is declarative (you're describing not the steps to do, but what the final image should be).
It's basically comparing bash script with bunch of "aws" CLI invocations to a terraform or cloudformation file.
It's not actually unreadable - you just have to learn convention on top of the Nix language. For instance, what mkDerivation does. Actually, the Nix language usage here is somewhat minimal. Mostly let bindings (aka lambda calculus).
I wouldn't expect a layman to be able to grok that file. That's fine though - it's not for laymen.
> It's not actually unreadable - you just have to learn convention on top of the Nix language. For instance, what mkDerivation does. Actually, the Nix language usage here is somewhat minimal. Mostly let bindings (aka lambda calculus).
> I wouldn't expect a layman to be able to grok that file. That's fine though - it's not for laymen.
This is the kind of comment that makes me want to stay far, far away from Nix and the Nix "community".
Why? Saying that Nix is complicated and isn't trivial to use or read without learning prerequisite knowledge is bad now?
I actually pointed out that mkDerivation is something helpful to learn - that's one thing I wish someone made me sit and learn when I first got exposed to Nix. It unlocks a lot.
I wouldn't state it's _bad_. It just adds another layer of complexity (by, for sure, also giving something back) and as someone not working in Fortune 500 (but rather in a SME with <20 people), another layer of conplexity & another language is sonetimes just not feasable.
What that file is doing, is building a package, and it essentially is a combination of what Makefile and what RPM spec file does.
I don't know if you're familiar with those tools, but if you aren't it takes some time to know them enough to understand what is happening. So why would be different here?
That's interesting. We have some applications, that produce PDFs, which use fonts, which usually requires a non-headless (headfull?) jdk. At AWS i wonder, what the default alpine jdk contains. And how much space could be saved, if people were more aware, that they can use a headless one.
I decided to participate in your challenge and cleaned up your Nix code a little bit. It seems like the main task of the challenge is building a really small JRE.
I've switched to using a headless OpenJDK build from Nixpkgs as a baseline instead of Zulu, to remove all the unnecessary dependencies on GUI libraries. Then I've used pkgs.jre_minimal to produce a custom minimal JRE with jlink.
The image size now comes out to 161MB, which is slightly larger than the demo_jlink image. This is because it actually includes all the modules required to run the application, resulting in a ~90MB JRE. The jdeps invocation in Dockerfile_jlink fails to detect all the modules, so that JRE is only built with java.base. Building my minimal JRE with only java.base brings the JRE size down to about 50MB, the resulting (broken) container image is 117MB according to Podman.
I've also removed the erroneous copyToRoot from your call to dockerTools.buildImage, which resulted in copying the app into the image a second time while the use of string context in config.Cmd would have already sufficed.
I've also switched to dockerTools.buildLayeredImage, which puts each individual store path into its own image layer, which is great for space scalability due to dependency sharing between multiple container images, but won't have an impact for this single-image experiment.
This is mostly a JRE size optimization challenge. The full list of dependencies and their respective size is as follows:
There's not much else that can be done here. glibc is the next largest dependency at ~30MB. This large size seems to be because Nixpkgs configures glibc to be built with support for many locales and character encodings. I don't know if it would be possible or practical to split these files out into separate derivations or outputs and make them optional that way. If you're using multiple images built by dockerTools.buildLayeredImage, glibc (and everything else) will be shared across all of them anyway (given you're using roughly the same Nixpkgs commit).
> While I was using Nix, the resulting image was twice the size of the image built without Nix.
I would be very interested to know where the difference is; is nix including things it doesn't need to? Is the non-nix build not including things it should?
This is great if you've already adopted Nix, and I'd love for nothing than more declarative package management solutions like Nix or Guix to take off.
If you're already using Docker but want to gradually adopt Nix, there is an alternative approach outlined by this talk: https://youtu.be/l17oRkhgqHE. Instead of migrating both the configuration AND container building to Nix straight away, you can keep the Dockerfile to build the nix configuration.
The biggest downside is that you don't take advantage of layers at all, but the upside is that you can gradually adapt your Dockerfiles, and reuse any Docker infrastructure or automation you already use.
But to briefly answer your specific questions: Docker files are commonly not reproducible because they contain arbitary stateful commands like `apt-get update`, `curl`, etc. For a layer with these kinds of commands to be reproducible you would need a mechanism to version and verify the result.
Nix provides such a mechanism, and a community package repository with versioned dependancies between packages. These are defined in a domain specific language called Nix (text files) and kept into a git repository. This should be familiar if you've used a package manager with lock files before.
You can guarentee the package version will stay in the repository by pinning your build to an exact commit hash in the repository.
Really random, but the illustrator who made the slide art for this (Annie Ruygt) also made logos and brand art for two YC startups, RethinkDB (rethinkdb.com) and Pachyderm (pachyderm.io) (where I worked, and which was founded by former RethinkDB engineers). She does great work!
I spent a half a day or so relatively recently trying to build our CI base image with Nix (at the recommendation of our infra team), but it was huge, and some stuff didn't work because of linking issues.
One issue that really bugged me was to build multi-arch images, it actually wants to execute stuff as the other architecture, and only supports using qemu with hardware virtualization for that. My build machine (and workstation) is a VM, so I don't have that. I do have binfmt-misc, though, so if you just happened to fork and exec the arm64 "mkdir" to "mkdir /tmp", it would have worked. Of course, this implementation is a travesty when docker layers are just tar files, and you can make the directory like this:
(As an aside, I'm sure this exact layer already exists somewhere. So users probably don't even have to download it.)
Every time I try nix, I feel like it's just a few months away from being something I'd use regularly. nixpkgs has a lot of packages, everything you could ever want. They all install OK onto my workstation. But "I need bash, python, build-essential, and Bazel" doesn't seem like something they're targeting the docker image builder at. I guess people just want to put their go binary in a docker image and ... you don't need nix for that. Pull distroless, stick your application in a tar file, and there's your container. (I personally use `rules_oci` with Bazel... but that's all it does behind the scenes. It just has some smarts about knowing how to build different binaries for different architectures and assembling and image index yaml file to push to your registry.)
> to build multi-arch images, it actually wants to execute stuff as the other architecture
You should be able to cross compile binaries for other architectures without actually running them. As long as the package's build files support it of course.
> and only supports using qemu with hardware virtualization for that
That doesn't sound right. You can use qemu for architectures that be only software emulated too.
I don't want to say it should be as simple as using pkgCross (https://nix.dev/tutorials/cross-compilation.html), but... are some specific issues with the usual process that you're running into?
> I spent a half a day or so relatively recently trying to build our CI base image with Nix (at the recommendation of our infra team), but it was huge, and some stuff didn't work because of linking issues.
You must be talking about the official Nix Docker image[1], which indeed is huge. I've been using it for years for a handful of projects, but if the size is an issue you can use the method mentioned in the article and build a very minimal image with only the stuff you specify.
Hmm? Cross compiling to docker images is exactly what I used nix for. I even had musl being used, it was the smallest image I could build with any tool and built the images quickly and consistently in ci with caching working well.
I never saw went being used so im a bit confused where that came into play for you
That's kind of an unfair comparison. The flake you linked:
1. Defines a Go binary (i.e., how to build it)
2. Defines a Docker image that uses said Go binary as an entry point
3. Defines a NixOS module that creates a systemd service that runs the Go binary (only relevant on NixOS)
4. Defines a NixOS test for the module that ensures that the NixOS module actually creates a systemd service that runs the Go binary as expected. The NixOS test framework is actually quite impressive - tests run in a QEMU VM that also runs NixOS :)
Note that only (1) and (2) are relevant to the linked article (+ some of the surrounding boilerplate).
I agree that it's not a fair comparison, but I will add that this is a big barrier to newcomers. Everyone experienced with nix builds their own ivory tower of a nix flake (myself included), so it's hard to find actually good examples of how to do basic things without wading through a bunch of other bullshit.
Newcomer here. Could anyone tell if std [0] is a good way to bring more sanity into flake design, esp. in avoiding ivory towery custom approaches? Using devenv.sh is another option, but I liked emphasis on creating a common mental picture of the architecture and focus on SLDC that std provides.
I haven't used std, but I would like to point you at what I think is the ideal way to organize a lot of nix: readTree from the virus lounge[0].
It doesn't add kitschy terms like "Cell" and "growOn", it's just a way to standardize attribute names according to where things live in the filesystem.
So in their repo, /foo/bar/baz.nix has the attribute path depot.foo.bar.baz
I will say that to understand how it works you need to have a solid grasp of the nix language. But once established I think it's a pattern that noobs could learn in a very quick and superficial way.
Well, I think that becomes clearer when reading comments on this HN thread. I found std after spending significant time to find out 1) wth is Nix/NixOS? 2) what would I use it for exactly? and 3) How to get going? Then on 3) I found reams of outdated or confusing docs making me doubt 1) and 2) again, as well as the frustrations of others on this.
Then this std background of "We bring clarity, clear mental picture, manageable Nix projects throughout the lifecycle." appealed a lot.
Does cell/cell block/target/actions terminology come from Nix then? I assumed that was std's solution (since it is under the heading Solution, and I haven't heard of it) so found the 'explanation' of it baffling. But if it comes from Nix it can be read sort of like a style guide for how to do what you're already doing with Nix?
If the target audience is limited to those already highly experienced (and yet frustrated) with Nix then I guess it's fine.
Ah, sorry I misinterpreted you before. Yes, these are std's abstractions to organize your Nix code and gradually 'grow' your solution. Rationale and explainers on these concepts are more spread about in the docs. The 'sales pitch' is another high-level txt than the one you passed on why to use std.
(PS. This cell breakdown reminded me a bit of Atomic Design for front-end UI to make that easier.)
I'm a beginner myself and have been trying my best to keep things simple. But I do agree that the complexity creep is quite tempting with Nix. Not sure why though..
I think nix has a fair amount of gravity - once you've started, assuming like it, you will quickly want to use it for everything.
I don't think most people's flakes are more complex than the alternative (which would be, I don't know, a bunch of different script, maybe some ansible playbooks?) but it is a bit daunting when all that complexity is wrangled into a single abstraction.
I thought I was weird for my bespoke ivory tower monorepo flake.nix, glad to hear I'm not the only one. It has been a tremendous help in managing my homelab.
Does Nix make it difficult to properly modularise nix files (flakes or not)?
Because it certainly looks like the things you listed are separate/orthogonal and should be in separate modules/files.
Having many years of Java experience this is the reason why I stick to Maven (not moving to Gradle) - it is opinionated and strongly encourages fine grained modularisation.
The thing I hate most about a Java codebase is 100 separate .java files with 30-50 lines each and somehow it indeed worked but I have no idea where to look at if I want to find out how is something implemented.
> Because it certainly looks like the things you listed are separate/orthogonal and should be in separate modules/files.
Nix absolutely allows that, to the point where I'm surprised that the linked example doesn't separate them. Most of my flakes have a flake.nix that's just a thin wrapper around a block that looks like
The thing that makes that miserable is all the inputs need to be declared in the top-level flake. They really should make using flakes in subdirectories of the same git repo painless. (Right now they're treated as if they were remote, and you need to "upgrade" them after every change; unusable.)
nix is a build tool a similae way that docker is a build tool. You define build scripts that call the tools you already use. The major difference is that a docker file gives you no way tl be sure you can reproduce that build in the future. A flake on the other hand gives you a high degree of trust.
Nothing got lost: both Nix and Maven are dependency management tools. Both are also build tools. The difference is that Maven was created as a Java build tool (and it stayed that way in general).
What we have today is that there is a multitude of dependency managers and - what's worse - all of them are _also_ build tools targeted at specific language.
Nix has a unique position because it is not language specific. Where it is lacking is missing standards and reusable libraries that would simplify common tasks.
I am comparing to Maven because I am looking for multi-platform Maven alternative. There is Bazel but its dependency management is non-existent. There is Buck2 which is great in theory but the lack of ecosystem makes it a non-starter.
Nix is the only contender in the space that offers almost everything and has a chance to become a de-facto standard of software delivery thanks to this.
What's missing though is... easy to use canned solutions similar to maven plugins.
In the link I referenced, it shows you how to use maven plugins in nix.
Nix has composable, reusable, and shareable functions. Nix is a full, be awkward, programming language. You'll find functions and flakes for nearly everything you might want to do. An example of one that is more plugin like is sops-nix.
Though have never used maven or maven plugins, I may be missing your overall point.
In Java/Maven ecosystem a lot of things is simple because there is a huge ecosystem of easy to integrate libraries/plugins.
Want a packaged spring boot application? There is a plugin for that. Want to package it in a container image? Just add a plugin dependency. Want to build an RPM or deb package? Add another two plugin dependencies. All various artifacts are going to be uploaded to a repository and made available as dependencies.
Missing a specific plugin? You can easily implement it in Java as a module and use it inside your project (and expose it as a standalone artifact as well).
I can’t find anything similar in Nix ecosystem.
Having a language allowing for this is not the same as having solutions already available.
I was reading this thread and now I finally understood what you mean by plugins.
Plugins kind of exist in Nix, except they're not called that. They are functions available in some specific module.
For example, `dockerTools` provides different functions like creating an image and other things. There is a module of fetchers, functions that retrieve source files from GitHub and other sources.
But I don't think there are many language-specific functions, like the ones you are describing. I can't think of any, except for the ones that build a Nix package from a language-specific package.
Yes, I know about flake.parts (didn't know about the other one).
But I'm not aware of the kind of libraries you mentioned.
There's FlakeHub[0], which is like a package index for flakes, so maybe we'll start to see there reusable stuff.
To be honest, now when I'm thinking about it - it seems to me Nix main weakness here is that it is a separate language and runtime.
Writing a Maven plugin (ie. a reusable piece of configuration management logic) is easy because you can use any library from the vast Java ecosystem (just add a library as a dependency of your plugin).
Doing the same in Nix requires recreating these libraries in Nix.
Looks like Maven might simply be the right choice...
You can have a hybrid approach, where you use Nix to provide the build environment (which includes Maven, for example). Then you use the Java-specific tools for the build itself. This should ensure your build has a high degree of reproducibility.
Yes, to the point where it can become more confusing than helpful. The fact that Nix handles merging data structures for you makes it easy to fall into over modularization.
Those 120 lines are not exactly representative. For example if I was writing this for a single service, I wouldn't bother making a new module and would inline it instead. Then, you've got many lines which would map 1:1 to an inlined systemd service description, so you're not getting rid of those whatever the system you choose. There's also a fancy way to declare multiple systems with an override.
This example is a "let's do a trivial thing the way you'd do a big serious thing". If you wanted to treat it as a one-off, I'm sure you could cut it down to 40 lines or so.
So I can write a Dockerfile using 17 verbs (13, actually), and it's understandable language to a 4th grader... or I can write 120 lines of complete abstract nix code that means nothing to someone doing software for 20 years.
Heredocs (described in the docs[1] under "shell form") was introduced 3 years ago[1]. The flag '--squash' has been in podman for quite a while[2]. It might be time for you to upgrade your tooling.
An alternative solution... for some definition of the term... is to write a "naive" Dockerfile like
RUN foo
RUN bar
RUN baz
and then just build it with something like... I think kaniko did this last I looked?... that smashes the whole thing into a single layer. Obviously that has other tradeoffs (no reuse of layers) but depending on your usecase it can be a good trade to make (why yes, I did cut my teeth in an environment where very few images shared a base layer, why do you ask?).
I addressed the 120 lines already and they're not completely abstract. You seem uncomfortable with an alternative approach and that's fine. But this is not a good intentions argument.
Dockerfile’s are absolutely not something a 4th grader would understand. It looks familiar to you, because you have already learnt it. They are definitely not trivial to understand before that, and the same is also true for Nix.
I just wanted to chime in here and say that Guix also has a nice and easy-to-use Docker option with "guix pack -f docker" [1]. Guix also has the advantage of using an already-used language (Guile/Scheme) rather than its own bespoke one. :)
I like the article but had a hard time following the specifics of the configs and commamds. Feels like it's more meant for people already familiar with nix, or sufficiently interested to study up while reading
That's very interesting, because as someone familiar with nix my take was that this was information I considered to be aimed at people who weren't familiar.
From experience - but I'm more than happy to be proven wrong as I would love to build all the Docker images with Nix.
What you linked is the equivalent of:
FROM scratch
COPY hello-world /hello-world
Of course that's small (hello-world is statically linked).
Try to add coreutils (or any other small package) and you'll see what I mean. In my experience the size of a Docker image built with some nix packages is greater than the Debian counterpart. I don't know why though.
No, that dockerfile is not equivalent because the hello-world is not statically built in the nix version.
However I'll give you that it could be smaller in more complex examples. For example glibcLocales is for all locales which is quite chunky but your application only needs one locale.
There are ways to properly build a nix container image so this kind of things doesn't happen. You'll find plenty of projects on GitHub dedicated to only that.
Coincidentally, Two days ago I was trying to adapt a flake to include a docker derivation. I came across with xelaso's page and inspired by the example provided (and after a few tries) I manage to compose a docker image. That was very cool!
BTW: Thanks Xelaso.
I didn’t fully grok how this works - what is the base image for the generated image? Also wouldn’t the image size be large if the glibc is copied over again
Default is none (i.e. like "FROM scratch" in a Dockerfile); you can specify a baseImage if needed, but I haven't had to yet. It works by copying parts of the nix store into the image as needed, but see also below.
> wouldn’t the image size be large if the glibc is copied over again
The original Nix docker-tools buildImage did suffer from poor reuse of common dependencies. Docker already has a way to reuse parts of images (e.g. if you build 7 images where the first N lines of a Dockerfile are the same, the 7 images will use a shared store for the results of running the first N lines). There are several backends for Docker storage that accomplish this in various ways (e.g. FS overlays, tricks with ZFS/btrfs snapshots).
Nix docker-tools now has a "buildLayeredImage" that uses this ability of Docker to share much of the storage for the dependencies, so if you build several images that all rely on glibc, you only pay the cost of storing glibc in docker once.
I've been using Dagger, it's awesome. It's the second take by the creators of Docker. It accomplishes most of what nix does for this problem
Write your pipelines and more in languages you already use. It unlocks the advanced features of BuildKit, like the layer graph and advanced caching.
nit: this post makes a point about deterministic builds and then uses "latest" for source image tags, which is not deterministic. I've always appreciated Kelsey's comment that "latest" is not a version
The fact that I couldn't point to one page on the docs that shows the tl;dr or the what problem is this solving
https://docs.dagger.io/quickstart/562821/hello just emits "Hello, world!" which is fantastic if you're writing a programming language but less helpful if you're trying to replace a CI/CD pipeline. Then, https://docs.dagger.io/quickstart/292472/argumentsdoubles down on that fallacy by going whole hog into "if you need printf in your pipline, dagger's got your back". The subsequent pages have a lot of english with little concrete examples of what's being shown.
I summarized my complaint in the linked thread as "less cowsay in the examples" but to be honest there are upteen bazillion GitHub Actions out in the world, not the very least of which your GHA pipelines use some https://github.com/dagger/dagger/blob/v0.10.2/.github/workfl...https://github.com/dagger/dagger/blob/v0.10.2/.github/workfl... so demonstrate to a potential user how they'd run any such pipeline in dagger, locally, or in Jenkins, or whatever by leveraging reusable CI functions that setup go or run trivy
Related to that, I was going to say "try incorporating some of the dagger that builds dagger" but while digging up an example, it seems that dagger doesn't make use of the functions yet <https://github.com/dagger/dagger/tree/v0.10.2/ci#readme> which is made worse by the perpetual reference to them as their internal codename of Zenith. So, even if it's not invoked by CI yet, pointing to a WIP PR or branch or something to give folks who have CI/CD problems in their head something concrete to map into how GHA or GitLabCI or Jenkins or something would go a long way
> The fact that I couldn't point to one page on the docs that shows the tl;dr or the what problem is this solving
Here's what the very first page of our documentation says (https://docs.dagger.io). I'd love suggestions for making it more clear.
Welcome to Dagger, a programmable tool that lets you replace your software project's artisanal scripts with a modern API and cross-language scripting engine.
[...]
Dagger may be a good fit if you are...
- Your team's "designated devops person", hoping to replace a pile of artisanal scripts with something more powerful.
- A platform engineer writing custom tooling, with the goal of unifying application delivery across organizational silos.
- A cloud-native developer advocate or solutions engineer, looking to demonstrate a complex integration on short notice.
Benefits to development teams:
- Reduce complexity: Even complex builds can be expressed as a few simple functions.
- No more "push and pray": Everything CI can do, your local dev environment can do too.
- Native language benefits: Use the same programming language to develop your application and its delivery tooling.
- Easy onboarding of new developers: If you can build, test and deploy, they can too.
- Caching by default: Dagger caches everything. Expect 2x to 10x speed-ups.
- Cross-team collaboration: Reuse another team's workflows without learning their stack.
Benefits to platform teams:
- Reduce CI lock-in: Dagger functions run on all major CI platforms - no proprietary DSL needed.
- Eliminate bottlenecks: Let application teams write their own functions. Enable standardization by providing them a library of reusable components.
- Save time and money with faster CI runs: CI pipelines that are "Daggerized" typically run 2x to 10x faster, thanks to caching and concurrency. This means developers waste less time waiting for CI, and you spend less money on CI compute.
- Benefit from a viable platform strategy: Development teams need flexibility, and you need control. Dagger gives you a way to reconcile the two, in an incremental way that leverages the stack you already have.
> https://docs.dagger.io/quickstart/562821/hello just emits "Hello, world!" which is fantastic if you're writing a programming language but less helpful if you're trying to replace a CI/CD pipeline
We went back and forth on this. On the one hand, starting with "hello world" makes it plain that Dagger Functions are "just functions", and then gradually introduces more concepts, such as a native `Container` and `Directory` type. On the other hand, you're right that "hello world" is not useful on its own, so you need to read through more pages before a realistic example. Note that this criticism applies equally to all "hello world" examples everywhere.
In any case, this is a valid criticism and I'm tempted to try going straight to a build function as you requested.
> Related to that, I was going to say "try incorporating some of the dagger that builds dagger" but while digging up an example, it seems that dagger doesn't make use of the functions yet <https://github.com/dagger/dagger/tree/v0.10.2/ci#readme> which is made worse by the perpetual reference to them as their internal codename of Zenith. So, even if it's not invoked by CI yet, pointing to a WIP PR or branch or something to give folks who have CI/CD problems in their head something concrete to map into how GHA or GitLabCI or Jenkins or something would go a long way
You are absolutely right, we have started porting over our CI to functions, but have not finished yet.
As many bazel rules, rules_oci's predecessor (rules_docker) was an unmaintained spagetti of hell, now we are pushed to rules_oci, and it's rpmtree recommended way of installing rpms which then in turn don't support post install script...
All this bazel-is-our-savior complex burns down when we want to build a tiny bit complicated thing with it. And we unfortunately do try to do that (our devbox image), which with full caching takes long minutes to an hour, and it's a freaking thousands pine mess instead of using a lined dockerfioe with pinned versions.
I absolutely hate bazel and its broken unmaintained Google-abandoned rulesets and I wish we either used either Docker or Buck2 for everything.
That's pretty much how building on nix works, except you don't need base image or your application file, you specify what command to run from which package and it will be placed in the container with all runtime dependencies automatically.
Of course you can customize the container further if needed.
Little known (possibly unintended) feature, but you can put the `toplevel` attribute of a nixosSystem into docker image `contents`, which lets you use NixOS modules to set things up. Just be sure to import the minimal preset, because those images get large.
Unfortunately booting the entire system with /init is largely broken, especially without --privileged. This would be an amazing feature if it didn't require so much extra tinkering.
If you skip the docker and use systemd-nspawn automated by NixOS, that's just `containers.foo = { autoStart=true; ...; config = {config,pkgs}: { just another nixos config here }; };`
I wanted to love nix. It seems like something I would like. I tried to compile rust using nix on my mac. Didn't work, known bug. I reinstalled my desktop to use nixos. I got lost between flakes, nixpkgs, homemanager. I managed to get vscode installed but when I added the nix extension (declared in nix) it would refuse to run vscode... It's just not a good experience so I reinstalled arch
I keep reading about Nix and I still don't understand what it does better than Docker, all the example in the post are trivial to do in a Dockerfile so where is the added value?
Docker build are deterministic and easily reproductible, you use a tagged image, that's it, it set in stone.
The 0.01% of Dockerfile that don't work, what does it even means, what does not work?
The other thing is about that buildGoModule module so now you need somehow a third party tool to use or build Go in a Docker image, when using a Dockerfile you just use regular Go commands such as go build and you know exactly what is going on and what args you use to build the binary.
As for the thing about using Ubuntu 18 which is out of date and not finding it, most orgs have docker image cache especially since docker hub closed access to large downloads, but more importantly there is a reason that's its not there anymore, it's not secure to use it, it's like wanting to use the JVM 6, you should not use something that is out of date security wise.
Docker builds are not deterministic, I don't get where you get that idea. I can't count the hours lost because the last guy who left one year ago built the image using duck tape and sed commands everywhere. The image is set in stone, but so is a zip file, there's nothing special here.
Building an image using nix solves many problems regarding not only reproducible environments that can be tested outside a container but also fully horizontal dependency management where each dependency gets a layer that's not stacked on one another like a typical apt/npm/cargo/pip command. And I don't have to reverse engineer the world just to see what files changed in the filesystem since everything has its place and has a systematic BOM.
So is it right, to make docker reproducible it needs to either build dependencies from source from say a git hash or use other package managers that are reproducible or rely on base images that are reproducible.
And that all relies on discipline.
Just like using a dynamically typed programming language can in theory have no type errors at run time, if you are careful enough.
Right; you could write a Dockerfile that went something like
FROM base-image@e70197813aa3b7c86586e6ecbbf0e18d2643dfc8a788aac79e8c906b9e2b0785
RUN pkg install foo=1.2.3 bar=2.3.4
RUN git clone https://some/source.git && cd source && git checkout f8b02f5809843d97553a1df02997a5896ba3c1c6
RUN gcc --reproducible-flags source/foo.c -o foo
but that's (IME) really rare; you're more likely to find `FROM debian:10` (which isn't too likely to change but is not pinned) and `RUN git clone -b v1.2.3 repo.git` (which is probably fixed but could change)...
And then there's the Dockerfiles that just `RUN git clone repo.git` and run with whatever happened to be in the latest commit at the moment...
Possible; I don't have a feel for the relative likelihoods. I think the thing nix has going for it is that you can write a nix package definition without having to actually hardcode anything in and nix itself will give you the defaults to make ex. compilers be deterministic/reproducible, and automate handling flake.lock so you don't have to actually pay attention to the pins yourself. Or put differently; you can make either one reproducible, but nix is designed to help you do that while docker really doesn't care.
It's actually how nix works by default. When you pull in a dependency, you are actually pulling in a full description of how to build it. And it pulls in full descriptions of how to build its dependencies and so on.
The only reason nix isn't dog slow is that it has really strong caching so it doesn't have to build everything from source.
Docker can resolve dependencies in a very similar manner to nix, via multi-stage builds. Each FROM makes one dependency available. However, you can only have direct access to the content from one of the dependencies resolved this way. The other ones, you have to COPY over the relevant content --from at build time.
You're totally right about the underlying container image format being much more powerful than what you can leverage from a Dockerfile. That's exactly the thing that makes nix a better Docker image builder than Docker! It leverages that power to create images that properly use layers to pull in many dependencies at the same time, and in a way that they can be freely shared in a composable way across multiple different images!
A Docker FROM is essentially the equivalent of a dependency in nix... but each RUN only has access to the stuff that comes from the FROM directly above it plus content that has been COPY-ed across (and COPY-ing destroys the ability to share data with the source of the COPY). For Docker to have a similar power to nix at building Docker images, you would need to be be able to union together an arbitrary number of FROM sources to create a composed filesystem.
Even with the Dockerfile format you can union those filesystems (COPY --link).
People use the Dockerfile format because it is accessible.
You can still use "docker build" with whatever format you want, or drive it completely via API where you have the full power of the system.
I actually hadn't heard of COPY --link, but it's interesting because it seems to finally create a way of establishing a graph of dependencies from a Dockerfile! It doesn't sound like it's quite good enough to let you build a nix-like system, though, because it can only copy to empty directories (at least based on what the docs say). You really need the ability to e.g. union together a bunch of libraries to from a composite /lib.
I'm not sure what you mean by 'You can still use "docker build" with whatever format you want'. As far as I'm aware, "docker build" can only build Dockerfiles.
I'm also not sure what you mean when you mention gaining extra abilities to make layered images via the API. As far as I can tell, the only way to make images from the API is to either run Dockerfiles or to freeze a running container's filesystem into an image.
docker build is backed by buildkit, which is available as a grpc service ("docker build" is a grpc client/server).
Buildkit operates on "LLB", which would be equivalent to llvm IR.
Dockerfile is a frontend.
Buildkit has the Dockerfile frontend built in, but you can use your own frontend as well.
If you ever see "syntax=docker/dockerfile:1.6", as an example, this triggers buildkit to fire up a container with that image and uses that as the front end instead of the builtin Dockerfile frontend.
Docker doesn't actually care what the format is.
Alternatively, you can access the same frontend api's from a client (which, technically, a frontend is just a client).
Frontends generate LLB which gets sent to the solver to execute.
OK, wow, this is interesting indeed. I didn't realize just how much of a re-do of the build engine Buildkit was, I had just thought of it as a next-gen internal build engine, running off of Dockerfiles.
Applying this information to the topic at hand:
Given what Buildkit actually does, I bet someone could create a compiler that does a decent job transforming nix "derivations", the underlying declarative format that the nix daemon uses to run builds, into these declarative Buildkit protobuf objects and run nix builds on Buildkit instead of the nix daemon. To make this concrete, we would be converting from something that looked like this: https://gist.github.com/clhodapp/5d378e452d1c4993a5e35cd043d.... So basically, run "bash" with those args and environment variables, with those derivations show below already built and their outputs made visible.
Once that exists, it should also be possible to create a frontend that consumes a list of nix "installables" (how you refer to specific concrete packages) and produces an oci image out of the nix package repository, without relying on the nix builder to actually run any of it.
If you're using Nix, that is what you are ultimately producing, it's buried under significant amounts of boilerplate and sensible defaults. Ultimately the output of Nix (called a derivation) reads a lot like a pile of references, build instructions, and checksums.
You can also use a hammer to put a screw in the wall.
Dockerfiles being at their core a set of instructions for producing a container image could of course be used to make a reproducible image. Although you'd have to be painfully verbose to ensure that you got the exact same output. You would actually likely need 2 files, the first being the build environment that the second actually get built in.
Or you could use Nix that is actually intended to do this and provides the necessary framework for reproducibility.
fun fact: there actually is a class of impact driver[1] that couples longitudinal force to rotation to prevent cam-out on screws like the Phillips head when high torque is required
Most Docker builds are not remotely deterministic or reproducible, as most of them pull in floating versions of their dependencies. This means that the same Dockerfile is likely to produce different results today than it did yesterday.
Aren’t Nix builds actually deterministic in that they’ll build the same each time? Docker doesn’t have that, you’re just using prebuilt images everywhere. Determinism has a computer science definition, it’s not “build once run anywhere,” it’s more like “builds the exact same binary each time.”
So, outside of the fact that a nix build disables networking (which you can actually do in a docker build, btw) how would you check all those build scripts in nix?
You don't. Those scripts will just fail forcing you to rewrite them. This is why some people trying to create new packages often complain, because they need to patch up original build for given application to not do those things.
There are still ways that package will not be fully reproducible, for example if it uses rand() during build, Nix doesn't patch that, but stuff like that is fortunately not common.
I'm not sure that this is a Docker problem but you do have a point. I've used docker from the very beginning and it always surprised me that users opted to use package managers over downloading the dependencies and then using ADD in the docker file.
Using this approach you get something reproducible. Using apt-get in a docker file is an antipattern
Why? — I agree that it’s not reproducible, but so what?
We have 2-3 service updates a day from a dozen engineers working asynchronously — and we allow non-critical packages to float their versions. I’d say that successfully applies a fix/security patch/etc far, far more often than it breaks things.
Presumably we’re trying to minimize developer effort or maximize system reliability — and in my experience, having fresh packages does both.
This feels like moving the goalposts. This is on a thread which began with the statement that Docker is reproducible. Will we next be saying that, OK, it's an issue that Docker isn't reproducible, but it's doing it for a noble reason?
Regardless.... I can give a few reasons that it matters, off the top of my head:
1) Debugging: It can make debugging more difficult because you can't trace your dependencies back to the source files they came from. To make it concrete, imagine debugging a stack trace but once you trace it into the code for your dependency, the line numbers don't seem to make any sense.
2) Compliance: It's extremely difficult to audit what version of what dependency was running in what environment at what time
3) Update Reliability: If you are depending on mutable Docker tags or floating dependency installation within your Dockerfile, you may be surprised to discover that it's extremely inconsistent when dependency updates actually get picked up, as it is on the whim of Docker caching. Using a system that always does proper pinning makes it more deterministic as to when updates will roll out.
4) Large Version Drift: If you only work on a given project infrequently, you may be surprised to find that the difference between the cached versions of your mutably-referenced dependencies and the actual latest has gotten MUCH bigger than you expected. And there may be no way to make any fixes (even critical bugfixes) while staying on known-working dependencies.
Docker doesn't give you the tooling to build a package and opts for you to bring the toolchain of your choice.
Docker executes your toolchain, and does not prescribe one to you except for how it is executed.
Nix is the toolchain, which of course has its advantages.
In terms of builds and dependency management, Docker and nix actually work pretty similarly under the covers:
Both are mostly running well-controlled shell commands and hashing their outputs, while tightly controlling what's visible to what processes in terms of the filesystem. The difference is that nix is just enough better at it that it's practical to rebase the whole ecosystem on top of it (what you refer to as a "toolchain") whereas Docker is slightly too limited to do this.
Uh, I never even mentioned apt. Docker and nix are, likewise, very different. In not super familiar with either but I do know docker isn’t reproducible by design whereas nix is. I’m not sure nix is always deterministic, though i know docker (and apt) certainly aren’t, nor are they reproducible by design.
So the thing here is docker provides the tooling to produce reproducible artifacts with graphs of content addressable inputs and outputs.
nix provides the toolchain of reproducible artifacts... and then uses that toolchain to build a graph of content addressable inputs in order to produce a content addressable output.
So yes they are very different, but not in the way you are describing.
Using nix, just like using docker, cannot guarantee a reproducible output.
Reproducible outputs are dependent on inputs.
If your inputs change (and inputs can even be a build timestamp you inject into a binary) then so does your output.
With nix, you just have to be careful not to do anything non deterministic to get a deterministic build. With docker build, you have to specifically design a deterministic build yourself. It’s easier to just not use inputs that change than to design a new build that’s perfectly deterministic.
The timestamps thing is part of ensuring that archives will have the correct hash. Nix ensures that the inputs to a build, that being the compiler, environment, dependencies, file system, are exactly the same. The idea being then that the compiler will produce an identical output. Hash's are used throughout the process to ensure this is actually the case, they are also used to identify specific outputs.
The Nix idea is to start building with a known state of the system and list every dependency explicitly (nothing is implicit, or downloaded over net during build).
This is achieved by building inside of a chroot, with blocked network access etc. Only the dependencies that are explicitly listed in the derivation are available.
> I still don't understand what it does better than Docker
It doesn't break as you scale. If you don't need that, then keep using Docker. (Personally, for me "scale" starts at "3 PC's in the home", so I eventually switched all of them to NixOS. I don't have time to babysit these computers.)
> Docker build are deterministic and easily reproductible
No, they definitely aren't. You don't really want to go down this rabbit hole, because at the end you realize Nix is still the simplest and most mature solution.
That's the interesting bit about Dockerfiles. They look _looks_ deterministic, and they even are for a while while you're looking at it as a developer. I've done a detailed writeup of how it's not deterministic in https://docs.stablebuild.com/why-stablebuild
I build a lot of Docker images using Nix, and while yes it’s generally more pleasant than using Dockerfiles, the 128 layer limit is really annoying and easy to hit when you start building images with Nix. The workaround of grouping store paths makes poor use of storage and bandwidth.
Yes, one of the main downsides of Docker images using Nix is the 128 layer limit. It means we have to use a heuristic to combine packages into the same layer and losing Nix’s package granularity. When building containers with Nix packages already on a Nix binary cache you also have to transform the Nix packages into layer tarballs effectively doubling the storage requirements.
Nix-snapshotter brings native understanding of Nix packages to the container ecosystem so the runtime prepares the container root filesystem directly from the Nix store. This means docker pull == Nix substitution and also at Nix package granularity. It goes a bit further with Kubernetes integration that you can read about in the repo.
What's the state of deployment for something like nix-snapshotter nowadays (with the realization that the answer depends on which of N k8s install methods might be in use)?
I assume it's mostly in the field of ... "you're making a semi-large investment on this enough that you're doing semi-custom kubernetes deployments with custom containerd?"
Or maybe the thought is nix-snapshotter users are running k8s/kubelet with nixos anyway so its not a big deal to swapout/add containerd config?
Yes it’s going to depend on which k8s distribution you’re using. We have work in-progress for k3s to natively support nix-snapshotter: https://github.com/k3s-io/k3s/pull/9319
For other distributions, nix-snapshotter works with official containerd releases so it’s just a matter of toml configuration and a systemd unit for nix-snapshotter.
We run Kubernetes outside of NixOS, but yes the NixOS modules provided by the nix-snapshotter certainly make it simple.
Sorry to bug you with more questions but I literally dreamed of nix-snapshotter for years, so I'm excited. Do you know how this (installation burden) translates in the real world (GKS? AKS? etc)?
Can one even get away with abusing DaemonSets/hostDir/privileged on hosted clusters to modify their own installations? Or is `nix-snapshotter` just sort of out of the question on those provided solutions?
I’m not sure what you mean by modifying your own installation. Like running k8s on NixOS and then using a nix-snapshotter based DaemonSet to modify k8s on the host? At first glance it seems like vanilla k8s can do this already, nix-snapshotter just makes it more efficient / binary matching.
I mean, I guess to put it simply "SSH to the worker node to fix it" isn't really viable. At all.
Some folks used to use DaemonSets + hostDir to do that node configuration instead of SSH. Which is weird, but less weird than "you can't autoscale nodes anymore because you have a manual bootstrap step".
I haven't tried it yet as I need to produce containers that can work on public cloud k8s, but it definitely looks like the way to go. All the existing methods for grouping store paths into layers are finnicky, brittle, and non-optimal.
It is truly magical for handling large, multi-layered containers. Instead of building the container archives themselves and storing them in the nix store, it builds a JSON manifest that is consumed by a lightly patched version of skopeo that streams the layers directly to either your local container engine or the registry.
This means you never rebuild or reupload a container layer that is unchanged.
Disclosure: I contributed a change in nix2container that allows cheaply pulling non-Nix layers into the build, just using the content hashes from their registry manifests.
Nixpkgs' streamDockerImage does something similar, instead of storing a multi-layer tarball it produces a script that cats them all together on demand, ready to feed into `docker load` or whatever.
In that case streamDockerImage produces script that you run then you pipe output to skopeo or docker.
nix2container wraps all of that and it automatically runs it in behind the scenes when you call nix run.
The whole image generation is much more efficient as well. In the standard streamDockerImage you get a script that generates a docker layers and the image. In nix2container all layers are stored in nix cache so subsequent run doesn't regenerate them. I believe that was the main goal behind this solution.
Another benefit (and I think it is even better than the caching) is that it also allows you manually specify what dependencies go to each layer.
The automatic way that the code offers, is nice in theory, but because docker has a limit of 128 layers, in practice it starts nicely then the last layer is a clump of remaining dependencies that didn't fit in previous layers.
With nix2container I managed for example to create the first layer which contains just Python and all of its dependencies, then the next layer are my application dependencies (Python packages) the last layer is my application.
With this approach a simple bugfix in the application only replaces the last layer that's the size of a few kb. Only a change of Python version will rebuild the whole thing.
Nix falls into a category where it's incompatible enough with upstream that it requires significant packaging effort, for most libraries. This is especially so when you have complex C or C++ libs with imperfect Cmake and Autotools build scripts or janky Makefiles. Wrapping everything for Nix becomes a significant portion of the work.
Frequently it's not packaged or some version was but the upstream build system changed an you have to do it yourself.
If you don't either benefit significantly from Nix's upsides (which most projects really don't), or you don't enjoy this and do it for fun, it's not a productive use for your time. If you can convince your employer to pay you to shave that yak, it's a net waste.
It is very easy to overload package versions locally for your needs, and it's quite easy to push that upstream to Guix so that others may benefit as well :)
Can you explain (or point to an explanation) of exactly how to do that? The Guix versions of Erlang and Elixir are way out of date, and I would like to push a fix.
See here[0] for pushing a fix, here[1] for the anatomy of a package definition (often you only need to bump the version number and update the hash, but compilers may be a bit more involved). It may be useful to define package variants[2], which is what I do for some packages locally. You can also see this page[3] for using ad-hoc package variants using command line flags. Hope this helps :)
Hm, simple version bumps tend to be outstanding for about a week for me before they are merged, and in the meantime I just add my package definition to my profile. Is there more to it than that?
Did they ever figure out how to run nix on top of MacOS? When I see a project uses Nix I know the creators aren't serious.
It's kind of like seeing something is written in Lisp or Scheme... somebody decided that functional systems and running code aren't worth it. Instead they'll build a system which is perfect in all ways in theory but nobody can or will ever use.
I've been using nix to build docker containers (from a Mac). I would like to skip docker as well, but I wouldn't know how. On the server, I use docker swarm, with traefik as load balancer, in a very small machine, which I can later grow. It works pretty well for me. Nix on the CI has never fail for anything but mistakes of my own.
What is automatic about docker? Do you mean other people have already put in the work? Or do you mean that it's more trivial to pip/npm/cargo install stuff?
Just override the font if you dislike it that much. That's the great thing about the interwebs, it's all just text, you can format it however you want.
It's actually a smooth readable (but tall-and-skinny) font on my personal laptop, and a pixelated mess on my work laptop; I've never figured out why... Oh, huh, they're using a custom font: https://xeiaso.net/blog/iaso-fonts/ so that should actually be consistent. (After the weekend I'll poke at it on the other laptop and see what's up.)
What I'm getting at is that they might (unintentionally) be complaining about a rendering bug and not the actual font...
Just for closure: back on the work machine and it looks fine. If I ever see the "pixelated" form again I'll get in touch (now that I know it's actually not supposed to look that way.)
The font is a custom build of Iosevka, which is almost certainly inspired by the commercial font Pragmata Pro (https://fsd.it/shop/fonts/pragmatapro/). When Pragmata Pro was first released a little over 10 years ago, it sold for around $400 (I know this because I and many, many others bought a copy back then).
As another commenter points out, you may have some rendering issue. Alternatively, you may just not like the font. Can't please everyone.
Having the author do this for a service written in Go is a mistake. Your first address for containerizing Go services should be ko: https://ko.build/ , and similar solutions like Jib in the Java ecosystem: https://github.com/GoogleContainerTools/jib . No need to require everyone to install something heavy like Nix, no need for privileged containers in CI to connect to a Docker daemon so that actual commands can be executed to determine filesystem contents, just the absolute bare minimum of a manifest defining a base layer + the compiled artifacts copied into the tarball at the correct positions. More languages should support this kind of model - when you see that pnpm's recipe (https://pnpm.io/docker), ultimately, is to pick a pre-existing node base image, then copy artifacts in and set some manifest settings, there's really no technical reason why something like "pnpm build-container-image", without a dependency on a Docker daemon, hasn't been implemented yet.
Using nix, or Dockerfile, or similar systems are, today, fundamentally additional complications to support building containerized systems that are not pure Go or pure Java etc. So we should stop recommending them as the default.
I have 2 systems running Nix, and I'm afraid to touch them. I've already broken both of them enough that I had to reinstall from scratch in the past (yes yes - it's supposed to be impossible I know), and now I've forgotten most of it. In theory, Nix is idempotent and deterministic, but the problem is "deterministic in what way?" Unless you intimately understand what every dependent part is doing, you're going to get strange results and absolutely bizarre and unhelpful errors (or far more likely: nothing at all, with no feedback). Nix feels more like alchemy than science. Like trying to get random Lisp packages to play nice together.
Documentation is just plain AWFUL (as in: complete and technically accurate, but maddeningly obtuse), and tutorials only get you part of the way. The moment you step off the 80% path, you're in for a world of hurt, because the underlying components are just not built to support anything else. Sure, you can always "build your own", but this requires years of experiential knowledge and layers upon layers of frustration that I just don't want to deal with anymore (which is also why I left Gentoo all those years ago). And woe unto you if you want to use a more modern version than the distribution supports!
The strength of Docker is the chaos itself. You can easily build pretty much anything, without needing much more than a cursory understanding of the shell and your distro's package manager. Or you can mix and match whatever the hell you want! When things break, it's MUCH easier to diagnose and fix the problems because all of the tooling has been around for decades, which makes it mature enough to handle edge cases (and breakage is almost ALWAYS about edge cases).
Nix is more like Emacs: It can do absolutely anything if you have the patience for it and the deep, arcane knowledge to keep it from exploding in a brilliant flash of octarine. You either go full-in and drink the kool aid, or you keep it at arm's length - smiling and nodding as you back slowly towards the door whenever an enthusiast speaks.