a) Allowing user to choose the location of the nix folder (for example $HOME/.nix) by using bwrap or proot
b) Same binary for Win/Linux/OSX on x86_64 (Also see https://ahgamut.github.io/2022/07/27/ape-rust-example/)
I've seen some of those changes from DavHau before, they're good! We've also been looking at the statically built version of Nix which is good for "single user" / home-directory-managed installs of Nix. It's a great idea and simplifies the onboarding a good bit. And, it adds some extra behavior which is a little weird. It also doesn't work on macOS, and we're hoping to nail down a (mostly...) consistent experience across the three.
No, it can't be used that way on NixOS, but we could put something out that sets up a similar experience on NixOS! Good idea!
re: NixOS, not sure exactly what this entails, but i'd love a standardized approach to getting my Nix Flakes onto a NixOS system.
Right now i have this adhoc bootstrap process and it feels a bit silly. I get it's slightly different to getting Nix itself installed, BUT, it's also somewhat related from a user point of view. It would be interesting to see something handle this out of the box.
I've tried Nix/Nixos a few times over the past years and at some point decided that the documentation was bad enough that any further tinkering would be a waste of my time. So, when it was announced that a group of users would be making a concerted effort to improve the documentation I was hopeful, but looking at this thread--in which Determinate Systems (or one of their employees) is blamed for running roughshod over other contributors--I'm not anymore: https://discourse.nixos.org/t/parting-from-the-documentation...
It's true that trust and respect between people who have poured themselves into Nix, NixOS and Nixpkgs was seriously damaged, and that this set the Nix documentation efforts back.
However, the documentation team still exists and meets— there are still people dedicating much effort to improving the official and community documentation in a concerted way.
I feel that those events are more an interpersonal wound internal to the community than they are the death knell for the overall effort to renew the Nix ecosystem's documentation, or a sign that Determinate Systems is not at all invested in the health and integrity of the community and its volunteer organizations. I think turning wholly pessimistic about those things at this point would be a big overcorrection.
No one said it's a death knell. I'm saying that the documentation has sucked for a long time, and that I'm not hopeful enough of its improvement that I would be willing to do the considerable work to overcome the shoddiness of the manuals, the many contradictory tutorials, etc.
Thanks for linking that. It was a pretty interesting read, one that I hadn't come across before. Has me seeing things slightly differently, and will keep it in mind when viewing future efforts.
I have nix currently installed from the official installer in multi-user mode on macOS - I believe it works fine, would there be any benefit to me uninstalling and reinstalling via your installer?
edit: i went ahead and uninstalled, and reinstalled via this, and WOW that was fast
I'm delighted to see your edit :). Our installer makes a few configuration choices that upstream doesn't, but the main driving goal for us is to be reliable, safe, and easy to undo.
Oh and as long as you're being opinionated about installing and making it easy for newcomers, have you considered including an option to install home-manager and nix-darwin? My uneducated guess is they are typically used by folks on OSX.
Have you used home-manager as well? Would be interested to hear your thoughts on the two, and what specifically makes nix-darwin so nice overall. I haven't used it before, just home-manager.
I've played with Home Manager, but I've never really used it long term as much as I have Nix-Darwin. (I'm not super interested in HM's dotfile management features, since I already have tools I like for that.)
What's nice about Nix-Darwin for me is mainly how NixOS-like it is. I like keeping its config under /etc and treating systems using it like NixOS. There are a few modules that I think are pretty impressive and atypical for the Nix world, like the Homebrew module. I'm not sure if HM has something comparable.
Homebrew is a big gap in home-manager. In fact, my current setup (and the most common I think) is to use homebrew for casks, home-manager for everything else (and I also do some Brewfile management with home-manager).
Does the multi-user mode work on SELinux-enabled systems and, if not, is it possible to install in single-user mode? The default installer's multi-user mode doesn't work with SELinux enforcing (https://github.com/NixOS/nix/issues/2374), and I didn't see a flag for enabling single-user mode when I used `install --help`. (There are some projects to make SELinux support work like https://github.com/dnkmmr69420/nix-with-selinux, but it's not clear to me if that would even work with this installer in particular.)
This would be huge for making Nix viable at certain organizations, as I'm sure you already understand! I would love to just be able to point people at an installer like this for usage with SELinux.
This looks good. I know I'm about to speak towards the OS version and not the Nix that can run atop other distros, but I definitely had some hiccups on the way to my NixOS-on-ZFS-root install (1), such as the configuration of the root user, whether to declaratively or dynamically configure non-root users and wifi networks (and having a working wifi driver that worked with my onboard chip to begin with... I truly believe that linux installs should not be expected to have to bootstrap via Ethernet at this point). Admittedly, a few of these things were due to not using the official GUI installer, but IMHO the commandline non-GUI install of NixOS should be as painless as possible as well.
Regarding your argument against Bash, isn't it true that this is just a different version of the same bootstrapping problem, and that downloading a temporary but specific Bash version would get around that? (I agree that working with Bash leaves a lot to be desired. But it's also more accessible to end-users to hack on, observe, and learn from, than a compiled Rust binary is. And tools like Copilot and ChatGPT and https://www.shellcheck.net/ make working in Bash MUCH more painless, I've found.)
Couple more questions: 1) Is your installer idempotent? (I think it is, since you track what's been done in a JSON structure; just confirming! If the JSON file is lost, are you still idempotent though? lol) 2) Are you a profitable company or just a group of Nix believers?
On NixOS: I generally agree, though it can be tricky to get that done with NixOS. I bet we can make some inroads there.
Easy questions:
> idempotent
It isn't currently idempotent, but we're working on "curing" which brings some idempotency to it, and allows for "rescuing" installations that aren't working correctly. Curing will also support creating a JSON document for installs that don't already have one.
> profitable company or just a group of Nix believers
We are a for-profit company, and we are a group of Nix believers :).
Bash:
Bash is really hard to get right, and really hard to make fast, and really hard to extend its featureset to more complicated use cases. We could "just" download a bash, but when we're trying to navigate disk encryption and APFS volumes and mounts ... it is quite a lot easier to write and test that code in a "bigger" language.
Also note that it isn't just Bash that we'd need to download, but coreutils and other utilities that Bash is really built on top of. We tried to fix a pretty simple bug last year in the Bash installer, and it took two weeks to validate a relatively straightforward, one line change was valid on all the platforms we target.
> Also note that it isn't just Bash that we'd need to download, but coreutils and other utilities that Bash is really built on top of. We tried to fix a pretty simple bug last year in the Bash installer, and it took two weeks to validate a relatively straightforward, one line change was valid on all the platforms we target.
This clarifies a lot of what confused me in TFA. I maintained millions of lines of bash code, some of which is decades old and run into only a single bash incompatibility that wasn't fixable with shopt. On the other hand, incompatibilities in programs bash is running are rampant, particularly between GNU and non-GNU systems (and even within GNU there are more surprises than you might think). The phrase "different bash implementations" was a major red herring for figuring out what you were talking about.
> but coreutils and other utilities that Bash is really built on top of
This really is the problem with bash. It doesn't have a stdlib. Posix tried but didn't succeed in sufficiently standardizing userlands. Bash is friendly and easy to hack on, but hard to manage at scale, for this reason. And this crops up all the damn time when you're working with compiled languages. Nobody wants to have two languages. But if one of your languages is compiled, then nobody wants to write and maintain literally everything in that language. At some point you need scripting. Bash is usually what's reached for. But OSX doesn't ship a recent bash. And your coders are probably running OSX. So you have to write scripts that target ancient bash so the devs can work locally, but also work in the cattle farm. You'll spend lots and lots of time fighting it. Or have ridiculous 'setup' docs that will trip up newbies and you'll have to troubleshoot their setups when they inevitably not follow them right or they just don't work leaving their personal systems in an unknown state.
Or just, yannow, use a scripting language to script with instead of a shell. Or, hear me out here, use the 'scripting' language to build your whole app. You very very very probably can. Ruby's object oriented and is way friendlier than bash, ditto for Python.
We're working on broadening the use-cases that Nix is appropriate for. Things like https://zero-to-nix.com/, https://www.riff.sh/, and the Determinate Nix Installer's goal is to make installing Nix a "low cost" / "low risk" activity to bring people in.
We're always looking for good writers, Rust devs, and security folks. People who are in to these things should reach out with their resume: careers@determinate.systems. Note that existing Nix expertise is not required.
> Regarding your argument against Bash, isn't it true that this is just a different version of the same bootstrapping problem, and that downloading a temporary but specific Bash version would get around that? (I agree that working with Bash leaves a lot to be desired. But it's also more accessible to end-users to hack on, observe, and learn from, than a compiled Rust binary is. And tools like Copilot and ChatGPT and https://www.shellcheck.net/ make working in Bash MUCH more painless, I've found.)
I haven't been working on the detsys rust installer, but I think I'm the person who's contributed the most to the official bash multi-user installer since Graham did a lot of work on it several years ago.
I ~like Shell, and I share your concern about it being less hackable. But the project has also been slowly bleeding goodwill around the lack of clean uninstall/reinstall for years, and I will make my peace with the compromises it takes to get there.
The problem isn't so much conjuring good shell expressions. Nix leans a lot on Shell, and I feel like the community has a fair number of people around who know Shell fairly well. The problems are more like:
- limited/tedious support for robust error-handling and recovery
- needing a significant amount of new code to handle removing what Nix adds (even though I've done a lot of work to develop the patterns to support eventually getting here)
- a fairly fat long tail of problems caused by things like edge-case environment/state differences (due to platform, due to users having surprising environments, etc.) and even flaky behavior on some systems
- screwing up the installer is fairly high stakes (and it's very common for even one-line changes to break the installer in some way for someone somewhere and end up needing to get reverted for a rethink)
- testing the installer is tedious (it's gotten a lot better over the past ~2 years--but that testing is also just covering the golden path), so iterating is slow
- writing and reviewing installer changes tends to entail (and sometimes suffer from a lack of) really broad knowledge of different platforms, shells, utilities, options, permissions, and user configuration ~patterns
Because the stakes are high and review is hard and testing is hard and iterating is slow, IME most of the people who are enthusiastic about working on it are naive about these headwinds. This might lead them to, say, submit a PR bigger than anyone will have the stomach to review/merge--or not have the time/energy to slog through weeks/months of review/edit/test iterations it takes to build high confidence in a big change. It's also a big drag on the number of people who have the stomach to continue making substantive installer PRs after they land one.
Before detsys started looking at the rust installer, I had picked at a POC for just bundling a bash and everything we touch (iirc at least coreutils, diff, patch). If their effort failed, that's definitely the approach I was going to recommend next to reduce the frequency with which we get bitten by a utility quirk/change. But that still would've left quite a bit of work to do to support uninstall and idempotent reinstalls.
I'm personally limiting the amount of non-fix work I do on the official installer to avoid falling into its gravity well and completely owning it. Once more, if any combination of the technical changes here are sufficient to multiply the number of people willing and able to touch the installer, I'll make my peace with the tradeoffs.
> if any combination of the technical changes here are sufficient to multiply the number of people willing and able to touch the installer, I'll make my peace with the tradeoffs.
Do you feel similarly about the Nix setup scripts that the installer works to ensure are sourced by user shells? I'd kinda love to see those go and be replaced by a little program that sets up a Nix environment and then launches a shell for you, kinda like the `login` program. The shell-based approach has made supporting incompatible shells kinda nasty and clunky.
I have managed to avoid fiddling with the init hook scripts themselves, so I hadn't actually thought about this specific idea. IDK if my thoughts will be productive/incisive.
I've definitely felt like the Nth-shell situation is untenable and that we'll either need to abstract a definition to generate the hooks from or find some other ~general/universal solution.
I like the way this suggestion sounds with respect to getting out of that maintainability problem, but I'm not sure if it'll work quite right?
Here's one crappy rabbit-hole we've gone down:
Since macOS changed its default shell to zsh, they've been overwriting /etc/zshrc with every update (this evicts Nix's hook, confuses/scares people, and makes them try to reinstall; note that they aren't using the Relocated Items treatment we'd expect from how they do this with most stuff in /etc/ including /etc/bashrc while it was still the default shell).
An enterprising zsh user PRed a fix that moves the hook to /etc/zshenv. macOS doesn't ship /etc/zshenv, so they don't overwrite it on update. It seems to work at a blush, but Nix and everything you install with Nix ends up at the end of your PATH instead of the beginning, so everything that depends on a PATH lookup will keep using your system utilities.
Why? IIRC because the /etc/zshprofile macOS ships will do "eval `/usr/libexec/path_helper -s`", and path_helper will add things from /etc/paths and /etc/paths.d to the left of the existing PATH. I suspect if we inject a binary or even a script to preconfigure this environment, the call to path_helper in the zsh init process is still going to leave everything we added at the end of the PATH. We could in theory add an /etc/paths.d/nix, but path_helper appends this after everything in /etc/paths, so it has the same problem. We could stick our Nix-related directories at the top of /etc/paths, but macOS does ship this file so we have no reason to have faith that macOS updates don't overwrite it or won't start doing so at some point.
(We might also be able to move this to /etc/zshenv and also define a `/usr/libexec/path_helper(){ ... }` function that behaves like WE want--but that could obviously have downstream effects on other code in user profile/init scripts that uses path_helper. No clue if this is common.)
Thank you, this is great! I love the focus on OSX support and on uninstalling. As a nix newbie, it can be difficult to resolve some issues and sometimes re-installing is the simplest option.
Is the end result of installing via the bash install script and this new installer the same? Can I use the new tool to uninstall nix installed via the bash script?
Thanks! The end result is not quite the same. For example, we turn on flakes, nix-command, and make a couple other minor tweaks to the Nix configuration to make what we think are user-friendly improvements. For example: https://github.com/DeterminateSystems/nix-installer/blob/mai...
Our installer also supports some environments that the upstream installer has a harder time with.
The new installer can't _yet_ uninstall a Nix installed from the upstream installer, but our upcoming work on "curing" will bring that.
There are no binaries for every package in Apple Silicon (but there are many).
Also, many Linux-purpose binaries install but they might not work after all. For example Buildah for Podman.
One if the bigger issues right now are mass rebuilds and content agreed derivations hopefully address that at least partly. Sadly there is not much work happening on them lately.
As somebody who was curious about Nix, I was pretty turned off by my first exerpeince.
I also found it humorous/ironic that for something that is all about like... deterministic and reproducible or whatever: https://github.com/NixOS/nix/issues/458#issuecomment-1019743... it touched the system in a bunch of places and was a zoo to uninstall
You used to be able to uninstall Nix on macOS with just `rm`. But the move towards multiuser by default combined with macOS changes restricting filesystem access really complicated things. :-\
Some of this, though:
> touched the system in a bunch of places
is probably unavoidable. For a daemon to persist, you have to plug into launchd. To get a writable /nix, you need a special volume. To plug into all the shell environments and set appropriate environment variables, you need to configure a bunch of shells.
Some of that can be improved, for sure. Some of the logic for setting Nix profiles could be moved out of shell scripts, and then only a few static environment variables could be set in /etc/environment instead of modifying shell startup configurations to source a special script, for example. But the messiest part of a tool like Nix is always going to be where it plugs into some other system which has none of Nix's guarantees and isn't under Nix's control.
An improved installer (maybe just like the one in the OP! I haven't tried it yet) is probably the best that can be done here, in automating the handling of those messy points of integration. But it will be a matter of hammering out bugs over time like for any old installer, rather than something that relies on the kinds of guarantees Nix tries to make for its own internal operations. An uninstaller for Nix is something that can (and should!) be done well, but not by simply relying on Nix's usual guide rails.
Yeah the Mac experience is less than ideal for sure. There seems to be some animosity to mac on the core team (not a dig, just an observation) so I think they tend to dance around the platform for UX things (and even functional support at times). I think Determinate are trying to fill that gap somewhat.
On Linux there are far few steps:
- delete /nix
- delete build users (could be removed in the future)
- delete she'll script files in /etc/profile
- delete systemd nix-daemon service
- delete symlinks in user directory
As a very nix-curious non-user, this looks like a good place to start; is it?
I'm mostly interested in managing my home directory to start (replace gnu stow for dotfiles), but also to manage all my other software in a way that ports nicely between systems and OSes (skip over brew, flatpak, apt, etc.). I think this means home-manager, right?
Should I just start here with Determinate Nix, or somewhere else?
I share a bunch of configuration between my M1 Macbook and x86 desktop in my office. I use home-manager with config that is organized like nixos/*, macos/*, and common/*. The first two just import the latter. Here's a link if it's helpful:
It's more NixOS oriented, but most of the knowledge is obviously transferable to a bare-bones (if one could say that) Nix installation on top of other OSes.
With regards to all the Nix/NixOS documentation already out there, I'm trying to keep my stuff extremely straight-forward and rather simplistic (but not superficial), so the people outside of Nix (and even outside of Linux) may give it a solid try.
Just to note, if you wish to subscribe to the newsletter, it's broken now, gonna fix it in a few days.
Also a question to @grhmc: would you maybe have some doc that describes how to UN_install nix via your installer on macOS?
Yes you are basically describing home-manager. I dont use NixOS so I'm not sure if home-manager is a necessary addon there, but on Mac I use nixpkgs + home-manager.
I still use stow, but for my nix and home-manager config dirs only. The rest of my dotfiles is generated from home-manager, and my system nix stuff from nixos-rebuild. I am successfully sharing a config between my work and my personal laptop, it's something really amazing.
I've only used nix on NixOS, so I don't know about installing nix on top of an existing (non-NixOS) OS. A collegue is running just home-manager on ubuntu, with some success.
Now that this binary installer exists I would definitely recommend it.
I’ve deleted Homebrew from my system and I’m now using the same home-manager setup for my macOS laptops as well as my WSL installation on windows.
It’s so great to have the exact same tools and shell configuration installed everywhere.
I have not used flakes yet but if that will help me deal with the differences between my systems I’m definitely for it!
E.g I have a work Mac, personal Mac, personal PC with WSL.
When on my work machine, usernames are different and my Git config should identify me as me, and I shouldn’t have access to my personal credential for personal GitHub, RSA private keys for personal servers, etc.
But all else should be the same.
Mostly there but the credentials part I am still figuring out.
You can try Nix first(the package manager) without bothering with the OS initially. I have nix with home-manager in my OSX laptop. Becoming comfortable with the package manager makes understanding NixOS much easier.
Of course you can dive straight into the whole thing if you would prefer. I've tried that too (in a former chromebook). NixOS is pretty eye opening.
This is really exciting, thank you for this work! We use nix at work to manage our dev environments, rust production builds, and docker images. We have a pretty solid setup path for new developers that basically looks like:
1) install nix
2) run setup
Looks like we might be able to get that down to just step two thanks to y'all.
I've been wanting to give Nix a try for a long time. However, I run Fedora and as far as I've seen Nix is pretty incompatible with SELinux. I'm not really ready to take the plunge and move to NixOS so I've just kind of been holding out.
curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install
on wsl 2 doesnt work gives error and quits saying can't get lock on /etc/passwd pid 629
I like nix because it seems like an alternative to docker. Yet. Im never going to use it, because it has complexity and i dont understand why. Which is why i dont use docker nor nix. I can work with both, i just dont want to waste my time on them unless i have no other option.
It's not a Docker alternative, they are very different. There's some overlap in the sense that you can package an app using both. But that would be like saying APT is a Docker alternative.
Docker (and containers in general) will provide filesystem, process and network isolation. Your process can pretend it has the whole machine for itself (and even a different linux distribution) even though it's not true.
A Nix package will not do any of that. The package manager solves the problem of managing application packages, versioning, and dependencies; and doing so in a way that's 'immutable'. There's some filesystem abstraction (specially if you use nix-shell) so you can pretend that a particular version is the only one that's installed in the machine, or that a package is installed when it's not.
> It's not a Docker alternative, they are very different. There's some overlap in the sense that you can package an app using both. But that would be like saying APT is a Docker alternative.
They do share one use case: run software without having to install it.
I think Docker mostly is about running services in an isolated manner, and is easy to distribute (because its images will run the same way in each host). -- But, I do see some CLI tools provide a docker container, too.
Nix, with its flakes, supports a `nix run` command which is very similar to `docker run` (if you're thinking of "run without installing" rather than "process / network isolation"). -- In that sense, it is like 'Docker, but without containers'.
e.g. you can run `nix run nixpkgs#helix` and try out helix, or even `github:helix-editor/helix` to build & run helix from its source.
I wonder if you took a look at some of the modifications done by portable-nix (https://github.com/DavHau/nix-portable), most important ones being:
And then, somewhat unrelated: