Heh, I think I need to add something to the README. I've been pronouncing it as "zeeb" in my head as in the first syllable Zebesian Space Pirate from Metroid, but TIL that that's canonically "Zay-bay-zee-uhn" so idk.
I kinda dig zeeb! Naming is hard. Really awesome project by the way! Should have mentioned that first. Build systems are neat. I've always wanted to try building a build system, in a "learn how it works" sense, not so much "yet another build tool".
Thanks! And go for it, it's a good learning experience! It's a really interesting problem domain and there's a lot of different directions you can take it.
According to Build systems à la carte, "it is not possible
to express dynamic dependencies in [Bazel's] user-defined build rules; however some of the pre-defined build rules require dynamic dependencies and the internal build engine can cope with them by using a restarting task scheduler, which is similar to that of Excel but does not use the calc chain." (p6)
IME import-from-derivation and similar in Nix is usually used for importing build configurations from remote repositories. Bazel has a repository rule system that is similar: https://bazel.build/extending/repo
So to answer your question: yes from the strictest possible definition, but in practice, I believe the tradeoffs are acceptable.
That's accurate (unless the config file attempts to read something from the build process, that will trigger a build).
It's a good point about debugging build problems. This is an issue I've experienced in Nix and Bazel as well. I'm not convinced that I have a great solution yet, but at least for my own debugging while using the system, I've included a `zb derivation env` command which spits out a .env file that matches the environment the builder runs under. I'd like to extend that to pop open a shell.
I'm not sure, I'm not a Nix expert. The comments here also refer to it as both instructions to build something, as well as the intermediate build artifact. This discussion[1] on the NixOS forums explains it as a "blueprint" or "recipe". So there's clearly a lot of confusion about what it is, yet everyone understands "blueprint", "recipe", or even "intermediate build artifact" if you want to be technical.
The same is true for "flakes". It's a uniquely Nix term with no previous technical usage AFAIK.
Ideally you want to avoid using specialized terms if possible. But if you do decide to do that, then your documentation needs to be very clear and precise, which is another thing that Nix(OS) spectacularly fumbles. Take this page[2] that's supposed to explain derivations, for example. The first sentence has a circular reference to the term, only mentioning in parenthesis that it's a "build task". So why not call it that? And why not start with the definition first, before bringing up technical terms like functions and attributes? There are examples like this in many places, even without general problems of it being outdated or incomplete.
Though I don't think going the other way and overloading general terms is a good idea either. For example, Homebrew likes to use terms like "tap" and "bottle" to describe technical concepts, which has the similar effect of having to explain what the thing actually is.
Docker is a good example of getting this right: containers, images, layers, build stages, intermediate images, etc. It uses already familiar technical terms and adopts them where they make most sense. When you additionally have excellent documentation, all these things come together for a good user experience, and become essential to a widespread adoption of the tool.
yes, i agree, nix should be considered the bible of bad documentations. it’s very bad at spotlighting the essentials and putting the non-essentials aside.
it’s especially surprising for derivations, because nix is really, in the end, a frontend for building derivations. everything else converges on it.
and then i go to nix.dev and derivations are presented after fetchers? no surprise it’s so confusing, even though the concept is quite simple.
a derivation is a dict that is composed of (1) a shell script and (2) environment parameters it will have access to. a nix command will read the derivation, create the environment with only these parameters and execute the script. that’s it.
everything else about nix language is about building derivations. like copying files into its store. for example, evaluating “${pkgs.hello}” will be interpolated into a path. so in your derivation, you can define an env variable “hello = ${pkgs.hello}/bin” and it will be available in your script as “$hello” and will have the value of “/nix/store/<hash>-hello/bin”. nix will do the fetching and storing for you. so you can have “command $hello” in your script. neat!
play around with evaluating the ‘derivation’ built-in function.
I'm 80% finished moving all of my servers from NixOS to Debian. I used NixOS for 3 years (even wrote some custom flakes) before finally giving up (for the final year I was just too scared to touch it, and then said "I shouldn't be scared of my OS"). I should know what "derivation" means, but I can't for the life of me remember...
I don’t know Nix, but I’ll describe how Guix works, and hopefully it will be obvious what the corresponding Nix concepts are.
A “package” is a high-level description (written in scheme) of how to build something, like: “using the GNU build system with inputs a, b, c, and configure flags x, y, z, build the source available at https://github.com/foo/bar”
The actual builder daemon doesn’t know about the GNU build system, or how to fetch things from GitHub, or how to compute nested dependencies, etc.; it is very simple. All it knows is how to build derivations, which are low-level descriptions of how to build something: “create a container that can see paths a, b, and c (which are themselves other derivations or files stored in the store and addressed by their hash), then invoke the builder script x.”
So when you ask guix to build something, it reads the package definition, finds the source and stores it in the store, generates the builder script (which is by convention usually also written in scheme, though theoretically nothing stops you from defining a package whose builder was written in some other language), computes the input derivation paths, etc., and ultimately generates a derivation which it then asks the daemon to build.
I believe in Nix, rather than scheme, packages are written in nix lang and builder scripts can be written in any language but by convention are usually bash.
So basically long story short, the package is the higher-level representation on the guix side, and the derivation is the lower-level representation on the guix-daemon side.
"Derivation" refers to the nix intermediate build artifact, a .drv file, which contains the instructions to perform the build itself. Basically a nix program compiles to a derivation file which gets run to produce the build outputs. The hash in the /nix/store for a dependency is the hash of the derivation. Conveniently if the hash is already in a build cache, you can download the cached build outputs instead of building it yourself.
Ah OK, then I'd actually never actually understood what a derivation is. But then again, the name "derivation" doesn't at all lead to guessing at such a definition, either.
Yeah I ended up with the same issue. While I’m technically inclined, I’m not nearly to the point where I can handle the fire hose of (badly named) abstraction at all levels like some people.
I could never have pulled off what this guy did https://roscidus.com/blog/blog/2021/03/07/qubes-lite-with-kv..., though ironically his journal is probably one of the best “how nix actually works” tutorials I’ve ever seen, even though it isn’t intended for that or complete for such a purpose. He’s the only reason I know that a derivation is basically an intermediate build object.
I'm about to start a project where I thought Nix might be useful. What do I need to watch out for? Where is it going to piss me off and send me back to Docker?
There are no guardrails. Whenever something goes wrong, you'll get weird cryptic errors in a seemingly unrelated area and have no clue how to fix it until you post to a support group to discover that you put a comma in the wrong place.
You'll spend a LOT of time fighting the system, which gets old fast. Docker may have a sucky plan format (and they STILL won't let you set your goddamn MAC address), but it's good enough for most things and not too terrible to learn.
Oh, and my personal favorite: Programming by maps.
Any time you put in a key that it doesn't recognize, it just gets ignored. So you get to spend hours/days trying to figure out why the hell your system won't do what you told it to, and won't even validate that what you put in makes sense.
I mean it has variable names, configurations, documentation, a file extension and lots of code and a history behind it, so the strange thing to me is trying to suggest a replacement phrase as if you don't know what it is, acting like it's some high-brow language used in a blog to look smart, complaining about how this makes it less accessible (paraphrasing a little), then rolling back saying you dabbled in Nix and acting like you know what it is.
I geniunely thought you knew nothing about derivations and were criticizing the blogger for writing the term in their blog, not the term standard to Nix itself. Which is just as weird to me as complaining about std::string, well why call it a string? it is obviously text!
> Which is just as weird to me as complaining about std::string, well why call it a string? it is obviously text!
It's really not, though. String is a common technical term used in programming languages for many decades. If a new language decided to call them "textrons", _that_ would be weird. And this is the exact thing Nix did with "derivations", "flakes", etc. There is no precedent for these terms in other software, so they're unfamiliar even to its core audience.
It would be different if Nix invented an entirely new branch of technology that didn't have any known precedent. But for a reproducible build system that uses a declarative language? C'mon.
No need to resort to obvious straw man arguments, you can just accept some people dislike the dev UX of Nix and move on, which is basically what me and others have been trying to say in this entire sub-thread, some much more detailed than me.
No idea why you keep digging at this, the takeaway was clear at least three comments ago.
One thing I like to see is a 'dry run' like 'make -n'. Although, maybe that's not possible in all cases.
Another possibility might be to output a something like a shell script that would do a rebuild the same way, so you can see what it did and hack it when debugging.
Yes. Dry runs at least, and better yet terraform-style planning that produces an artifact that can be applied. These should really be more common with all kinds of software
I would like to see more tools iterate on trying to do terraform-like output because while terraform diffs are interesting, practically most of my teammates couldn’t tell what the fuck they said and a couple times I missed important lines that caused us prod issues. I think we can do a better job than showing a wall of text.
Presentation is a separate matter though, just like with git diffs ideally you could choose a wall of text or a side by side ui, see things at a high level or drill down to line by line. A tag layer plus custom validation between plan/apply gives you an automatic way to short circuit things. But none of it can work without a plan as a first class object.
Thing is the plan/apply split isn’t even primarily for users necessarily, it’s just good design. It makes testing easier, and leaves open the possibility for plugging in totally different resolution strategies without rewriting the whole core. The benefits are so big there that I’d strongly prefer that more software is using it more often, even if I’m going to shut my eyes and auto apply every time without glancing at the plan.
MinGW is used to build Lua using cgo. I'd like to remove that part, see https://github.com/256lights/zb/issues/28 I haven't started the userspace for Windows yet (https://github.com/256lights/zb/issues/6), but I suspect that it will be more "download the Visual C++ compiler binary from this URL" than the Linux source bootstrap.
Yeah, I'm happy with live-bootstrap, too! I tried emulating Guix's bootstrap, but it depended a little too much on Scheme for me to use as-is. live-bootstrap has mostly worked out-of-the-box, which was a great validation test for this approach.
