Hacker News new | past | comments | ask | show | jobs | submit login

They really don't, and it was funny that period where you'd see Dockerfiles with all the commands in a single invocation to avoid "bloating" the resulting image with unnecessary intermediate products that ended up deleted.

Maybe it's out there and I've just missed it, but I really wish there were richer ways to build up a container FS than just the usual layers approach with sets of invocations to get from one layer to the next, especially when it's common to see invocations that mean totally different things depending on when they're run (eg "apt update") and then special cache-busting steps like printing the current time.

I know part of this is just using a better package manager like nix, but I feel like on the docker side you could do interesting stuff like independently run steps A, B, and C against some base image X, and then create a container that's the FS of X+A+B+C, even though the original source containers were X+A, X+B, and X+C.

Not sure if you're aware since you already mentioned Nix, but Nixpkgs has a nice function to export "graph-layered" Docker images[0]. There is some overhead in the conversion, but the rest of the build can be parallelized by Nix as usual.

[0]: https://grahamc.com/blog/nix-and-layered-docker-images

Fantastic, this is exactly what I was looking for, thanks for the pointer!

It looks like the layer limitation comes from the underlying union filesystem stuff and not from anything inherent in Docker itself. I wonder if it would be possible to build a new filesystem driver for Docker that could serve up an infinite ecosystem of immutable nix packages without having to actually unpack them. Whether such a thing would actually be of value would probably depend a lot on the use case, but I could imagine a CI type scenario where you wanted to have a shared binary cache and individual nodes able to come up very fast without having to unpack the pkgs and prepare a full rootfs each time.

That's already possible to some extent by comining staged builds and the experimental buildkit engine which can run independent steps concurrently.

I’m still researching, but I got the impression that buildah from Redhat can do this.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact