We've been experimenting at my day-job with using the nix package manager to
automate the fetching and building of dependencies, and also isolate them per
project/revision.
In the root of each project repository, we have a shell.nix file that lists all
dependencies needed to build and develop the project. Most of them are from the
latest major nixpkgs tarball and we include its hash to account for the hashes
of every package that it provides. Other packages are from e.g. PyPI, crates.io
or internal repositories and each of those have their own hashes.
With the shell.nix file we can then get a working environment set up by running
nix-shell or automatically when we enter the directory by using direnv. The
first time it will fetch and build everything but it will be cached after that.
This has worked quite well for us so far, it makes it very convenient to
maintain dependencies. We can try out new dependencies on a feature branch and
easily run CI on it, and when we merge the branch to master the dependencies
follow along automatically for both local and CI machines.
For older revisions or branches it is also very convenient to get a working
environment as you just need to check it out and load its shell.nix file. No
need to make sure your system has e.g. an older Python version available, it
will be fetched automatically. In theory, older revisions should be usable
indefinitively, as long as the sources we fetch the dependencies from are still
available.
To try to ensure that all required dependencies are included in the shell.nix
file our CI jobs run with nix-shell --pure to make sure it does not use any
system programs. However, this does not prevent e.g. using files or binaries
specified by an absolute path, if it is available both locally and from CI.
Preventing unneeded dependencies from being listed is also difficult, as that
requires removing a dependency and re-running everything to make sure nothing
breaks without it.
In the root of each project repository, we have a shell.nix file that lists all dependencies needed to build and develop the project. Most of them are from the latest major nixpkgs tarball and we include its hash to account for the hashes of every package that it provides. Other packages are from e.g. PyPI, crates.io or internal repositories and each of those have their own hashes.
With the shell.nix file we can then get a working environment set up by running nix-shell or automatically when we enter the directory by using direnv. The first time it will fetch and build everything but it will be cached after that.
This has worked quite well for us so far, it makes it very convenient to maintain dependencies. We can try out new dependencies on a feature branch and easily run CI on it, and when we merge the branch to master the dependencies follow along automatically for both local and CI machines.
For older revisions or branches it is also very convenient to get a working environment as you just need to check it out and load its shell.nix file. No need to make sure your system has e.g. an older Python version available, it will be fetched automatically. In theory, older revisions should be usable indefinitively, as long as the sources we fetch the dependencies from are still available.
To try to ensure that all required dependencies are included in the shell.nix file our CI jobs run with nix-shell --pure to make sure it does not use any system programs. However, this does not prevent e.g. using files or binaries specified by an absolute path, if it is available both locally and from CI. Preventing unneeded dependencies from being listed is also difficult, as that requires removing a dependency and re-running everything to make sure nothing breaks without it.
reply