
Guix-tox, a functional version of tox - civodul
http://blogs.rdoproject.org/7843/guix-tox-a-functional-version-of-tox
======
travjones
I honestly skimmed the article and didn't read it deeply because I have to run
in a bit. But the fact that GUIX is a language-agnostic package manager sounds
very neat. I write Go code and one of the things I had to cope with in the
beginning was not having a really good package management system. Go still
gets criticized for this (although we can always vendor our apps/packages to
make sure all dependencies are included). It seems as though GUIX could be
used as a package management solution in Go. It might be worthwhile for
someone to write a tutorial on GUIX + Go dev.

~~~
amirouche
IIUC Guix lakes "genuine" Go compiler but support GCC's Go.

It's not only Go that miss a "complete package management system". IIRC guix-
tox can replace both regular CI system and python-tox as test runner.

~~~
codemac
I'm working on the Go compiler support. A small hitch in cgo compilation and
then it'll be done. Have to work with the Guix maintainers to get the package
up to snuff as well.

------
liotier
Naming collision... [https://tox.chat](https://tox.chat)

~~~
dschep
AFAICT tox the python testing/virtualenv tool predates Tox the chat tool.

~~~
vangale
Yep, initial commit for tox is 2010. tox.chat is 2013.

------
calpaterson
This sounds great. For projects that use lxml/numpy/scipy/pandas this could be
a real help for getting set up on different machines especially if there is
some kind of caching that can be done for the built packages.

It's a shame it can't use all of PyPI though.

~~~
davexunit
>It's a shame it can't use all of PyPI though.

People often have this disappointment about Guix not being some sort of meta-
package manager that can use any package format right off-the-shelf, which I
think is caused by a misunderstanding of how the system works. Guix packages
describe the _precise_ dependency graph of any piece of software, all the way
down to the C compiler's compiler. A package on PyPI, or anywhere else for
that matter, does not. For example, Python eggs that build native extensions
need certain shared libraries and C header files available on the system
before they can be installed, but PyPI has no way of installing those things
or otherwise capturing that information in a precise way.

However, us Guix devs can, and should, make it easy for users to take the
metadata for a package available in another system such as PyPI and produce,
to the best of our ability, a Scheme snippet that gets one most of the way to
a working Guix package. We wrote the 'guix import' utility for this purpose.
Right now, it can generate Scheme snippets based on packages at PyPI, CPAN,
Hackage, ELPA, RubyGems, and CRAN. The user can take the output of the
importer, and add the (hopefully few) missing pieces, such as the C library a
Python native extension needs.

~~~
riffraff
If I import a new package, what is the effort required to submit the new
definition into the guix repo?

Is that also integrated with the guix command line tool? If not, why not? It
would be super neat :)

~~~
davexunit
Roughly:

\- Clone the git repo, build Guix, and run the build daemon

\- Add the Scheme code for the package recipe to the relevant module

\- Run 'guix build' to make sure it builds and runs

\- Run 'guix lint' to catch common style issues

\- Make a commit

\- Send patch file to our mailing list

Full details about contributing can be found here:
[http://www.gnu.org/software/guix/manual/html_node/Contributi...](http://www.gnu.org/software/guix/manual/html_node/Contributing.html#Contributing)

I welcome you to give packaging a shot. The IRC channel and mailing list are
full of nice people that can help you if you struggle with something or
encounter a bug.

------
panglott
Since each language has a language-specific but OS-independent package
manager, this is a language-independent but OS-dependent package manager?

~~~
derefr
I've never understood why there isn't a "meta-package manager" that gives you
one OS-and-platform-neutral CLI tool for invoking whatever the most relevant
package management tooling would be, and then polyfills in the differences.

At a base level, I want to just not have to care whether I need to "brew
install", "gem install", or "npm install -g" something: just give me one
namespace and put everything I need in it once. If the tool could do just
that, I'd be sold.

However, I can't see why there couldn't be a more-complex namespace-mapping
database, such that I could e.g. export an installed-above-baseline package-
list from Ubuntu and import it into OSX, or Windows+Cygwin, to get equivalent
effects. I wouldn't need the same _versions_ of anything, necessarily; just
the same commands and libraries.

~~~
davexunit
>I've never understood why there isn't a "meta-package manager"

Because it's a fool's errand. The resulting mess would be the lowest common
denominator of all the supported packaging tools. All of the various language-
specific package managers do not touch things outside of their domain, leaving
huge gaps to fill in. How would a meta-package manager know that installing
the pg Ruby gem required having the PostgreSQL shared library and header files
available at build time? What version of that library would it use? How would
you reconcile where each package manager installs things? Traditional system
package managers install to the global /usr directory. npm makes nested
directory structures anywhere. Brew installs to /usr/local. How would you deal
with the source vs. binary based package managers? Centralized vs.
decentralized? How about package managers that require root privileges vs.
unprivileged ones? What software bootstraps the entire system?

I could go on, but I hope you can see the futility. Even if you could somehow
reconcile all of these things, what did it get you? You still wouldn't know
the precise dependency graph of your software because most package managers
can't tell you that. You wouldn't get build reproducibility because most
package managers don't address it. You wouldn't get transactional upgrade and
rollback of software because most package managers do not use atomic
operations.

Guix can handle software written in any language, allows unprivileged package
management, allows you to create isolated environments of packages (like
virtualenv, bundler, etc.), deduplicates all packages system-wide via a
content-addressable storage system, uses atomic operations for transactional
upgrades and rollbacks, and more. A meta-package manager built on top the
plethora of today's awful package management tools couldn't easily achieve any
of these features.

~~~
derefr
> How would a meta-package manager know that installing the pg Ruby gem
> required having the PostgreSQL shared library and header files available at
> build time?

Because that dependency is in the meta-package's manifest. Realize that I'm
talking here about an entire ecosystem of "meta-package formulae" that contain
instructions to make particular tool-calls on particular systems.

> What version of that library would it use? How would you reconcile where
> each package manager installs things? Traditional system package managers
> install to the global /usr directory. npm makes nested directory structures
> anywhere. Brew installs to /usr/local. How would you deal with the source
> vs. binary based package managers? Centralized vs. decentralized?

You wouldn't reconcile any of these things, any more than things are
reconciled now when you "brew install" one thing and "gem install" another and
"npm install -g" a third. The result is a heterogenous multi-runtime system,
but it continues to work, because _each tool_ knows how to manage its own
ecosystem. A meta-package manager would call through to the regular package
managers as its API for each ecosystem.

> How about package managers that require root privileges vs. unprivileged
> ones?

The meta-package manager could assume the existence of a self-elevating
wrapper tool like Debian's wajig(1), and provide such shims for package
managers that don't have one.

> What software bootstraps the entire system?

The regular OS package manager.

\---

I don't think I'm being clear here; the point wouldn't be to _replace_ the
system package manager. The point is to _automate the use of_ all the various
package-management tooling _for configuration management_ in a unified,
declarative manner.

I would expect the meta-package for e.g. Postgres to look a lot like a _Chef
cookbook_ for installing Postgres, but with the limitation that—where a Chef
cookbook would first attempt to use a package manager, and then "fall back to"
mutating the system itself in a way no tooling on the system is tracking—a
meta-package would instead just fail the installation if there was no package-
manager source satisfying its constraints.

Guix is great, but it is its own OS with its own ecosystem. The point here
would be to unify the API to package management on the ecosystems that
_already_ exist, such that you could interact with the package manager in
ignorance of its host OS the same way you can currently interact with e.g. a
Docker daemon while ignorant of its host OS.

If this tooling was available and baked into server VM images and so forth,
you could, for example, mostly replace the "cloudinit" instance-initialization
system with a simple package list; or create something like a "virtual AMI"
that takes a parent AMI ID and a "[{package, version constraint}]" list, and
derives a new child image to launch on a set update schedule.

~~~
lfam
> Guix is great, but it is its own OS with its own ecosystem. The point here
> would be to unify the API to package management on the ecosystems that
> already exist, such that you could interact with the package manager in
> ignorance of its host OS the same way you can currently interact with e.g. a
> Docker daemon while ignorant of its host OS.

I think there has been a misunderstanding. Guix is a standalone OS-independent
(Linux only) package manager. I am using it on Debian right now. The software
installed by Debian has no knowledge of the Guix software and vice versa.

GuixSD is the OS built around Guix.

~~~
panglott
Wow, it looks like GuixSD has made a lot of progress since the last time I
looked into it.
[https://www.gnu.org/software/guix/](https://www.gnu.org/software/guix/)

------
wyldfire
I haven't encountered these problems with tox. But I'm all for having more
explicit environment definitions while testing.

------
sschueller
For a moment I though this was about [https://tox.chat/](https://tox.chat/)

~~~
davexunit
If the tox.chat devs would just make some alpha release tarballs I would
package toxcore for Guix.

~~~
drdaeman
Gentoo and Arch packaging pulls source from git. Can Guix packaging be done
from a VCS checkout?

~~~
davexunit
Yes, and I've packaged that them that way for my own personal use, but it's
really annoying to deal with because I have no idea which clients work with
which revisions of the toxcore git repo. Some official alpha releases would
make this much more manageable.

~~~
robertely
I try to take it easy on them. They had some pretty serious setbacks and
drama([https://blog.tox.chat/2015/07/current-
situation-3/](https://blog.tox.chat/2015/07/current-situation-3/)).

They really are getting back on track this month with irungentoo working full
time.

