
Ruby on Guix - rekado
http://dthompson.us/ruby-on-guix.html
======
jfaucett
I completely see the benefits of using guix, transactional package management
seems at the very least an interesting alternative to docker in some
scenarios, and could be very interesting from a tool automation perspective.

I don't want to sound negative, but I think the biggest problem with getting
me or other ruby developers to package our gems to guix would be the
following. The bundler/rbenv system I've been using for years is pretty darn
good, personally I haven't had problems with it. I've used it in massive and
small projects, and admittedly the only inconvenience is having to manually or
through orchestration install the dependencies of certain gems, of which there
are also only a few (mysql2 with libmysqlclient-dev and his nokogiri mention
are probably the most notable). He does point this out but if you're using
docker or some automation tool this is a pretty minor point.

Also I already ensure my gems run their tests on mri, jruby, rubinius, etc.
and this seems like extra work for little benefit, especially when most in
house deployment scenarios are running with docker or chef/puppet on top of
the classic rbenv/rvm and bundler setup. My biggest question is after
upgrading a gem version how easy is it for me to push the new version of the
package to guix?

So essentially, I wish this article had more arguments to show why guix will
better the life of ruby devs.

~~~
zimbatm
I don't know if it's the same with Guix but in NixOS it's just a matter of
running `bundix` and all the Gemfile.lock dependencies are converted to a nix
expression. Gem developers can keep developing as usual.

Maybe the only restriction is to make sure the gem has configurable paths if
it tries to access system files.

I'm experimenting with it at the moment and converted my blog to be managed by
nix on OSX. It's not a walk in the park but it's working. Clone the repo (
[https://github.com/zimbatm/zimbatm.com](https://github.com/zimbatm/zimbatm.com)
) and type `nix-shell` (needs to be installed first) and you should get all
the blog dependencies available in your environment.

~~~
cstrahan
> I don't know if it's the same with Guix but in NixOS it's just a matter of
> running `bundix` and all the Gemfile.lock dependencies are converted to a
> nix expression.

I'm glad to hear that the infrastructure is working for you. There are,
admittedly, some kinks on master, though I have a PR to make things much, much
more robust:

[https://github.com/NixOS/nixpkgs/pull/8612](https://github.com/NixOS/nixpkgs/pull/8612)

Unfortunately, while Bundler is generally a great, useful piece of software,
there are aspects of its design that make it very difficult - borderline
antithetical - to cooperate with packaging; if you look at the commit message
in the above PR, you can see a thorough description of the challenges.

I have an open PR for Bundler to try to make things a bit more amenable, and
recently got the thumbs up with respect to the general idea:

[https://github.com/bundler/bundler/pull/3775](https://github.com/bundler/bundler/pull/3775)

Unfortunately, most Ruby deployments are quite imperative in nature - just
push to a git repository, bundle install, and then bundle exec (that's leaving
out all of the subtle prereqs: `apt-get install ...`, etc), as with
Capistrano. There doesn't appear to be as much of a culture around packaging
(yet), and so there hasn't been a demand for the tools to meet those needs.

I hope that we can push through those problems and make it trivial for people
to enjoy the ease/dependability/resiliency of package management, as many
people have no idea what they're missing out on.

~~~
zimbatm
Nice ! That should solve the issue where all gems are re-installed whenever
the Gemfile.lock is changed.

I agree that bundler is mainly focusing on the user interactions and is good
at that but never look inside. I'm not even sure if the dependency resolution
mechanism can be used as a library.

In my opinion bundler should only be used as a dependency-resolution manager.
Once gems are installed then they should use good-ol' $LOAD_PATH.
Unfortunately there are the git gems and also rails which insists on using
bundler.

EDIT: got any thought on adding system dependencies per gem ?

------
ownedthx
As someone who deploys Ruby software to our servers, I've come up with a
formula that's served us well.

We take all gems associated with the app (like a Rails app, but we do it with
others), and stuff them along with the code into a package created by fpm.

In other words, the packages are completely self-contained; no gems are used
from the environment.

The key to this is the --path option to bundle install, i.e.:

bundle install --path vendor/bundle

This command makes sure all my gems are in a sub directory of my project
called vendor/bundle. When creating the debian package using fpm which
includes all the source code, vendor/bundle also comes along with the package.

After installing the app on our Linux servers, 'bundle exec rails server' uses
the gems from vendor/bundle.

This approach does require that I create the debian package on a machine that
closely mirrors the deployed servers, so that compiled gems make any sense.

Most all the issues described in the article have not been problems because of
this approach. I used to use rvm as part of my build process; it was very
brittle. I'm completely happy now; just wanted to share in case that helps
someone else.

~~~
davexunit
>We take all gems associated with the app (like a Rails app, but we do it with
others), and stuff them along with the code into a package created by fpm.

That is exactly what I've started doing at work, actually. It's a short-term
win and rather simple, but it's a pretty big hack. Package bundling is quite
poor form as it leads to file system duplication and for each duplication you
have an additional location to patch when a security vulnerability is
inevitably discovered. Ideally each Ruby gem would be its own package. I'm
also concerned with reproducibility, so I build packages in as isolated of an
environment as possible where I run 'bundle install' with a fresh
vendor/bundle directory. Builds take longer than they should because of all
the re-downloading. Using Guix, where I can easily represent each Ruby
component in a package object, I can take advantage of its content-addressable
storage to save time on repeated builds by using the cached gem builds from
previous times.

~~~
ownedthx
Valid points.

We use a 'monorepo', so everytime we update any gem dependency, all of our
apps get rebuilt entirely. So, updating all of our packages or not isn't a
problem.

And I like the duplication, believe it or not. After having debugged
environmental problems with dependencies in various languages over the years,
I'm happiest knowing the dependencies in question are in vendor/bundle; end of
story. Sometimes, in case of a nasty bug, I have a one-liner fix and I can go
to vender/bundle, tweak the gem, and know I only affected the app using it.
Those are things I prefer over some lost hard drive space.

Build times are a little long for us, though. That is true. I do a fresh
bundle install on every build. That is the second longest part of the build,
behind running our dog slow web tests.

~~~
davexunit
The duplication is less of an issue in the corporate, one application per VM
environment. However, my focus isn't solely on that. Web applications are
_really_ hard for most people to self-host. They are made a bit easier by
things like OmniBus packages that bundle absolutely everything, which makes
the user dependent on each application author to ship security fixes to
software they didn't write.

There's also the issue of stateful package managers that can break in the
middle of a package install and you're screwed. Most of the time that doesn't
happen, but I've been bitten before. So the fpm approach is a great short-term
win and I want to roll it out to the production systems at work soon, but in
the long term I think we need functional package management to make our
systems more resilient to failure.

------
buren
Looks pretty cool, though it is a pretty daunting task to convince all gem
developers to test/make their gems compatible.

Perhaps this could make [http://devblog.avdi.org/2015/08/11/what-its-like-to-
come-bac...](http://devblog.avdi.org/2015/08/11/what-its-like-to-come-back-to-
a-ruby-project-after-6-months/) easier ;)

~~~
davexunit
I don't expect to convince all gem developers by any means, but it doesn't
hurt to ask nicely if the brave few would give it a try. ;)

~~~
buren
I certainly will :)

------
3pt14159
What is exactly wrong with piping curl into bash? I'm trusting RVM _anyway_.
As for functional package managers, I'll believe it when I see it. The reason
things are brittle is that any time something in the middle of a dependency
chain changes the whole chain may break. The more you freeze things, the
greater chance there is to freeze up a security hole (see what is going on
with Docker). So you have two choices: be ok with the freeze, or update the
whole chain. In this respect I prefer the latter, and I especially prefer the
way Ruby does it where, due to the global namespace, I'm forced to upgrade the
gem _everywhere_ so I end up fixing things and keeping the whole project up to
date. While this seems frustrating at first, once you realize that everyone in
the Ruby community is doing it, you realize that the reason it is usually
painless to update your gems is because other people have fixed those other
holes. What is especially nice is that you only have to remember things that
are broken at T0, unlike, say, Node, where a popular nested node_module may
have 10 different versions in your dependencies and you keep hitting the same
bug or security hole over and over again.

~~~
burke
I've found all the hate surrounding this practice to be pretty misdirected. As
long as it's served over https, it's functionally equivalent to "download and
run this random package installer!", which is generally tolerated.

~~~
davexunit
For the record, I do not tolerate "download and run this random package
installer!" either, without GPG signed binaries and corresponding source.

~~~
heroic
I am quite sure RVM now offers a signed download too.

[https://rvm.io/rvm/security](https://rvm.io/rvm/security)

------
1_player
Can guix be used somewhat like virtualenv?

I'm not sure I want to install guix on my main Archlinux box, as I don't want
conflicts with the existing libraries and applications, but I would very much
play with it if it were possible to do something like this:

    
    
        guix virtualenv myapp
        cd myapp
        guix activate
        guix install postgres # installs a local postgres installation available only on this "environment"
        guix environment -l myapp.scm # install app dependencies
        <do work>
        guix deactivate
    

This would be a killer feature for me and I would ditch virtualenv, pip, npm,
perhaps Docker in a heartbeat.

~~~
davexunit
Yes, the 'guix environment' tool is like a language-agnostic virtualenv. Guix
does not use or interfere with any components from the host distro, so you can
install Guix on top of Arch Linux without fear. We have a binary installation
method[0] that isn't too difficult that you can try out. I use Guix on top of
an Ubuntu host at work.

For your PostgreSQL example, you'd still need to do things like initialize the
DB cluster and start the daemon on your own (the GuixSD distro has a system
service for this[1]), but to spawn a shell with the postgres daemon and client
programs available you would run:

    
    
        guix environment --ad-hoc postgresql
    

Exiting the sub-shell would do essentially what you had in mind with 'guix
deactivate': Make postgresql available for garbage collection (via 'guix gc')
if/when nothing else is using it.

[0] [https://gnu.org/software/guix/manual/html_node/Binary-
Instal...](https://gnu.org/software/guix/manual/html_node/Binary-
Installation.html#Binary-Installation)

[1] [https://gnu.org/software/guix/manual/html_node/Database-
Serv...](https://gnu.org/software/guix/manual/html_node/Database-
Services.html#Database-Services)

~~~
1_player
The point was to install the deps into my project tree, instead of putting
everything into /gnu, but to think of it that's not actually much of a
problem, it's just personal preference/bikeshedding.

I'm installing it and see how it goes, thanks.

~~~
rekado
"Installing" a package with Guix really just means that it is referenced
through a link. /gnu is just a cache for "evaluated" package recipes. You
could create a profile that contains your symlinks to the items in the store
wherever you want, including the currenty directory.

    
    
        guix package -p $(pwd)/.guix-profile -i what ever you want
    

Programmes in this profile can then be executed like so:

    
    
        ./.guix-profile/bin/what
    

But that's not necessary when you could just use `guix environment` and enter
a shell where everything is already hooked up.

------
bro-stick
Notice the striking similarities of package management definition files and
configuration management, where the latter does more related to state,
coordination, attributes and templating files. Packages are just more generic
containers of resources and metadata, configuration management makes those
resources concrete on specific systems and applies tweaks to converge on the
desired catalog state.

Btw, has anyone written a usable Guile-based cfg mgmt tool yet?

~~~
davexunit
>Btw, has anyone written a usable Guile-based cfg mgmt tool yet?

That would be Guix. We use the same primitive utilities for package management
and system configuration management. I'm typing this from a laptop running the
GuixSD distro, and if a system upgrade were to break things, I can roll-back
to a previous, working generation of the system at boot time and keep on
going.

~~~
bro-stick
That make a ton of sense because the boundaries between packaging and cfg
management are artificial.

------
waxjar
I like the approach of using gs (simple gemsets) [1] and dep (simple
dependencies) [2]. I've been using this approach of handling dependencies for
a while now.

I've written a simple shell script as a replacement for gs (since I constantly
forget if I was in a sub-shell or not). All it does is change $GEM_PATH and
$GEM_HOME when $PWD changes (while staying in the same shell). If the current
$PWD has a directory ".gs", `gem install` will install them there instead of
globally. All I have to remember is to create a .gs directory when I start a
new project.

It does not address all issues highlighted in the article, nor do I know how
well this would work in a production environment with automated deployments.
It's probably more of a work around than a solution. I vastly prefer it for
personal projects, though. Hate solving gem conflicts with "bundle exec".

[1]: [https://github.com/soveran/gs](https://github.com/soveran/gs) [2]:
[https://github.com/cyx/dep](https://github.com/cyx/dep)

~~~
regularfry
Instead of changing environment variables when $PWD changes, I went the other
way: make it obvious when you're in a subshell by strapping $VIRTUAL_ENV onto
$PS1. Again, simple shell scripts win[0][1].

I'm still interested in a cross-language approach, but I'm suspicious of
anything which requires an explicit import step for rubygems that doesn't
involve resolving the dependency tree. Unless Gemfile.lock is involved, it's
going to need to copy Bundler's resolution algorithm, which seems fraught.

[0]:
[https://github.com/regularfry/gemsh](https://github.com/regularfry/gemsh)
[1]: [https://github.com/regularfry/rv](https://github.com/regularfry/rv)

~~~
davexunit
>but I'm suspicious of anything which requires an explicit import step for
rubygems that doesn't involve resolving the dependency tree.

The current 'guix import gem' utility doesn't do dependency resolution at all.
It just generates some Scheme code for you to start with and you fill in the
gaps. It's important to understand that Guix packages specify the _exact_
versions of its dependencies. That includes not only the version number, but
its explicit dependencies, the Ruby used during the build, the GCC used for
native extensions, the full build script, etc. All of these various inputs to
the build function are reflected in the SHA256 hash that uniquely identifies
that package.

The loose versioning that Bundler uses is indeed very flawed and prone to
strange errors. Guix simply doesn't have this issue.

------
lighthawk
Nothing about rbenv in the post. Any reason why not? Some of the things you
pointed out about rvm would be handled by using rbenv instead.

~~~
davexunit
I should have made a note that I've been using rbenv as an escape from rvm and
it has roughly the same set of problems, but it is a bit less enraging.

~~~
pdkl95
As someone who tried RVM for a while and until it's problems became obvious,
and lived (very successfully) with rbenv as an escape for a long time
afterwords, I have been liking chruby:

[https://github.com/postmodern/chruby](https://github.com/postmodern/chruby)

It doesn't have most of the problems of RVM/rbenv, and mainly provides an easy
way to switch between rubies while leaving everything else as full-path
references instead of the "magic" tht RVM tries to provide.

That said, I really like the local $GEM_HOME method, and guix sounds like an
interesting solution.

~~~
davexunit
Thank you. I think the benefits of Guix start to become even more evident when
you work in a multi-language environment with god knows how many package
managers. We don't yet have all the packages that people need, but that will
happen in time if other people see value in Guix. We will see.

------
frontsideair
I'm starting to think that I should go back to GoboLinux from Ubuntu.

------
roller
I imagine everything said in the article also applies to the Nix package
manager[0]. One of its nice features is the Nix expression language (aka Nix)
[1]. It has an elegant mixture of purity and pragmatism. I imagine guile
scheme would have a lot more integration opportunities outside of just
system/build/configuration management. It'd be interesting to see Guix
expressions that actually mix other scheme libraries or concepts, rather than
just taking Nix expressions and removing the syntax.

[0]: [http://nixos.org/nix/](http://nixos.org/nix/) [1]:
[http://nixos.org/nix/manual/#chap-writing-nix-
expressions](http://nixos.org/nix/manual/#chap-writing-nix-expressions)

~~~
rekado
I very much like how Guix is just a Guile Scheme library and can itself be
integrated with other Scheme code. There is, for example, guix-web, a web
interface to the package manager; there also is an Emacs interface using
geiser.

Using Guile has turned out to be a great feature in itself and it is still
fueling the development of new features that otherwise would be awkward to
implement, such as the new "guix graph" command to visualise dependencies.

------
davidroetzel
> I'm not sure how you feel, dear reader, but my Ruby environments feel like
> one giant, brittle hack

Of course things are not perfect (and guix may have a few cool ideas), but I
think we should keep in mind that both rvm and bundler have been huge
improvements.

The hacks people needed to come up with before rvm and bundler existed were an
order of a magnitude more brittle.

------
transfire
Some of these issues can be resolved using Ruby Install
[https://github.com/postmodern/ruby-
install](https://github.com/postmodern/ruby-install)

~~~
rokgarbas
i think guix/nix solution (can) goes beyond ruby platform and you can define
dependencies on non ruby components: postresql, mysql, mongodb, ...

------
schneems
I really hope no one is ever doing `rails >= 4.0` in their Gemfile.

~~~
sanderjd
Yeah, while I think the approach outlined in the post makes sense and could be
really great, I really haven't had any of the problems outlined since the bad
old `config.gem` days. In my experience, everyone defaults `~> X.Y.Z`
dependencies, with more lenient patterns justified by actual testing, and I
can't remember the last time I saw a blanket `>= X` requirement. This has
meant that all, or very nearly so, of the dependency problems I find myself
handling are due to actual incompatibility, which I'm glad were caught.

------
paublyrne
NB, this is a post by David Thomas, not Dave Thomas, the longtime and well
known Rubyist.

~~~
davexunit
Thompson, actually. I am not the Wendy's guy. :)

~~~
paublyrne
Oops, sorry :)

