Obviously I'm not in their place, but in hindsight, my first thought when reading their initial issue was "Why not make a container that serves the minimal subset of Rubygems stuff necessary to bootstrap Rubygems, and use that as a source in the event that the main side died and needed to be kickstarted again.
Docker itself used a similar strategy for their build process: the recommended build process uses containers to build the Docker binary, so they made a precompiled binary available that was crafted "the hard way" that could then be used for running the saner build process.
The point that they want to track the versions of gems rubygems.org depends on in a VCS and track the dependency for each version of rubygems.org. You can build a container that includes those, but then you'd have to keep the container in a VCS - that's obviously possible but tangential to the problem.
Docker itself used a similar strategy for their build process: the recommended build process uses containers to build the Docker binary, so they made a precompiled binary available that was crafted "the hard way" that could then be used for running the saner build process.