Nice post. I think this is a really cool idea in general. A few of us on one of my projects have been thinking myself recently about a package manager based on git, where you could literally merge in feature branches or patch branches as you wanted...
While a nice idea, it would be worth finding out why github abandoned this very concept in favor of gemcutter.org before diving in head first. I trust the github guys as being far more competent than most in all matters of distributed source control especially when it comes to package management.
The idea is sound, but it was a distraction for us, so we were more than happy to offload rubygems to a dedicated service that could give it the attention it deserves.