Hacker News new | past | comments | ask | show | jobs | submit login

FWIW, during KDE's (one of the largest open source communities around, with about 2500 active developer accounts with direct write access to many millions of lines of code across dozens of products, and large numbers of external contributors) ongoing migration from SVN to Git, GitHub was never considered as an option because the community considers it unacceptable for an open source community to throw their weight behind a proprietary solution.

Instead we seriously considered the open source alternative Gitorious.org for a while, but ultimately were unable to come to an agreement with Shortcut AS with regard to a satisfactory hosting plan, and also had troubles making its ACL system work for our needs in some exploratory migrations to the platform.

Self-hosting the Gitorious software also fell out of the running fairly quickly due to the ACL problems and other concerns about the software's features and implementation; while KDE probably would have been able to put together a squad of coders to work on the software in its interest, Shortcut at the time had failed to get a dev community around Gitorious off the ground and was unacceptably slow at processing merge requests - meaning a fork might have become necessary, which is always an icky proposition not to mention a big burden on resources.

Ultimately we ended up banging together our own Git infrastructure out of several smaller components, some of which we wrote ourselves:

* To handle access controls as well as the developer-facing repo management knobs (personal server-side clones, personal scratch repos, access management for those, a trashcan for repos, the works) we chose gitolite, which was definitely the best choice we could have made. It's reliable, well-documented, fast and rich in features, and its principal developer Sitaram Chamarty has been an incredible aid to our efforts both with many hours of work spent coding additional features we needed and tons of advice. We did write a couple of add-ins to gitolite ourselves to provide some additional commands to developers, which we found pleasant to do.

* As web-based repository viewer we use an inhouse-modified version of Redmine. As with gitolite, we've found the Redmine crew to be easy to approach and get help from, and while we ran into some trouble with the scalability of its Git support, it proved easy enough to modify without requiring massive forking. Redmine's database of projects serves a vital role in the system in that the gitolite config is generated from Redmine's database (e.g user access roles), and we also generate a custom-format XML from it that is used by end-user clone/build tools, our translation infrastructure, LXR and many other bits and pieces that have a need to discover projects and repositories and their metadata (which can be modified by project managers in the Redmine web UI, e.g. setting description texts or translation branches).

* For review, we use ReviewBoard, continuing on from SVN. This is definitely the weak spot in our setup right now; ReviewBoard just doesn't support Git very well since it's designed to handle single patches rather than series of them and thus also can't track branches. We've been looking at Gerrit on and off to replace it, but so far can't really bring ourselves to commit to it since it's a fairly unwieldy beast.

* We wrote our own repository hooks in Python, which are in charge of things like sending out commit mails to mailing lists, CIA.vc, the bug tracker and people mentioned in commit messages behind special keywords, evaluating other keywords to close bug tickets or review requests, check for trivial errors (EOL style, license headers, that sort of thing), ref backups and a bunch of other tasks. Some of which run through asynchronous job queues as required to avoid blocking.

* We wrote our own mirroring system to provide several read-only git:// mirror servers behind a geo-ip DNS load balancer. The mirrors get updated whenever a push to the master happens.

* KDE has an LDAP-based single sign-on system in use at many of its web properties including Redmine and ReviewBoard, currently using an inhouse-modified (mostly expanded with our own modules) version of GOsa as web frontend. GOsa is also used to request developer privileges and manage the SSH keys for Git write access.

* There are a couple of other minor bits and pieces flying about, like a commits.kde.org/<repoid/<abbreviated or full sha1> web app written in Ruby+Sinatra that forwards to the Redmine page for the commit; those URLs pop up in various places, e.g. in a developer's terminal output after pushing, in mail headers, CIA.vc postings, etc.

KDE's migration to Git is still ongoing, with many large and actively developed codebases still remaining in SVN at this time, as well as the entire translation effort. To actually migrate things we use our own tool called svn2git which takes raw SVN repo data, an account conversion map (supplied as plain text generated from our LDAP) and a plain text file with regex-based rules as input and produces git repositories. Writing the necessary rules files is unfortunately a time-intensive task especially given some very complex project histories originally imported from CVS, but additional tools exist to partially auto-generate them by analyzing SVN history programmatically.

Here's a couple of links of interest:

* The developer-facing user manual for KDE's Git infra: http://community.kde.org/Sysadmin/GitKdeOrgManual

* The initial posting of the sysadmin working group's plan for the above setup, and our evaluation of this stack vs. the Gitorious software (ODF of the latter is attached to the first reply): http://mail.kde.org/pipermail/kde-scm-interest/2010-June/001...

* The XML export from Redmine mentioned above: https://projects.kde.org/kde_projects.xml




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: