
How do people manage hundreds of Git repositories? - Sevii
I have worked with several large companies using git as their version control system. One hosts several hundred git repositories in bitbucket stash, the other has around a hundred repositories in github. 
We recently ran into an issue with joda-time where upgrading past java8u60 caused apps to fail on startup. So my team is tasked with getting all of our microservices working with the latest jdk.<p>What tools are there to modify and commit similar changes across 100+ repositories at once?<p>I have found gitwalk https:&#x2F;&#x2F;www.npmjs.com&#x2F;package&#x2F;gitwalk which does a subset of what I want focused on github. But google has not revealed many other tools that can do discovery and transformations across 100s of repositories.
======
ericfrederich
This looks like a one-off thing you need to do semi-manually and methodically.

Every step of the way you need to balance the effort of doing something with a
script vs doing it manually.

How long would it take to manually make the change in all 100+ repos... vs.
how long would it take to write a script to do it. Would the script be robust
enough to properly make the change in like 90% of them... etc.

These are things you need to ask yourself in your endeavor.

You'd want to do all this work on a new similarly named branch (or branches
depending on your workflow) for every repo.

Perhaps write a script to validate changes to all 100+ repos on that branch.
Then when all the work is done... do a massive merge.

------
tedmiston
It probably depends on exactly what and how complex your “transformation” is.
You definitely can do lots of automation with $ git itself in terms of
committing, merging, opening PRs on GitHub via $ hub, etc.

Another idea is if you can refactor the common bits into a shared standard
library, for instance the design pattern is something like a system with a
bunch of plugins, then most of the plugin repos become very tiny. Ideally
you’d also have some sort of infrastructure automation around deployment.

~~~
Sevii
We have shared libraries and maven plugins setup. This is a situation where I
really wish we used a monorepo and I could just do one commit across 100
microservices using grep and other tools to update pom.xml files. I have read
whitepapers from google on how they support this kind of operation, but I need
it for many git repos.

~~~
ynezz
This sounds like StackOverflow question. Anyway, if you're proficient in Java,
why don't you write small app which will iterate over all Git repo dirs,
modifies the files, makes commit, push and creates PRs? Not a big deal, right?

~~~
Sevii
That is the current plan, combined with plenty of manual work verifying
jenkins jobs and deployments.

------
sethish
I haven't found anything off-the-shelf that helps. I manage a project with >
50k repositories in a GitHub organization. For the past few years I've been
developing software that helps with my [particular
goals]([https://github.com/gitenberg-
dev/gitberg](https://github.com/gitenberg-dev/gitberg)), but I'm afraid there
isn't much there that may help you.

------
tenken
Have you looked at a tool like:
[https://github.com/mixu/gr](https://github.com/mixu/gr)

~~~
Sevii
Going to look at it, thanks.

------
zeveb
It's too late for you, but I think that it's useful for other folks to see
your example and realise why multiple repos are a bad idea.

In the general case, each organisation should have a single repo containing
all of its code. There are times to violate this rule, but unless you're in
_extraordinary_ circumstances, more than a dozen repos in an organisation is
an extremely bad mistake.

~~~
allwein
Can you elaborate on this a bit? What are the advantages to keeping everything
in a single repo for the organization vs. a different repo per project?

~~~
zeveb
Basically, having multiple repos means that one person can change a dependency
of someone else's code without fixing the dependent code. This is unavoidable
with independent organisations, but is easily avoidable within an organisation
which uses unit & integration testing within a single repo (the use of a
single repo means that a suite of changes may be applied atomically).

In general, a process which allows one person in an organisation to stymie
another's progress is a broken system.

------
codeonfire
Grep + sed or replace in Files for the changes. And gitall to commit them all.
There is no need to get fancy if simple tools will do the job.

------
frou_dh
`mr` is good for keeping track of, and working with, many repos in bulk.

[http://myrepos.branchable.com/](http://myrepos.branchable.com/)

IIRC it has a good man page explaining more advanced use.

