
Scaling Mercurial at Facebook (2014) - based2
https://engineering.fb.com/core-data/scaling-mercurial-at-facebook/
======
tehlike
As an engineer that worked both at google and Facebook, I vastly prefer
googles monorepo on perforce. Combining that with citc was pretty solid way to
develop your project.

Hg is a bit of a nightmare in the wfh situation. Really slow, hangs for a long
time if you haven't synced in a few days. Yes, im sure there are ways to
tweak, but not sure if you can tweak them enough!

~~~
Lt_Riza_Hawkeye
Somehow FB has made all the common operations work in <6 seconds with millions
of files...

~~~
gravypod
It's an impressive feat but something I find even more impressive is the scale
they need to be prepared for. The performance challenges of maintaining
infrastructure for as many developers as Facebook/Google employ is constant
and demanding.

At my extremely small startup in the past ~6 months I've made about 1,000
files (@ HEAD, not including deletions). So, maybe, at the absolute worst case
each engineer could produce ~150 files/month. Facebook likely has ~1,000
engineers. That's a worst case 1,800,000 files/year + an existing massive code
base.

Good luck to the engineers who are forever battling the scaling issues
underlying these orgs! It's extremely impressive to even hear about overcoming
these huge hurdles.

~~~
tehlike
Facebook has way more than 1000 engineers :)

------
dang
If curious see also

2016
[https://news.ycombinator.com/item?id=11789182](https://news.ycombinator.com/item?id=11789182)

Discussed at the time:
[https://news.ycombinator.com/item?id=7019673](https://news.ycombinator.com/item?id=7019673)

------
based2
[https://github.com/facebookexperimental/eden](https://github.com/facebookexperimental/eden)
[https://news.ycombinator.com/item?id=23124095](https://news.ycombinator.com/item?id=23124095)

------
tikhonj
Note: this article is from 2014; it would be interesting to see how Mercurial
has worked for Facebook since then.

~~~
nuclear_eclipse
It's a _lot_ better than it was in 2014, but I still miss using git.

~~~
ForHackernews
Why? Git is such a worse, over-engineered experience.

Github won, so we're all forced to use git now, but mercurial is really a
simpler interface and a more straightforward mental model.

~~~
greggman3
I started on mercurial. I vastly prefer git. I admit it took a few months of
asking teammates to help me get it but i'd never go back.

~~~
throwdbaaway
Same here. I used to regard [http://jordi.inversethought.com/blog/i-hate-
git/](http://jordi.inversethought.com/blog/i-hate-git/) as my hero, and then I
had to start using git in a new gig, and never touched mercurial since. If the
goal of using a vcs is to have meaningful atomic commits, nothing beats git
staging area and interactive rebase.

------
jeffbee
Anyone know if they're still on Mercurial?

~~~
sulam
I can't say for sure, but I'd be incredibly surprised if they weren't. What
else would they be on? (Don't say Git, they tried to make Git scale for their
monorepo and it would have required a lot of changes that the Git community
wasn't interested in.)

~~~
yodon
Microsoft uses git for all its Windows development, which is a pretty big
scale (the repo was 300GB back in 2017)

~~~
jeffbee
Google's main repo was 86TB in January 2015.

~~~
ian-g
Is this 86TB of files at that time? Or 86TB of files cumulatively over time +
metadata?

------
setheron
Objectively great contributions to OSS and mercurial but I wonder how many
companies or projects will need to do something similar.

I enjoy the patches the make mercurial usable without the crazy extensions
that few of us will ever use.

------
qeternity
In 2014, a large site but not the behemoth it is today, how did Facebook
manage to clock in at 17 million LOC?? Genuinely curious for anyone with first
party experience.

------
rektide
Does anyone have any good videos about Facebook's use of watchman to ship
changes to central systems & be performing continuous test/analysis on code as
it is developed?

I feel like I saw a good video in 2017 or so but cannot find much these days.

------
samfisher83
Why not just have bunch of repos? You can use what google does with chrome and
use depot tools to checkout from a bunch of repos.

Edit: Why am I getting downvoted for asking a question?

We use multiple repos and we have a pretty large codebase consuming a TB a
day.

~~~
q3k
Because you lose the 'single head' concept of actually being able to always
have a clear view of the current newest version and of linear version history.
The ability to map a single revision number into the state of _all_ source
code (including third party dependencies) is extremely powerful. You also lose
the ability of performing sweeping changes across an entire codebase in
lockstep (think: backwards compatibility breaking API changes).

Not to mention that Google internally avoids multiple repositories (and
instead runs of a single monorepo) - it's just the android/chrome/public stuff
that's mostly split up.

~~~
erik_seaberg
Don't make breaking API changes, you can't deploy them atomically. You have to
support the old API until everyone has safely migrated and confirmed no
rollback will be needed.

~~~
fsociety
Then services would be stuck in a sea of technical debt. There are too many
moving pieces to do that.

~~~
erik_seaberg
Technical debt is a tool to be used (with caution). Getting ludicrously far
behind is a risk I'd avoid, but our stability is more important than staying
on the bleeding edge and testing every single version of each of our
dependencies.

------
jinwoo68
2014

------
azangru
POSTED ON JAN 7, 2014

