
Buck – A build system developed and used by Facebook - Nimsical
https://buckbuild.com/
======
danso
I don't work in a shop where performance/speed is important, but I am looking
for other ways to do things I would do in Make but...not in Make.

For example, my use-case is similar to what Mike Bostock described in "Why Use
Make" [0] when explaining how he uses Make to build out his data tranformation
process. Most of my work is data transformation/small-scale ETL, but I just
haven't been able to get into Make beyond trivial work, and I often end up
writing things in Rake (Ruby).

So I was wondering if other devs had tried using Buck/Bazel for everyday
hobbies and projects, and whether you stuck with the new tool or went back to
Make? The portability of Makefiles isn't a high priority for me, and I like
experimenting with different systems for my own projects.

[0] [https://bost.ocks.org/mike/make/](https://bost.ocks.org/mike/make/)

~~~
zuzun
These build systems become super handy once you're working on a large tree of
many different software projects. For small projects not so much. It's
probably easier to use the default build system of your programming language.

When make fails me for personal stuff, I switch to ninja [1]. The Chromium
people use it in their Bazel-like build system. A hidden gem I recently
discovered is doit [2]. I found this one incredibly helpful when I had a build
with tons of 1:N, N:1, N:M dependencies.

[1] [https://github.com/ninja-build/ninja](https://github.com/ninja-
build/ninja) [2] [http://pydoit.org/](http://pydoit.org/)

~~~
danso
Thanks for the suggestions! That "doit" looks like it's simple enough for my
fairly pedestrian needs. I wonder if my aversion to Make is that I don't spend
enough time thinking/making Makefiles, but having something a little
friendlier and simpler -- at the cost of ubiquity/portability -- might be the
kind of training wheels I need.

Edit: FWIW, I posted doit to r/python and someone suggestion Snakemake, which
also looks excellent and well-maintained:
[http://snakemake.readthedocs.io/en/stable/](http://snakemake.readthedocs.io/en/stable/)

------
lacker
Buck has been open source for a while! It's especially popular among larger
companies who have a ton of mobile developers contributing to their apps, but
I'm not sure if FB has ever publicized who exactly is using Buck. Other
companies have been contributing to the Buck ecosystem, too, though, like
[https://github.com/uber/okbuck](https://github.com/uber/okbuck)

~~~
LegNeato
It looks like they are starting to put together a list of who is using it:

[https://buckbuild.com/about/showcase.html](https://buckbuild.com/about/showcase.html)

------
spankalee
This looks like a clone of Google's Blaze/Bazel:
[https://bazel.build/](https://bazel.build/)

~~~
Xyz9292929
Buck was started after former google employees at facebook wanted to use
something like blaze (ended up being called bazel). Kinda like a lot of other
things they copied from google at facebook. Dremel -> presto, etc.

~~~
bubersson
What are some other things apart from dremel?

------
tboyd47
> "Buck is a build system developed and used by Facebook."

I have really, really grown to resent this culture of proud and unabashed
cargo-culting that we've arrived at in the open source world. Why is this the
first sentence describing a new project? Why do we need a Facebook™-approved
build system? Does that somehow make it better than the others? And why does
Facebook need their own build system? Was the existing ecosystem technically
insufficient for them, or was the issue a legal one?

Whenever I make this point in dev circles, someone will reply, "They serve X
amount of visitors a day, so they must know something!" Well, they also have a
firehose of ad money pointed at them 24/7.

~~~
gregmac
It does actually make a difference. Contrast:

> "x is a build system developed by me as a side-project that I might drop at
> any time, and its only production use is to compile itself and a hello world
> app"

Not to say people's side projects aren't useful, but there's only so many
hours in a day, and there are thousands and thousands of open source projects,
so you need some way to evaluate what's worth looking at. Knowing it has more
than one person behind it, some time in field (so it's not 0.0.1 alpha), and
is actively being used (you're not writing the first production code using it)
goes a long way to separating it from the pack.

~~~
westerlee
This does not logically follow. Why does facebook know best what kind of build
system is suitable for my needs, given that I'm probably nothing like
facebook? Why does George Foreman know how I want my grill?

I understand the human need to optimize attention, use basal heuristics to
weed out unattractive options, and nurture a need to belong, but this is not
logic. It's a rationalization.

It's also why this works, and why facebook (and every other major tech player)
does it. It's a brilliant marketing and recruiting tool and a way to insert
influence under the guise of open benevolence.

Hidden out there is probably a better tool for your needs. But now you'll
never find it.

~~~
panopticon
> I understand the human need to optimize attention, use basal heuristics to
> weed out unattractive options, and nurture a need to belong, but this is not
> logic. It's a rationalization.

There is an underlying logic though when you frame it in as a time vs reward
problem. I _could_ spend a few months evaluating all of the various build
tools out there and find the perfect fit for my team. Or I could spend a few
days evaluating the projects with the most support (be that large companies,
large communities, whatever). I'll concede that I can't guarantee it's the
_perfect_ fit, but it's likely good enough when you compare the opportunity
costs.

------
beefsack
Continuing the trend of Facebook creating competing tools more often than
persisting with and improving existing ones, which I feel dilutes effort and
has fragmented a number of ecosystems.

We've got a couple of Facebook fans at work, which has left us with a number
of our systems using different tools to accomplish essentially the same tasks,
and no strong case on either side for us to standardise on one of them.

Maybe I'm just being bitter because I've had a bad experience, but it's been a
maintenance nightmare for us in the office.

~~~
LegNeato
When Buck was created (my team created it!) there wasn't anything that
supported what Facebook needed in a build tool. There still really isn't as
all the Blaze work-a-likes are focusing on various different needs.

I would argue Buck actually helps to _solve_ what you are concerned about. At
Facebook, everything builds with Buck (and Google with Bazel I hear)...which
means you learn it once and you know it for your Objective-C library, your
Android app, your Cxx service, your python scripts, etc. It really helps to
standardize on a build system, and at many companies you can only do that if
it supports windows/mac/linux, it supports many languages and platforms, is
fast, and is easy to pick up. Buck is all of those things.

~~~
fs111
> At Facebook, everything builds with Buck

Looking at [https://github.com/facebook](https://github.com/facebook) you seem
to be using all sorts of build tools...

~~~
sdwilsh
What Facebook uses internally in their monorepo and what they publish to open
source for external consumption doesn't have to be the same build tool.

Facebook uses fbshipit
([https://github.com/facebook/fbshipit/](https://github.com/facebook/fbshipit/))
to take slices of their monorepo and publish them to GitHub.

------
bolinfest
My name is Michael Bolin and I created Buck.

When I started the project, Buck had one specific goal: to make Android builds
faster ([https://youtu.be/CdNw6mRpsDI](https://youtu.be/CdNw6mRpsDI)). At the
time, the recommended way of building Android (from Google) was to use Ant. So
when someone points to Buck as an example of "creating competing tools more
often than persisting with and improving existing ones," I'd like to point out
that you can't fix Ant if these are your issues with Ant:

* It is unsound. * Because it is unsound, it is irreparably slow. * It uses XML as a build language.

Yes, in July 2012, there were a number build systems on the market (though
Bazel was not one of them, but Pants was), and none of them focused on
building Android. And even if they did, few (if any) software companies were
building an Android app as large as Facebook, so it was unlikely that anyone
else was going to design for our scale.

It also wasn't just about build times, but about how I wanted to see us
organize code in our repository. At the time, there was a flat list of folders
in the Android repo, each called lib-something. This drives me insane because
you inevitably end up with two (or more!) people creating
com.facebook.common.StringUtils, each in their own lib-something. (It's also
annoying to `ls` this "lib-" directory over time.)

In contrast, Buck/Bazel encourage the use of a unified tree, but still
encourage fine-grained modularization (which is key as your build graph gets
very large). This has been shown to scale to extremely large monorepos at both
Facebook and Google.

Finally, by having total control of the build system, we were able to build in
all sorts of cool tricks to build Android very fast, both in the large and in
the small: [https://youtu.be/Y9MfGS3qfoM](https://youtu.be/Y9MfGS3qfoM). I
don't think there is any other build system we could have decided to work with
at the time to achieve these gains.

Buck has since evolved to build everything else at Facebook. This is not
because the Buck team set out to conquer the world, but because people
internally wanted the benefits of Buck for their builds. Building an
alternative toolchain to xcodebuild was a mammoth effort (and one for which I
take no credit). Having one build language for a heterogeneous collection of
programming languages in a monorepo is no small feat.

Finally, to the people who believe "The big guys want developers locked into
their ecosystems," I have news for you: the Buck team is not offended if you
use Bazel, Gradle, Make, or anything else. Buck is open source because we
wanted to share it with the community, not dominate it. Like many of you,
people are excited to show their work and learn from others.

~~~
emelski
I think when you say "make Android builds faster", you mean "make Android
application builds faster" \-- as opposed to making Android operating system
builds faster. Those are two very different things, and for the uninitiated
the casual use of language here is confusing. The Android operating system has
never been built with ant, but historically was built with make until around
Android N when that team started migrating to ninja-based builds instead.

~~~
vorg
I got confused by the exact same language 5 years ago when one of the Apache
Groovy project managers (before it joined the ASF) started repeatedly saying
"Google have now chosen Groovy and Gradle for building Android." I didn't know
if they meant building Android at Google, or as the default build system
shipping with their (then) new Android Studio tool.

------
dyu-
Buck is a great tool but doesn't work on windows, where bazel is now starting
to support. So for crossplatform builds, either GN or bazel.

~~~
sdwilsh
The only thing that I'm aware of these days that doesn't work on Windows is
C++ code (but it works if you are building for Android on Windows). It's even
covered in the getting started guide:
[https://buckbuild.com/setup/getting_started.html](https://buckbuild.com/setup/getting_started.html)

~~~
dyu-
Yea that's what I meant for crossplatform builds when I compared it with GN
(c/c++ only, production-ready) and bazel (multilang, unstable).

Are you a member of the buck team? Your new account only has activity on this
submission.

------
MichaelMoser123
Wow, so many build systems got mentioned here, when do people have the time to
check them all out? I stick with gnu make because that's the evil I know, not
because its the best tool imaginable...

~~~
fredley
Not everyone here is an expert in a particular system yet. If a build system
runs 20% faster than what you're currently using, and you plan on using it for
a good few years, the overall time saved is not insubstantial.

------
sly010
Or just use Nix as your build system.

~~~
thinkpad20
Surprised no one else has mentioned nix, because this seems very much inspired
by it. As someone who uses nix extensively this is interesting but doesn't
seem as powerful or general.

------
kkapelon
Can somebody compare this to Gradle?

I mean somebody who has actually used both of them (and not read some versus
blog posts).

~~~
LegNeato
I think the best people to speak to this would be the Uber folks, as I believe
they still use both via okbuck
([https://github.com/uber/okbuck](https://github.com/uber/okbuck)):

[https://eng.uber.com/ios-monorepo/](https://eng.uber.com/ios-monorepo/)

------
samfisher83
I would recommend gn and ninja. It's the chromium build system. The make file
is generated in less than a minute with gn, and ninja does a good job with
incremental builds. It's also been around for a few years so it's been proven.

------
ww520
Is there a watch command to detect file change and kick off the build steps?

~~~
LegNeato
Buck has a daemon that does it automatically. Often when you go to build it
takes 0 seconds because the daemon has already done it before you get to it:

[https://buckbuild.com/command/buckd.html](https://buckbuild.com/command/buckd.html)

Note the daemon is automatically started when you `buck build`.

(disclosure: my team created Buck when I was at FB)

~~~
sdwilsh
Although the daemon doesn't automatically build things. Buck does use watchman
([https://github.com/facebook/watchman](https://github.com/facebook/watchman))
to watch files though, and it has a way to run commands when files change:
[https://facebook.github.io/watchman/docs/watchman-
make.html](https://facebook.github.io/watchman/docs/watchman-make.html)

------
cafebeen
Tools like Make are also useful for simple data analysis workflows, and I'm
curious to hear any thoughts from any Buck users as to whether it would be
useful in those cases too.

------
waruqi
Great! There's also a Lua-based build tool.

[https://news.ycombinator.com/item?id=13867347](https://news.ycombinator.com/item?id=13867347)

------
GingerBoats
Just what we need.... another build tool!

------
nat
I sometimes suspect that the build tool ecosystem would be very different if
only Makefiles allowed soft tabs.

------
zjfroot
Wonder if this can/will replace CMake for cross platform builds.

~~~
dyu-
Doesn't support windows. You're better off with bazel or GN.

~~~
revelation
How do you in 2017 come up with a build system that doesn't even run on
Windows.

~~~
dyu-
They made buck for themselves where their infrastructure runs on (*nix).
Google's internal build tool (before bazel) was the same. Now chromium's GN
build tool was made for their product that was targetted to run everywhere,
which is why windows support is good.

------
leog7
Do such tools have any quality/security standard?

------
coldcode
Does not support Swift at the moment.

~~~
sdwilsh
It does, although it may not be 100%. I've seen lots of PRs going by that fix
problems with Swift support.

------
omarforgotpwd
Nice build output. Unless you want to look over it afterwards that is.

~~~
LegNeato
A couple of quick notes:

* It also writes build output to an output directory

* It detects a tty and does the right thing, so on CI you get a linear log

* In the output directory it also includes timeline graphs that can be loaded in Chrome's devtools traceview ([https://github.com/catapult-project/catapult/blob/master/tra...](https://github.com/catapult-project/catapult/blob/master/tracing/README.md) and in Chrome proper).

* The tracing can be viewed live [https://buckbuild.com/command/server.html](https://buckbuild.com/command/server.html)

(disclosure: my team created Buck when I was at FB)

~~~
omarforgotpwd
Sweet! Looks like a great project, congrats!

------
NeverTrump
Anyone care to compare Buck vs pants vs Bazel/Blaze?

Or perhaps write out your individual experiences mentioning:

1) Team size

2) Repo size

3) Repo programming language distribution.

4) development/Testing/build/publishing strategy

5) Of course, the actual experience

------
edblarney
So much complexity to put some pictures on a screen and have people click
'like'.

It's dizzying.

I wonder if build-system complexity is an artifact of reducible complexity in
other areas.

Perhaps the next time we develop a language, it should comprise of it's own
build system that doesn't require any configuration, or rather minimal. To the
point wherein we didn't need to think that much beyond the obvious.

~~~
SmirkingRevenge
I think a few people had that same thought, and then invented Golang. Love it
or hate it, building a typical go app, or even a suite of apps, is pretty darn
simple.

If you can get by using only go for everything, you'll have a great time. But
complexity starts to become unavoidable when requirements move beyond single-
language ecosystems. At some point simplicity can end up costing more.

------
yueq
How come this tool became no.1 trending on HN?

------
fibo
I agree with the comments in this thread, and I add that Facebook knows
fanboys are stupid and there are a lot of them, so they try to take advantage
of it.

> Buck: A high-performance build tool

And in the title you read "a fast build tool", like yarn, as soon as it was
released it was the faster one.

F: let's use yarn/buck G: why? F: cause it is faster G: Did you already tryed
it? Did you measured or benchmarked it? F: No, but Facebook claims it is fast,
come on!

By the way, sometimes yarn does not work and you need to add a file to manage
it. Furthermore facebook is using the npm registry, do they pay for it or
support it?

Other than that, thanks to Facebook to bring awesome tools to the public, like
React.

~~~
noir_lord
For my needs yarn was a drop in replacement for npm and it _is_ faster, 5s vs
17s for a fresh install but critically it's reproducible, I get the exact same
output in node_modules every time I run it and that _alone_ was worth the
switch.

As for using the npm registry (by default) so what? Why would the npm folks
care, MS uses it as well with vscode ans its automatic resolution.

~~~
jcheng
Agreed that yarn is faster and reproducibility is critical, but in case you
aren't aware, you can have reproducibility with npm too by using the "npm
shrinkwrap" command.

~~~
TheCoelacanth
Updating a single package version in a yarn.lock file is much easier than
updating a single package version in an npm shrinkwrap file, in my experience.
With yarn it's just a single command. With npm shrinkwrap, you have install
everything from the current snapshot, then install the package you want to
update, then run npm prune, then regenerate the shrinkwrap file, then look
through hundreds of lines of mostly irrelevant diff to make sure that it did
what you wanted it to.

