
Buck: A high-performance build tool - geezerjay
https://buckbuild.com/
======
jitl
I get the sense that Bazel will end up with a larger ecosystem than Buck,
because Google’s Cloud business depends on open-sourcing more stuff, so
they’ll put more effort into open-source Bazel.

Uber is migrating from Buck to Bazel, for instance.

~~~
diebir
Buck and monorepo is a disaster (there). A bunch of people from Google forced
that stuff down everyone's throat. Concerns and real life problems were simply
discarded. Another solution in search of a problem. Not invented here at the
extreme.

~~~
Game_Ender
What made Buck fail at Uber vs Facebook? I thought they successfully used it
for larger repos there.

~~~
lhorie
My understanding is that Facebook doesn't do Go, so it's always been a second
class citizen for Buck. Go folks at Uber were spending a significant amount of
time (in the scale of several months) trying to upstream fixes to Buck core,
but found that Bazel extensibility allowed them to get similar functionality
in a matter of weeks or even days. Buck also didn't seem to have a good story
around Thrift/Proto and JS/node.

At Uber, the Java stack is still sort of ok w/ Buck (because Buck handles some
things better there), so they're taking a wait-and-see approach until the
Bazel ecosystem catches up w/ Buck on the concerns they care about. But long
term, we are envisioning implementing a company-wide monorepo, and that sort
of entails having a unified build system.

------
entelechy
We think Buck is great. It's Deterministic hermetic builds and it's composable
and declarative high-level build description language made packaging very
easy. We built even built a package manager for Buck:
[https://github.com/LoopPerfect/buckaroo](https://github.com/LoopPerfect/buckaroo)

Currently we marketing it for C++ but it can be used for any language that is
supported by Buck.

~~~
Game_Ender
What’s your take on Bazel? Have you considered offering support in Bazel for
your package manager?

~~~
entelechy
> What's your take on Bazel?

Here a couple key points:

\- Buck and Bazel are very similar.

\- Buck currently models C++ projects better [1].

\- Buck leverages remote caches currently better than Bazel [3] \- Bazel is
very easy to install

\- Bazels "toolchains" make it very easy to onboard newcomers (to any project
and language) but also ensure the build will run as expected.

\- Bazel is less opinionated and more extensible than Buck.

In fact Bazel is so powerful that you can have Buildfiles that download a
package manager and use it to resolve more dependencies. This is great to get
things off the ground, but makes things less composable because the package
manager won't see the whole dependency graph. As a result you might get
version conflicts somewhere down the line.

To summarize: I think having a _very_ opinionated build-system is easier to
reason and scales usually better.

Communities with very opinionated packaging and build-systems are proving this
by having orders of magnitude more packages that eg. the highly fragmented C++
community where configuration is prefered over convention.

> Have you considered offering support in Bazel for your package manager?

Yes we did. As soon as this feature [1] is implemented we will have a 1:1
mapping for C++ Buck Projects and Bazel. Then after a small (automated)
refactoring of our Buckaroo packages, you should be able to build any package
from the Buckaroo ecosystem with either Buck or Bazel.

Btw. The cppslack community is attempting to create a feature matrix of
various build-systems here [2]

[1]
[https://github.com/bazelbuild/bazel/issues/7568](https://github.com/bazelbuild/bazel/issues/7568)

[2]
[https://docs.google.com/document/d/1y5ZD8ETyGtxCmtT9dIMDTnWw...](https://docs.google.com/document/d/1y5ZD8ETyGtxCmtT9dIMDTnWwbeg-
HCpLkNiFJPO4Rdw/edit#)

[3]
[https://github.com/bazelbuild/bazel/issues/7664](https://github.com/bazelbuild/bazel/issues/7664)

~~~
hlopko
I'd say the main the challenge that both Bazel and Buck have to face in the
C++ land is that it's still too much work to migrate an existing
cmake/autotools project to them. One nice project that tries to tackle that is
[https://github.com/bazelbuild/rules_foreign_cc](https://github.com/bazelbuild/rules_foreign_cc).
But yes, there are minor things that make migration from Bazel to Buck or vice
versa not as smooth as it could be, and I'm confident those will be fixed.

To me the biggest added value of Bazel is the remote (build and test)
execution (which will get a nice performance boost from
[https://github.com/bazelbuild/bazel/issues/6862](https://github.com/bazelbuild/bazel/issues/6862)
in Bazel 0.25; also mentioned in [3]).

(Your [3] doesn't compare Bazel and Buck, only Bazel with remote caching and
without it, so it's not clear from it that Buck leverages caches better than
Bazel).

And one nit, Bazel doesn't allow you to download anything in the loading,
analysis, or execution phases ("the BUILD files"), those are completely
hermetic, sandboxed, and reproducible (when compilers are). The package
manager integrations happen when resolving external repositories ("the
WORKSPACE file"), where non-hermetic behavior is allowed and is used e.g. to
autoconfigure C++ toolchain, or download npm packages.

------
malkia
Also (bazel already mentioned, so no need to), but:

Pants -
[https://www.pantsbuild.org/index.html](https://www.pantsbuild.org/index.html)

Please.Build - [https://please.build/](https://please.build/)

Closely related, but functioning bit different:

Gn - [https://gn.googlesource.com/gn/](https://gn.googlesource.com/gn/)
(targets ninja). Used by Chromium, Fuchsia, and others

Soong -
[https://android.googlesource.com/platform/build/+/master/REA...](https://android.googlesource.com/platform/build/+/master/README.md)
(targeting Kati?)

~~~
antpls
Would it be interesting to list Infrastructure-as-Code tools and cloud
pipeline tools in parallel ? At some point, build tools and infrastructure
tools may converge :

Kubeflow Pipeline (for ML) -
[https://www.kubeflow.org/docs/pipelines/pipelines-
overview/](https://www.kubeflow.org/docs/pipelines/pipelines-overview/)

Tekton Pipeline -
[https://github.com/tektoncd/pipeline](https://github.com/tektoncd/pipeline)

TF Extended - [https://www.tensorflow.org/tfx](https://www.tensorflow.org/tfx)

~~~
Bombthecat
I'm also pretty sure that this will happen.

------
antimora
This is great that we have many options. However, as I am reviewing the
various build tools I can't find that suits my needs. Pretty much a lot of the
build tools, such as bazel, nix, take over the native language dependency
management and build operations. What I would like is a simple build
dependency tool, similar to what Amazon has for their build tool called
(brazil) that can manage pulling packages locally to build.

Here is what I would like to achieve in own projects:

1\. Work on a NodeJS project A that can locally install NPM packages based on
packages.json.

2\. Have another Python or Go project that build depends on project B.

3\. The build tool allows the dependencies pulled either from artifact
repository or use checked out version locally.

This build system focuses on the WHAT to build vs HOW to build, which will be
driven by project's own build tool, e.g. ant, npm, maven, etc.

I am considering creating my own simple build tool which is inspired by the
Amazon's own tooling.

BTW, does not anyone know if I am allowed to develop my own tooling inspired
on company's internal tooling? Obviously it won't be exactly the same but
taken with lots of inspirational values.

Here is more details on the Amazon's build tool:
[https://gist.github.com/terabyte/15a2d3d407285b8b5a0a7964dd6...](https://gist.github.com/terabyte/15a2d3d407285b8b5a0a7964dd6283b0)

------
kadaj
Software development is all about not re-inventing the wheel. Yet there are
countless build systems, tons of pointless frameworks. I wonder developers
have nothing else to do. Every company releases their own version of
everything.

~~~
nimih
My understand of Buck (and Pants) is that they are the result of ex-Google
employees going to other companies (resp. Facebook, Twitter), realizing that
Blaze was more or less the Right Way to do a build system (at least for their
set of circumstances), and then being forced to re-implement the ideas from
scratch (edit: or memory, or exfiltrated docs/code) because Blaze was not open
source. Reusing extant software is great, but it requires that software to be
available to you in the first place.

~~~
shereadsthenews
“Reimplementation from scratch” seems like a generous description of Buck. It
was initially so like Blaze that I always assumed a xoogler exfiltrated at
least the documentation of Blaze.

~~~
nimih
I really can't speak to that in any way, I was just trying to answer the OP's
question as to why there are a number of--on face very similar--Bazel-style
build systems. It certainly seems like a fair assumption, though.

------
duality
Can anyone who's used both compare Buck to Blaze/Bazel?

~~~
lhorie
Uber is migrating away from Buck to Bazel. One major reason is poor support
for a variety of languages (Go being one big one, where fixing issues was
historically slow due to a need to upstream fixes to Buck core).

I haven't worked with Buck myself, but colleagues who evaluated it for JS have
expressed concerns with lack of support/ecosystem there as well. In
comparison, there are various Bazel rulesets for JS/Typescript, and I've had
some pretty good experience w/ implementing rules myself. The Starlark docs
are good.

Another thing going for Bazel is its ability to embed external codebases into
a build system. This mechanism allows rules to be shared among repositories in
a reusable fashion.

~~~
mgoblu3
Does this include mobile projects? I know Uber was/is pretty big on the Buck
migration

~~~
lhorie
Yes

------
blattimwind
> Buck looks at the contents of your inputs, not their timestamps to figure
> out what needs to be built. As a result, incremental builds should always be
> correct, so there's no need to perform a clean build.

At least one of these new-fangled tools gets at least one part right. The
salient point is now of course whether Buck considers the correct set of
dependencies as well as negative dependencies.

~~~
madhadron
Buck makes you declare every dependency explicitly. A particular target must
specify exactly what files it uses, and when it makes a build, it copies those
files from the target and all its dependencies into an empty sandbox and
builds there. It's wonderful.

~~~
Aeolun
Wonderful until you have a few thousand files you need to declare explicitly.

Or do I misunderstand something here?

~~~
jingwen
[https://buckbuild.com/function/glob.html](https://buckbuild.com/function/glob.html)

------
appleflaxen
why are there so many build systems?

it's understandable why programmers have opinions about aesthetics regarding
their IDE, language, or framework of choice, but what is there to be
opinionated about with a build system?

~~~
jasode
_> , but what is there to be opinionated about with a build system?_

I think you're saying this because _your_ idea of a "build system" is a very
simple set of sequential steps from known source code input files to an output
binary.

The differing _opinions_ come in when the "build system" includes various
_contradictory philosophies_ of how to configure and specify the building of
complex software.

Different opinions on:

\- syntax : should build config be XML, or JSON, or YAML, or custom syntax?
E.g. Ant and Maven used XML but Gradle does not

\- dependencies search: should build system _implicitly_ find and add relevant
dependencies? Or should programmer _explicitly_ specify each one?

\- reproducible builds (version pinning) vs auto-updated dependencies.

\- should the build system be "smart" about cross-platform differences? If
yes, you end up with complicated build tools like GNU Autoconf "configure"
bash script, or CMake's complex syntax.

\- should build system do optimization and bundling tasks that's not strictly
limited to "compile" steps? Some Javascript build tools try to eliminate
redundant or unused js code to make downloads smaller.

\- etc, etc.

I also recommend reading the blog post _" So you want to write a package
manager"_[1] to get an idea of how complicated a build system can get. The
title says _" package manager"_ but much of the material is also about _build
systems_.

The bottom line is that reasonable people can disagree on the priorities and
therefore, you can't create The One & Only Build System to End All Other Build
Systems.

[1] [https://medium.com/@sdboyer/so-you-want-to-write-a-
package-m...](https://medium.com/@sdboyer/so-you-want-to-write-a-package-
manager-4ae9c17d9527)

~~~
Aeolun
I feel like there is a ‘best’ answer to most if not all of these questions,
and the build system should standardize on whatever that best practice is.

It’s great that Timmy thinks an obscure extension of YAML is the best way to
define build configs, but all the rest of the engineers use JSON, so that is
what we use.

------
techntoke
Is it possible to do distributed builds with GCC and LLVM? I would like to see
more options for distributing builds across a cluster of servers in order to
significantly speed up the build process.

~~~
shereadsthenews
[https://docs.bazel.build/versions/master/remote-
execution.ht...](https://docs.bazel.build/versions/master/remote-
execution.html)

------
dikei
Can Buck handle Maven dependencies yet ? We don't want to vendor hundreds of
jars into our repositories like before maven.

~~~
gecko
As noted below, no, but Bazel now officially does, if you'd like something
similar (this is new as of a couple weeks ago):
[https://github.com/bazelbuild/rules_jvm_external](https://github.com/bazelbuild/rules_jvm_external)

------
michaelvoz
Bazel is significantly superior.

~~~
nixgeek
Are you able to elaborate on why?

------
QuadrupleA
Edit: nevermind, saw facebook and build tool and assumed it was site assets.
Note to self, read whole articles before commenting.

~~~
twic
This is a build tool for Java and C++ code. Good luck deploying those without
a build step.

