
Bazel 1.0 - vips7L
https://opensource.googleblog.com/2019/10/bazel-reaches-10-milestone.html
======
choppaface
One of the major time savers of bazel is Remote Build Execution (RBE), which
allows you to build modules in parallel in the cloud. So if you have 1000
CPUs, you can really just have a client do `bazel build -j 1000 //...` and you
can get a huge speed-up. Remote (and local) builds all happen in a sandbox, so
you don't have to worry about e.g. preparing a docker image with the worker /
build slave environment. (You do, however, have to register your dependencies
with Bazel, which can be a hard at first). To add to this, bazel also has a
remote _global_ cache which can benefit large teams.

For fairly large C++ codebases, RBE is really a competitive advantage. I've
seen RBE cut down iteration time by an order of magnitude. I love CMake, and
CMake can get you plenty of parallelism, but CMake doesn't really provide a
tool for building several CMake sub-projects in parallel, and bazel handles
this really well.

Sadly Bazel RBE is still primarily a Google Cloud product. Also, GCE is slow
to work on supporting auto-scale, so you have to pay for unused workers. (Like
most products in Google Cloud, Google is ripping you off with alpha-quality
stuff). There's some very rough open source RBE stuff on Github that you can
run yourself, but nothing really production-grade yet.

gg ( [https://github.com/StanfordSNR/gg](https://github.com/StanfordSNR/gg) )
is a promising-looking alternative. It's research code, but it might be the
community's best hope for a non-Google alternative (that e.g. supports AWS
Lambda for parallelism). Bazel is great, but without independence (e.g. what
Kubernetes achieved) it's difficult to see bazel as dependable as make or
CMake long term.

~~~
9nGQluzmnq3M
> GCE is slow to work on supporting auto-scale, so you have to pay for unused
> workers.

Can you expand on what you mean here? GCE has had autoscaling for years and
it's quite configurable, I don't see why you would need to keep unused workers
around unless your build load is extremely spiky.

~~~
choppaface
Not auto-scaling VM compute instances, auto-scaling the Google Cloud RBE
service (I think it's called Cloud Build right now?). If you have a worker
pool of size 1,000, you have to pay for that even if your dev team only uses
it at full capacity 8-10hrs per day. Or maybe the Google Cloud sales people
working with my company are misinformed? IDK, but my experience with TPUs,
GPUs, AI Engine, Dataproc, BigQuery, etc... every GCloud feature I've used
(besides vanilla Compute VMs) have gotchas and/or time sinks that are costly
unless you have a load of free credits. For AI Engine, it took like 6 months
to upgrade their internal Kubernetes cluster; in the meantime, crashed jobs
would burn an extra 15 minutes of GPU dollars.

There's a lot of good stuff in Google Cloud, but be sure to get Google to pay
you to wade through the construction with free credits.

~~~
philwo
Cloud Build and Cloud RBE are two different products. The former is Google
Cloud's CI service ([https://cloud.google.com/cloud-
build/](https://cloud.google.com/cloud-build/)), while the latter is a hosted
backend for Bazel's remote execution feature (some infos are here:
[https://groups.google.com/forum/#!forum/rbe-alpha-
customers](https://groups.google.com/forum/#!forum/rbe-alpha-customers)). They
are not related.

Disclaimer: I work on the Bazel team at Google.

------
RcouF1uZ4gsC
I only wish Bazel had been written in a language that produced native binaries
like C++ or Go instead of Java. It seems like a waste for me as a non-Java
developer to install Java on my dev machine just for Bazel.

~~~
arianvanp
We didn't have Bazel on ARM for NixOS because we can't bootstrap openJDK on
ARM. There's no source code for a JVM that compiles on ARM afaik. Though in
theory we could package binary blobs from oracle and bootstrap JDK and Bazel
from there, it means now our trust path for your critical build tool has a
random Oracle blob in its trust path there that is extremely hard to get rid
off.

Build systems should be easy to build. Bazel is not by a far shot due to java
being a mess in this area (ironic given it being advertised as a cross
platform language).

These issues are not important for everyone perhaps. (Heck the entire
Kubernetes ecosystem pulls random binaries from docker hub without any
cryptographic signatures.) but it's still a shame; especially for a build
system.

~~~
blub
Google builds stuff for themselves and then dumps it for extra reputation
points on the dev community.

The part where this starts to suck is when their tools become pseudo-standards
and everyone has to live with their decisions.

~~~
munificent
I'm biased because I work at Google on an open source project, but the
complaint here seems to be:

1\. A corporation spends significant resources developing a project.

2\. Then they release to allow the public to use it at zero cost. Often they
even invest in supporting that external use.

3\. The project provides enough value to so many users (who chose it of their
own volition) that it becomes nearly a standard.

And this is somehow a bad thing?

~~~
blub
The complaint is that the tech mega-corps are too big and should be broken up.
Then they wouldn't distort markets (among other things) by dumping free
products subsidized through ads on them, only to forget about said products
two years later.

Regarding your specific interpretation:

1\. The product was developed for internal use AFAIK, then released
externally.

2\. Google is getting something out of this: a PR boost, better engineer
retention and the engineers feel good about themselves.

3\. All these projects not only don't become standards, but they just add to
the pile of things one has to learn because there are still too many people
that are fawning over anything published by FAANG.

------
pcj-github
For me, the difference between other build systems and bazel was like the
shift from CVS to git... Profoundly better internal data model, but definitely
a learning curve.

~~~
spuz
That makes sense but there aren't many people out there using CVS right now.
What were you using before Bazel? I.e. what are you comparing Bazel to?

------
gravypod
Bazel, the tool, is fantastic. Haven't had a better build tool that I've seen
yet. The build _rules_ are awful. Almost every single rule is broken in some
catastrophic way and, as is typical with Google open source, it is very
difficult to ask for things to either be fixed or designed differently.

Major examples: rules_python flat out does not work for python 3, rules_docker
does not work without python 2, rules_proto cannot generate stubs for
languages, you cannot specify third party deps at a package level and there is
no explanation as to why (they will say it would destroy hermetic builds or
reproducable builds but that doesn't make sense when you think about ways to
support both).

~~~
EdSchouten
This has improved lately. Rules_docker's image pusher has been replaced with a
copy written in Go. This pusher is built using rules_go, so that it does not
rely on any host tooling.

~~~
gravypod
I'll have to update my tag and try again. I wish it could just use my docker
daemon which likely already had the image and layers.

------
taneq
Problem: There are N confusing build systems with arcane rules and you just
want to compile your program.

Solution: Design a simple, straightforward build tool.

Problem: There are N+1 confusing build systems with arcane rules.

~~~
halflings
Bazel is almost a standard for build systems at this point.

Did you ever use it, or at least see what a bazel build rule looks like [1]
before calling it arcane?

[1]
[https://docs.bazel.build/versions/1.0.0/tutorial/cpp.html#un...](https://docs.bazel.build/versions/1.0.0/tutorial/cpp.html#understand-
the-build-file)

~~~
pja
Build rules for bazel are straightforward when you’re on the happy path &
trying to do something that bazel already understands. It’s when you’re not on
the happy path that things get awkward.

E.g. How easy is it to integrate a new C++ compiler into bazel? (and I don’t
just mean what you get by changing CC to point somewhere else. I want the full
integration with the bazel build system in order to get those hermetic builds
that are the whole point of using bazel in the first place).

How easy is it to build things which are not in the set of blessed languages?
What if I have a Haskell component? Can I easily integrate that into my bazel
build? Last time I tried the answer was definitely not.

If you can use the pre-defined BUILD rules then bazel is pretty great. If you
can’t and have to start wading into the weeds of bazel’s hinterland then it
rapidly stops being so. If you’re inside Google, then you’re probably fine
because a blaze/bazel engineer is ultimately only an email away. For the rest
of us outside the Googleplex, extending bazel is an extended exercise in
frustration in my experience.

~~~
ondrasej
There's genrule() which basically runs a shell script. With that, you can
build anything for which you can write a command-line. And you can go as far
as invoke a binary that is built by another bazel rule.

You can also define your own rules in a subset of Python (by wrapping other
rules including genrule()), so adding new languages should be simple enough.

~~~
pja
Based on my experience with bazel the phrase “should be simple enough” is
doing an awful lot of work in that sentence ;)

It’s just a SMOP, right?

------
lenkite
Whats the advantage of using Bazel over CMake ? What does one do about package
management ?

~~~
sayrer
For packages, you download things by specifying an archive (zip file, git
repo, etc), and a SHA256 hash. Think of it as extremely-precise version
pinning. You can also refer to other Bazel projects and use their internal
rules if permitted.

The biggest advantages of Bazel over something like CMake are:

1) It is very general, so you can easily express dependencies of this sort:

\- concat three text files, then

\- run them through a parser generator, then

\- use the generated parser in a C library, then

\- use that C library in a Go server library and a Rust client library, then

\- package the Go server into a Docker container and build an iOS binary that
uses the Rust client

This is a cross-language dependency chain that also runs a few Unix tools. In
this example, if you edit the Rust client, only the Rust library and the iOS
client will be rebuilt. If you edit one of the text files that compose the
grammar, everything will get rebuilt.

The dependencies are also a DAG, so if your projects have a shared Go library
and you change that, the Go server would get rebuilt, as would the Docker
container.

While all this might seem distressingly general, it removes the need for
things like "bootstrap.sh" files, "build.rs" files, and all of the unstated
dependencies they entail.

2) If you download all of the toolchains (C++, Java, Rust, Go, etc), it's
impossible to have "works on my machine" problems. Exceptions are use of
"local_*" rules, and Apple binaries (you need machine-local SDKs).

3) This allows great caching. You can run the equivalent of ccache for all of
this stuff, locally or shared on a LAN. RBE is even more advanced.

~~~
fanf2
(1) sounds like table stakes for any build system - `make` has done all that
for decades

~~~
gravypod
Make cannot guarantee that this is the case. Bazel sandboxs all builds and
hides them from non-reproducable things (internet, files they shouldn't get to
see, etc) to make it "impossible" for you to have a build that isn't specified
correctly in the DAG

------
ahupp
I used buck at Facebook quite a bit, and it's my favorite build system. I
assume most of the stuff I liked about buck translates to bazel:

* Build rules were extremely simple to write

* Sandboxing meant we could safely share a global cache of build artifacts, massively reducing build times

* It runs a persistent server process that caches dependency and file change state, so incremental builds are fast

* Buck had a lot of android-specific magic to make APK generation fast as well, not sure if that's in bazel.

A lot of the stuff that's nice about these build systems only really becomes
an issue on large codebases (thousands of engineers, tens of millions of LOC).

The main weakness of buck (not sure about bazel) was handling 3rd party code;
either you'd checkin prebuilts or port the build to buck.

------
throwaway1777
In my opinion Bazel/Blaze and Buck are mainly useful to solve the problem of
having too much code to build on one machine. Reproducible builds which can be
done in a distributed fashion aren’t really useful when you can build your
whole project in a minute or two on one machine. That being said, once you get
the hang of it the overhead is not too high compared to simpler systems.

~~~
kartayyar
Xoogler here who used the internal version of Bazel ( blaze ) extensively,
liked it and now opted to use it for our 2 person startup ( at least for the
backend ).

You don't need a giant codebase to start benefitting from it. Even if you have
a smaller codebase, and properly use bazel to setup tests, you only test what
needs testing, and that makes a huge difference in productivity.

~~~
MapleWalnut
What languages are you using with Bazel? I have a project that uses Typescript
and Python. Are you using Bazel with interpreted languages?

~~~
evmar
Bazel has experimental but pretty sophisticated support for TypeScript, with
arguably a better compilation experience for large projects than the best
known alternatives. (Disclaimer: I'm biased because I work on it, but I also
work on non-bazel TypeScript projects so I am aware of the tradeoffs.)

[https://bazelbuild.github.io/rules_nodejs/TypeScript.html](https://bazelbuild.github.io/rules_nodejs/TypeScript.html)

------
bazza451
Does anyone know of a real beginners level guide to Bazel that isn’t the docs.
Got a typescript monorepo with a few backend services/react frontends I wanted
to build and I got a bit lost in the complexity of it due to package.json
handling

~~~
G2H
We wrote a bit about how we build a TS monorepo with Bazel on our company's
blog [1] if that can be helpful

[1] [https://dataform.co/blog/typescript-monorepo-with-
bazel/](https://dataform.co/blog/typescript-monorepo-with-bazel/)

~~~
bazza451
thanks will check it out

------
scele
One thing that I like about bazel is the way it integrates test and build
actions to the same action graph and behind the same CLI. Tests are then also
hermetic and sandboxed, and can be cached (or executed) remotely.

For example, you could configure your CI to push test results to a remote
cache, and when developers check out a clean master, ”bazel test //...”
against that cache will report all tests as pass without running anything.

------
slavik81
The lack of backwards compatibility between Bazel versions has been the deal-
killer for me thus far. Old build instructions should work with newer releases
of the same tools. It's important for the tools to remain compatible because
you can't change the instructions contained in old releases.

I hope the release of 1.0 is an indication that Bazel is taking interface
stability more seriously.

~~~
ageofwant
Right. Every time I build tensorflow I have to build bazel first, or get the
bazel version for the tf version.

I have to say, having used a number of Google's tools over the years: basel,
coral sdk, maps api and perhaps others, I'm always surprised at how 'how ya
doing' the experience is. Don't get me wrong, I appreciate their tools and its
openness, but I wish Google spend a bit more time doing software engineering,
and less throwing shit against a wall ascertaining stickiness.

~~~
philwo
> Every time I build tensorflow I have to build bazel first, or get the bazel
> version for the tf version.

This is fixed now:
[https://github.com/tensorflow/tensorflow/commit/991aec351b57...](https://github.com/tensorflow/tensorflow/commit/991aec351b574c4ec9157b9e557fb0f7f53bc4d9)

If you use Bazelisk
([https://github.com/bazelbuild/bazelisk](https://github.com/bazelbuild/bazelisk)),
you'll automatically get the matching Bazel version for TensorFlow in the
future (and all other projects that have adopted the .bazelversion file).

~~~
slavik81
Awesome. Bazelisk provides exactly the sort of compatibility I was hoping to
see.

Though, manually specifying a maximum Bazel version doesn't seem like it
should be necessary. The authors of Bazel know what features they have changed
since the version specified as the minimum, and Bazel itself knows what
features the BUILD file uses. That information together could be used to
calculate the maximum version.

------
krapht
Is it worth giving up language-specific build tools (CMake, Maven, Gradle and
others) for one build tool to rule them all?

~~~
jrockway
My experience with using the Google-internal version of Bazel is that yes, you
want to give up all other tools to use this. Having a proper dependency graph
makes it possible to cache maximally and accurately, and that results in fast
builds. You don't need it if you only have like one go program that you're
building, but you will start to see the disadvantages when you have a handful
of things you build. Did you build all the apps that depend on the proto you
updated? Do you need to build your PHP app because you change an internal go
library? Tools like "docker build" have no idea and tend to rebuild too much
or too little, and yield different results on different machines. This causes
a lot of headaches.

The thing that kills me about Bazel is that it is quite painful to deal with
Java. I never seem to get a working version on a Linux distribution. On
Windows it's always sitting down in the taskbar showing ads. Oracle calls you
to demand money. You check the bounds of an array and they sue you. It's just
not worth it. So I don't actually use Bazel.

~~~
loeg
> Having a proper dependency graph makes it possible to cache maximally and
> accurately, and that results in fast builds.

Pretty much every other build system (e.g., make) is rooted in having a proper
dependency graph. This is not a unique property of Bazel.

~~~
rienbdj
Make (to use your example) does not do any sandboxing to ensure that the
declared dependency graph is actually correct. This is the big innovation in
Bazel (and Co).

~~~
emelski
Electric Make has had sandboxing since 2002, no conversion from your familiar
make-based builds to a new shiny build tool required, and it can make on-the-
fly corrections to execution order if that sandboxing reveals that incomplete
dependency specifications caused something to run in the wrong order (relative
to a strictly serial build).

"Sandboxing" in a build tool cannot be claimed as Bazel's innovation.

~~~
adgasf
Can you link to the project?

~~~
emelski
Sorry for the belated response. Electric Make is part of CloudBees
Accelerator: [https://www.cloudbees.com/cloudbees-
accelerator](https://www.cloudbees.com/cloudbees-accelerator).

------
dwrodri
My only experience with Bazel is using it for Tensorflow (TF). It seemed very
roundabout for what was required and definitely had some painful version
issues as TF got closer to 2.0. Hopefully this means more stability and ease
of use going forward for TF users.

------
tln
> We will have a window of at least three months between major (breaking)
> releases

That seems... short

~~~
wffurr
Well you can keep using the LTS releases. I can't find offhand how long "long"
is. Presumably much longer than three months...

------
wpdev_63
It would great to see benchmarks against gradle.

Gradle can become really slow in large projects.

------
shusson
Anyone had any experience using Bazel with Angular projects?

------
human_error
How is it different than Buck or Pants? What makes Bazel better than them? If
I already use Buck or Pants what benefit would I get if I move to Bazel?

~~~
adgasf
Bazel is more popular, with a larger community and better language support
than the other two. It seems that Bazel has "won".

------
panpanna
Looking forward to seeing this as a requirement in projects that already
require npm, phyton, cmake, gradle and ninja to compile :(

------
jdkee
What does Bazel do exactly?

~~~
mc3
Bazel is an open-source build and test tool similar to Make, Maven, and
Gradle. It uses a human-readable, high-level build language. Bazel supports
projects in multiple languages and builds outputs for multiple platforms.
Bazel supports large codebases across multiple repositories, and large numbers
of users.

~~~
kajecounterhack
It also builds code repeatably & hermetically (interpreted languages excepted;
at google python is built super hermetically where a python binary +
environment is bundled with python script(s) into an executable; some
companies like dropbox have copied this). You run build, and you get binaries
for N kinds of systems (e.g x86_64, ARM). It builds a dependency graph of your
code that you can query and results in cache-able builds (only rebuild the
part you need). Since building dependencies in a tree can be massively
parallelized, it also allows you to parallelize your builds well across
machines (with some effort).

You use the one tool to compile C++, Java, Python, Go, Rust, etc.

------
person_of_color
Learn CMake.

