Hacker News new | past | comments | ask | show | jobs | submit login
Bazel Release 1.0 (github.com/bazelbuild)
263 points by klodolph on Oct 10, 2019 | hide | past | favorite | 174 comments

Bazel may be 1.0 and the internal abstractions of dag solving etc are rock solid from years of use inside Google, but the ecosystem around Bazel is quite bad at the moment. They've been breaking backwards compatibility constantly, so rules authors have struggled to keep up (hopefully done now that it's 1.0).

The reason the ecosystem maturity is so important for bazel is because its design encourages complete reimplementations. The rust packaging rules for bazel reimplement a subset of cargo's features from scratch. The docker rules reimplement image building from scratch (they don't actually use docker). The (Google maintained) python rules shell out to pip, but it's got a ton of problems, some packages are unbuildable, and it busts the cache constantly and redownloads everything.

If you're using C++ or Java, I can heartily recommend Bazel as an improvement to your current tool. If you're using anything else, hold off until the ecosystem is more mature.

I can believe all of these reimplementations are worth it for hermeticity, but I seriously doubt most shops need the extreme hermeticity Google does. It's a thing that really matters at gargantuan scale.

> They've been breaking backwards compatibility constantly

I maintain a ruleset and have been on top of Bazel upgrades for a few months. My experience hasn't been so bad. There definitely are a lot of deprecations, and some code changes required, but it was pretty manageable IMHO.

With that said, I agree that some stacks are more first class citizens than others. But I feel this is more due to some stacks' departure from the C++ style of build management than an issue of Bazel changes. At our company, the Go org has had a pretty good experience with migrating from Buck to Bazel. For our web org, the main challenges have been that Bazel doesn't really lend itself well to dealing with codebases where inputs and outputs co-exist in the same filesystem tree (and this is incredibly common in the web world with things like babel transpiled files, yarn offline caches, lerna, etc) or where the ecosystem has a bad relationship with symlink handling (e.g. jest, flow, webpack, --preserve-symlinks, etc)

> The reason the ecosystem maturity is so important for bazel is because its design encourages complete reimplementations.

My experience from before Bazel is that reimplementations were the way to go most times you had projects with multiple languages, because the alternatives lead you down dark paths—calling into a separate build system suffers from the same problems as recursive make.

What excites me about 1.0 is the possibility that the third-party rules can mature. Its not too bad if you only need a couple sets of third-party rules but my experience is like yours-when a new Bazel release came out, you had to figure out when to upgrade based on your dependencies on third-party rules. If you had several of these, it made upgrading Bazel infeasible.

Amazon’s build system handled this gracefully. Native build tool chains were always used, but wrapped in a relatively thin facade. As long as the buildtool could respond to a `release` target and put artifacts in a well known location it didn’t matter what the underlying implementation was.

The biggest drawback that I could see was all dependencies and tool chain needed to be modeled, but once they were, incremental builds, dependency rebuilds, etc. all Just Worked™.

Getting to that point requires a lot of upfront work, however, which can be hard to justify in smaller groups.

Can it do "bulletproof", repeatable incremental builds? I'm asking because I don't see how it would be able to, unless the toolchain is wired into your build system _at least_ as deeply as it is in Bazel.

Slightly OT, but Tup[1] handles this by sanitizing the environment and building in a FUSE filesystem to track reads and writes. It's bulletproof except for system header files (or more properly anything outside the root of your build, which usually is just system header files).

Nix[2] accomplishes bulletproof repeatable builds by sandboxing everything, but I don't believe incremental builds are a design goal (though if you modularize everything into different nix expressions you will in fact get annoyingly hard to use bulletproof incremental builds).

1: http://gittup.org/tup/

2: https://nixos.org/nix/

So bazel and brazil solve similar but very distinct problems. bazel is solving the problem of fast, repeatable, "bulletproof" builds for (very large) monolithic repos, whereas brazil is solving for fast, repeatable, "bulletproof" builds for (very large) collections of repositories or packages.

brazil is EXTREMELY agnostic to what happens within any given package build, with the associated tooling primarily focused on dependency artifact discovery. The (IMO) most important component of brazil is the version set, which is a versioned dependency closure that allows for one-off builds (so your auto-build application packages) as well as merges from upstream version sets (where library vendors release their code). they are the glue that makes the distributed, manyrepo software development model at Amazon feel safe and consistent.

If I had my way, I'd combine the single codebase UX of bazel with the multi-code base dependency tracking of brazil, because i think they solve complimentary problems extremely well.

Google just tests the living Jesus out of everything, and only versions a few core packages such as protobufs, grpc, and other packages used by pretty much everybody (this is called the "crust").

Everything else is tip of tree, and things are automatically re-tested using checked-in tests if they are affected by your changelist. You basically can't submit if you break anything. So in a way, Google doesn't need "versioning". Whatever is currently checked in is good to go.

Tests are required, of course, and a Google reviewer won't let you submit anything if your tests suck. This, obviously, precludes the use of such a set-up in, shall we say, "more agile" orgs which don't have good test coverage.

Blaze (at Google) is also not just a build system, but also an interface to a much larger distributed build and test backend, which lets you rebuild everything from the kernel upwards in seconds (by caching petabytes of build products at thousands of possible revisions), serves up source code views for developer workstations (code is not stored there either), and sustains the scale of distributed testing needed for this setup to work. As a result, nobody builds or tests on their own workstation, and there's close to zero (or maybe even zero, period) binaries checked into Google3 monorepo. If you need a Haskell compiler and nobody used it in a while, it'll be rebuilt from source and cached for future use. :-)

Fundamentally, I think Google got things very, very right with Blaze. Bazel is but a pale shadow of what Blaze is, but even in its present state it is better than most (all?) other build systems.

> Blaze (at Google) is also not just a build system, but also an interface to a much larger distributed build and test backend, which lets you rebuild everything from the kernel upwards in seconds (by caching petabytes of build products at thousands of possible revisions), serves up source code views for developer workstations (code is not stored there either), and sustains the scale of distributed testing needed for this setup to work. As a result, nobody builds or tests on their own workstation, and there's close to zero (or maybe even zero, period) binaries checked into Google3 monorepo. If you need a Haskell compiler and nobody used it in a while, it'll be rebuilt from source and cached for future use. :-)

The distributed, cached builds are effectively the same thing for Brazil (at the package level), with applications, libraries, system packages, all the way down to compilers being built from source. Brazil doesn't have the level of file system abstraction that Blaze/Bazel does, and prefers towards local development using overlays version sets for certain core build tools.

>Fundamentally, I think Google got things very, very right with Blaze. Bazel is but a pale shadow of what Blaze is, but even in its present state it is better than most (all?) other build systems.

I think Bazel gets things amazingly right for a single monorepo even w/o all of the fanciness that comes from internal systems and services, but the mechanisms for managing out of source dependencies is just flat out clunky. I suspect this is because Blaze never had to really solve that problem, and I think Brazil is a much better tool for solving it because it's what it had to do.

>> mechanisms for managing out of source dependencies is just flat out clunky

I'm not sure I agree. You can reference arbitrary, versioned out of tree deps via WORKSPACE file. You can pull from git at tag or revision, you can pull from cloud storage, maven, pypi, etc. Just about any realistic scenario is supported, and those that aren't yet are easily scriptable using a restricted python-like extension language.

Perforce effectively provided this interface as long as you stuck to packages within the same depot. Multi-package reviews and commits were very natural. The git migration eliminated that and moved the concern up to build time selection of versions. Not terrible, but definitely less convenient.

It would be neat if all build systems had a standardized way to return their dependency graph (like gcc -MD) so they can be used recursively by other build systems. I'm not aware of any that does and there's far from a standard which allows easy chaining of multiple different sub build systems. Which unfortunately either leads to recursive make, remodeling the dependencies in two places or forcing everything into the same build system.

Agree. I've been dealing with Python rules for Bazel at work and the default implementation of re-downloading everything with pip became a real pain real fast. Decided to generate the third party py_library with a separate tool instead and much happier for it. As an aside for those looking to start with Bazel, it's worth checking out the BazelBuild Slack -- https://slack.bazel.build/

just curious: is that separate tool open source?

What are the reasons to prefer Bazel to Maven or Gradle foe Java? I know it is supposed to be faster but on the other side last time i checked it it was much more complex and time consuming (expecially compared to Maven). Are there other good reasons?

I've worked with both and while Bazel definetely has a learning curve, mostly because is very different than Maven and Gradle, I actually find it considerably simpler than those two. Maven dependency management is really complex, and I'm tired of dealing with dependency hell. And Gradle doubles down on Maven complexity where you stop having declarative XML config and instead have actual Groovy/Kotlin programs.

So, please consider the difference between complex and unfamiliar.

Speaking as someone who just got bitten by Maven’s dependency resolution now only building broken artefacts. (Sub-dependency’s dependency is loose enough to pull in a version built with an incompatible JDK) I prefer Bazel

my organization's goals for java builds are consistency (we want to be able to build deterministic jars), reliability, and playing nicely with IDEs via tooling.

given that, as far as i can tell, there's nothing so compelling that it's worth switching over your project or organization. the tooling around maven is so good, and the inertia so strong (for my organization anyway), that there's no real incentive to switch.

we've found that non-maven developers grumble a lot when forced to use it but then find the system, while verbose, is really solid. we are certainly not willing to throw that all away to play with google's latest cool thing.

I grumbled a lot against Maven when my employer at the time decided to throw away our Ant build for Maven, considering that:

- We actually made good use of Ant tasks and macros for having convention over configuration build infrastructure

- Maven was still on 1.0 beta releases, with incomplete support for the plugins we required

However this was more than 10 years ago, and nowadays I am yet to find something that beats Maven eco-system, specially given that I don't suffer from XML allergy, rather it is my favorite configuration format.

I cry a little beat every time I have to wait for Android Gradle builds.

Guess what, the upcoming Android Summit is enjoying yet another Gradle performance improvements talk. They can't get enough of them.

I don't work in Java, but just so you know, you can define Maven deps in your WORKSPACE file, and Bazel will automatically download and cache them (and their dependencies) much like Maven would. The rest of the build is trivial to set up.

I think for Java projects Bazel is a very bad choice. Last time I've checked it was not even able to resolve dependencies transitively. And I doubt that it's faster than recent versions of Gradle.

The Bazel team published official rules for managing transitive dependencies from Maven: https://github.com/bazelbuild/rules_jvm_external

There are also community created solutions like https://github.com/johnynek/bazel-deps, and https://github.com/square/bazel_maven_repository.

Disclosure: I maintain rules_jvm_external.

For a mono-repo setup I find Bazel much better than those two. If you have few hundred projects or more, Maven is a pretty bad choice. Gradle is slightly better, but still much slower than Bazel.

But it's not just speed, Bazel does do true CI: you make a change in a a lib and then you can immediately compile and run tests on anything that depends on that directly or indirectly. I have not been able to do that with Gradle, forget about Maven.

Finally, if you use other languages, with shared artifact such as protobuf files, Bazel is simply amazing: change a proto file and recompile everything that depends on that on every language.

I very much doubt that it's faster than Gradle, but Gradle does do quite a bit more than Gradle.

It's not an absolute, but a contextual thing. For a clean build, on a single machine, under normal circumstances, gradle will often outperform bazel, because bazel pays a cost for isolation, parallelism infrastructure, and trying to max out your machine. OTOH, the abi-based compilation-avoidance, higher sensitivity to caching opportunities, and parallelism can make incremental builds much much faster on Bazel. Add in test-result-caching for non-flaky tests, and it's really a solid win for builds that don't invalidate most of the build graph.

So it depends on (a) the shape of your graph, and (b) the nature of the change being built (e.g., does it invalidate a core library and all its downstream, or is it close to the app?).

The other thing is remote build execution - the high focus on hermetic builds and parallelism makes building on a cluster of build workers extremely powerful, reducing build times, reducing load on developer machines, etc. And there are out-of-the-box solutions, initially from google on GCP to let you do this yourself.

So... is bazel faster? Hell yeah, in some situations. Kinda in others. Not in some others. You need to think through what your development model is, your scale, how you construct (or want to construct) your build graphs, isolation of components, etc.

Bazel incremental build is much much faster than Gradle, if you follow the one-package-per-build rule.

They're really excited about 1.0 for this reason, they're tired of breaking other people's builds too. The goal is to limit that with the 1.0 release, so expect this to get a lot better.

I follow some of the relevant repositories, and there has been a big effort over the last few months to get many many loose ends cleaned up, toward a stable set of tools that will change in a predictable cadence without undue breakage.

They’re underinvesting in their Python story in my opinion.

To my knowledge rules_python still doesn’t support pip3 installs, and from browsing Github, Twitter, and the Bazel Slack it seems everyone is reimplementing their own package management integration because the default is so undercooked.

Right now rules_pygen is the best open source option, but it isn’t fully featured and was broken by Bazel 0.28.

About the container stuff specifically, I think the different solution provided here versus off-the-shelf Docker is not merely to be compatible with Bazel. Rather it looks like, from having used it modestly, an intentionally different way of thinking about container builds, aiming toward different performance characteristics that scale etc.

I can't think of any reason you couldn't call the docker tools in a genrule() though, if you prefer.

Docker's build system is really strange. The build cache assumes that your dependency tree is linear, and it will invalidate the build cache if any file dependency changes, but a `RUN apt install foo` doesn't invalidate the cache--it's just assumed that every RUN statement is idempotent. Further, the Dockerfile's COPY is close to worthless--if you want to import all of your repo's Python packages, you have to put them all under a single directory or you have to copy them one by one (or you copy the whole repo and deal with the fact that every JS and README change will trigger a complete rebuild). Also, everything depends on a running Docker daemon and last I checked, the performance degraded if you tried to build multiple images in parallel.

Strange doesn't begin to cover it :). It's just broken.

Yeah, I actually like the speed with which it can build images. But it's a risk that it's an entirely different implementation. If you have a team of people to deal with maintenance, then it's worth it. If you don't, then when an incompatibility crops up, you're stuck until you can hack around it. Having a really well maintained docker ruleset is crucial if you don't have that team

OCI provides a standard container image format, at least. Assuming your whole toolchain is compatible, then it shouldn't be an issue.

> but the ecosystem around Bazel is quite bad at the moment

I've noticed this as well. I'd love to use Bazel for our Python 3 project, but as far as I can tell, the advertised Python 3 support doesn't actually work. There are several issues filed for this, but apparently no progress since 2018. Some people reported workarounds, but none that I could reproduce.

I believe the core is solid, but language support feels alpha at best.

Python support is very much alpha. We use rules_python and the “reported workarounds” include forking rules_python, reimplementing a Bazel pip package manager integration, and adding a few twists to your Python codebase to handle Bazel idiosyncrasies.

It’s dozens of hours of work, and even then you quite easily can find yourself not enjoying Bazel’s caching features because of pip wheel being non-deterministic. It’s been a hard road.

That makes sense. What doesn't make so much sense is that the documentation indicates that Python 3 is supported, but everything I've tried and everyone I've spoken to have indicated that it's just broken. In any case, I hope I can use it eventually.

Any idea if deterministic wheels is on the Python project's radar?

EDIT: Quick Google search answered my question--the first result is an issue you filed here: https://github.com/pypa/pip/issues/6505

I haven't been super happy with the prioritisation of that issue in rules_python.

In our Java+Scala+Python monorepo we've got our Java+Scala tests caching so that part of our test suite gets a low as 2 minutes in CI. The Python part of our codebase is maybe 5-10% of the codebase but its tests take ~5 minutes in CI because of no caching.

I've been using Bazel full time for over two years now to build Java/Go/Containers/gRPC and everything in between, and I must really say that it's one of the best tools that I've stumbled upon.

A huge congratulations to the Bazel team for shipping 1.0!

It really is that good. I’ve been using it only over a year, but I’ve done a lot of build system work with different tools and Bazel stands out.

Most other tools seem to be competitors to Make (SCons, Jam, tup), souped-up scripting systems (Ant, Gradle), or configuration toolboxes (CMake). The ones that really stand out are the tools that tackle the problem of expressing builds in a way that’s both expressive and declarative. In my mind, these fall into two families: Gyp and Bazel (which includes Buck, Pants, Please.build, and that new custom thing Chrome uses).

(And then there’s Ninja, which I appreciate for taking a good / useful subset of Make’s features and doing it really well.)

In the topic of classifying build systems, there was a really interesting paper about that published at ICFP last year


They classify it based on how the build system detects what needs to be rebuilt and how the build system chooses the order things should be built in

I'd like to get a good grasp of the differences between Bazel and Buck. I've used the Bazel but not Buck - most descriptions of Buck describe it as being pretty similar to Bazel. (Also, that's a very interesting paper, will read in depth.)

From a historical perspective:

Bazel is the open sourced version of Blaze, google's internal build system. Buck is Facebook's open source version of their own implementation of a Blaze-like system. Pants was built as an open source implementation of Buck, before Buck was open sourced.

From a bird's eye view, knowing this history of their creators, I would guess that Bazel is more complicated internally, its API has a wider surface area, and ships with many optimizations included for the use case of building extremely large artifacts with many, many dependencies.

Someone who's more familiar with both, feel free to add details or correct my over generalizations! I've used Pants professionally and tried setting up Bazel for personal projects, but found it was too complicated for my needs.

one of the biggest differences between buck and bazel is,

when you want to write new build rules(to support your specific use case or to support an unsupported language etc):

- in buck you should update the buck code itself(you should touch so many places in JAVA) - whereas in bazel, you can extend bazel with starlark(subset of python) language and do all sorts of things like supporting whole new language. without touching even a tiny bit into bazel core.

This was the main reason my company is switching to bazel from buck

I use bazel at work heavily in a very large project. I’ve always worked with scripting languages so I’m much less familiar with build systems.

I’m not sure I get what makes bazel so good. It seems pretty simple to me. You have a bunch of directories with BUIILD files that are each sort of like Makefiles.

Am I missing something? It kind of just seems like a hodgepodge of scripts. I don’t dislike it, but I’m also not seeing anything amazing.

I’ll take a stab at it, since I’ve done some migrations to Bazel (and also away from Bazel). The comparison to Make and makefiles is good because Make, unlike some other build systems, is mostly declarative. Most of your makefile is going to declare what the inputs and outputs are.

If you use Make long enough, there are some obvious improvements you want. Multiple outputs, rebuild when options change, and easier cross-compiling are the top ones. Various build systems attempt to add these features. In my mind, Ninja is the only build system that added these features well, and it worked because Ninja removed all the other features to focus on just the build process (as opposed to specification / configuration).

If you think about these problems with Make, you realize that it kind of boils down to one big thing: you want your build system to always rebuild when necessary, and you want it to almost never rebuild when unnecessary. (Plus the bit about cross-compiling.)

Other build systems rely on the developer writing the build scripts to just “get it right”. Bazel is different because it sandboxes the rules to enforce hermeticity. In Make, I can include "pear.h" which includes "orange.h", but let’s suppose that "orange.h" is actually a generated source file… now, try writing this out in a Makefile (if you’re a masochist, say you’re cross-compiling). Yes, "orange.h" should be declared as an input to anything that includes "pear.h", but in practice, developers are going to screw it up. At that point you can end up with a build that uses two different versions of "orange.h".

Bazel sandboxes the commands so that any rule not declared to depend on "orange.h" will not be able to open "orange.h" at all. The process won’t see the file at all.

This opens the door for all sorts of optimizations and query features that are simply unreliable if you have to trust that human developers are writing the rules correctly. These optimizations, for large projects, result in radical build time improvements. For most build systems, a shared build cache would come with a risk of bad cache entries, but with Bazel, the risk is substantially lower. It’s also easier to get reproducible builds, which make it substantially easier to do certain types of auditing.

Bazel (and Bazel-derivatives/alikes) is not unique in fixing all of these problems. For example, there is build2, which is arguably a lighter-weight and closer to make (in spirit, not in syntax) solution (no Java dependency, etc). Here is an intro, if anyone is interested: https://build2.org/build2/doc/build2-build-system-manual.xht...

I did a quick skim and it doesn't seem to sandbox the build steps like Bazel? Maybe I missed something.

It doesn't do Bazel's style sandboxing where your entire compiler toolchain is part of your build system. It does what we call high-fidelity builds: it tracks changes not only to inputs but also to compile options, compiler itself (checksum), environment, etc., and if any of these auxiliary dependencies change, it triggers a rebuild. There are advantages and disadvantages to both approaches with build2's being lighter weight.

I don't see how build2 can be in the same spirit as Bazel when it misses the main advantage of Bazel

I never said build2 is "in the same spirit" as Bazel (whatever that means). I said it fixes the same problems as OP listed.

> I said it fixes the same problems as OP listed.

The OP listed noted that hermeticity is required for fast builds, and that humans can't be relied upon to provide hermeticity as a property. How does build2 guarantee hermeticity or otherwise support fast builds without a clean room solution a la bazel?

The OP was vague on why exactly hermetic builds are required to achieve fast builds. The only concrete thing they mentioned is caching which doesn't require hermetic builds (in the strict sense, as in preventing any outside changes) provided you can detect changes accurately.

To give a specific example, Bazel may prevent you from accidentally using a different version of the compiler while build2 will detect that you are attempting to use a different version.

Caching depends on a correct model of the dependency tree. If a dependency isn't included in the model, then a non-hermetic tool will silently succeed, incorrectly while a hermetic tool will do the correct thing and fail the build. Keeping the model accurate is too hard for humans, so in practice, non-hermetic build tools can't cache correctly.

It would be interesting to see how build2 achieves these things (hermeticity, cross-building, shared caches). I can’t see anything in the linked documentation that mentions these problems at all.

It currently doesn't do fully hermetic configurations (as in, where it is impossible for any inputs to come outside of the project). Instead build2 does high-fidelity builds where it makes sure that if any of the inputs change (including options, environment, compiler itself, etc), the target gets rebuilt. See my reply to a sibling comment for some details. We do plan to add support for hermetic builds though it won't be exactly like Bazel's -- the idea is to prevent and, where not possible, detect external changes and fail rather than rebuilding.

Regarding caching (and distributed compilation), this is currently on the TODO list though a lot of the infrastructure is already there. For example, the same change detection that is used for high-fidelity builds will be used to decide if what's in the cache is usable in any particular build.

While I agree we should mention these points in the documentation (things are still WIP on that front), I don't think cross-compilation deserves mentioning: for any modern build system it should just work. In build2 we simply implement things in the "cross-compile first" way, with native compilation being a special case (host == target).

Does build2 handle the case where code has

#include "foo.h"

and then, after a successful build, you create a new file named "foo.h" in a directory earlier in the search path than the foo.h that was used in the previous compile?

It does but not in a way you probably expect it to be handled: i.e., with some filesystem mechanisms to make the update command only see what has been declared as target's dependencies -- I must admit I don't know Bazel does this in a cross-platform manner (Windows, Mac OS); copying seems way too heavy-handed.

In any case, in build2 this is "handled" by not including headers as "foo.h" but as <libfoo/foo.h>, that is, with the project prefix. You can read more on this here: https://build2.org/build2-toolchain/doc/build2-toolchain-int... And the proper fix will hopefully come with C++20 modules.

Great explanation, thank you!

You get a lot of features for very cheap compared to other systems. That's the best way I can describe it.

Bazel is basically cross platform out of the box if one is careful with it: that includes a consistent build organization across platforms, cross builds if configured, and so on. It can build a library declaration for mobile and desktop and embedded and web in a single workspace; try that with CMAKE.

Bazel has a universal package system (e.g. download an archive or git repo) that allows for custom ecosystems (yes some of which are not great yet) to exist regardless of what is normal that ecosystem. This is especially notable for C++ where using CMAKE as a package system is a nightmare, with bazel I can just download any C++ repo off the internet and ignore it's CMAKE file for my own BUILD file. Also notably it's the first build system for C++ that hasn't required me to build or configure boost myself, someone can run bazel build against a repo of mine with boost in it without even knowing what boost is and it just builds.

Which brings me to hermeticy and reproducibility. If one is careful running bazel it always uses the same code for a platform (I've never had or seen weird "on my machine" issues with it which is impressive; reverting/stashing changes has always gotten people a working build again). All of the sources are version pinned, and so on. Getting to hermetic takes some work, but it's possible which is nice. A side effect of all this is my instructions for a bazel project are usually: install bazel, run build; and it just works! Yes parts of the ecosystem suck and break this, but that's a work in progress.

There are other features, like the query system, the test runner, the macro system, the local override idiom, the parallel build. The point is it really is a build tool for whatever needs to be built, however it needs to be built, and not just a scripting language useful for building things.

My experience is that I realized the value when I switched to a project that didn’t use it. Especially when I wanted to use other Google technologies like protos, or when I wanted to incrementally rebuild dependencies.

Disclosure: I work at Google, with Blaze, but not on it, or on Bazel. All opinions mine.

Woo hoo! I got to spend a few days with the Bazel team in the NYC office a few weeks ago and they were really excited about this announcement coming out soon. Congrats!

One of my favorite uses of Bazel is in CI/CD. I built a demo which builds the applications, creates Docker images, and then applies a K8s manifest to a cluster. It's OSS now under the GCP handle: https://github.com/GoogleCloudPlatform/gke-bazel-demo

Happy to answer any questions, public or private (email / website in bio).

That's cool, thanks Miles, I was looking at exactly this yesterday. I think I'll be able to finish figuring it out in the next few days when I have an hour or two.

Does it work with VB6?

You jest, but you could totally write the rules for it to support VB6.

Don’t look down your nose at VB, snobs.

Yeah we have a right to know

I hope that this 1.0 status will prompt the Qt people to take a closer look at Bazel, but that's probably strongly conditioned on the quality of Windows support. Qt Co. have indicated that CMake is the most likely replacement for qmake in Qt 6, which would be a lateral change at best.

Is anyone using Bazel to ship cross-platform GUI applications?

> I hope that this 1.0 status will prompt the Qt people to take a closer look at Bazel

By the sake of God no.

The last thing I need to compile Qt is a gigantic framework requiring the JVM. Plus the fact Bazel has so many side effect that that even quantum physics experiments looks more reproducible: Just try to compile tensorflow and enjoy the fun.

Stick to CMake, thanks.

I get not wanting a JVM (although all else equal I’ll take the JVM dependency any day over CMake), but how is CMake _more_ reproducible than Bazel? The latter builds in a clean room, Make does not.

> how is CMake _more_ reproducible than Bazel?

Because CMake contrary to Bazel do not break its retro-compatibility in its options and config every two minor versions ?

> The latter builds in a clean room, Make does not.

Sandboxing should be the concern of the package manager / deployment pipeline, Not the build system concern. That just makes things redundant and painful to debug.

> Because CMake contrary to Bazel do not break its retro-compatibility in its options and config every two minor versions ?

Lol that’s only recently true. CMake was infamous for breaking compatibility. Anyway, Bazel was pre-1.0 until just now, so of course expect some instability.

> Sandboxing should be the concern of the package manager / deployment pipeline, Not the build system concern. That just makes things redundant and painful to debug.

It makes things specifically very easy to debug. Not everyone enjoys troubleshooting issues on their machine due to their local environment.

Isn’t Bazel jre based? I don’t think that something like Qt will ever go for a build tool that requires the whole java runtime as a dependency.

(Also, cmake has become the de facto solution for open source libraries, not sure why they would select something else)

Being written in Java is essentially a non-issue. Bazel ships as a binary that self-extracts a JRE for its own use. You don't have to install Java unless you want to build Java apps.

The real issue for highly reusable products like Qt is portability of the build tool. CMake works on the long tail of OSes, some of which do not have any usable JRE available. That limits the ability to use Bazel as the only build tool.

For Bazel to be useful to projects like Qt (and other horizontal libraries like openssl, ICU, curl, ...) one would need a system for generating CMakefiles from BUILD files. The developers of those apps could use Bazel to improve build and test scaling, and then generate CMake (or other build tool) files as part of their packaging and distribution. I don't propose that this is an easy thing to do - all rules would need translation into other languages - just that it is a path which may benefit some projects.

Qt is moving to CMake for Qt6, they are already working on it. Qt6 release is planned in a year or so.

I’m aware, that’s why I mentioned it.

CMake may have a longer history of being used by open source, but Meson has been growing in popularity for some important projects. I definitely prefer Meson to CMake.

Bazel is also a great build system, but yeah, the JRE dependency will obviously make it less desirable for some users. Still, I hope people consider it anyways, because its focus on correctness is pretty great.

Last time I checked meson still doesn't have great Qt support e.g. it can't track rcc dependencies [0]. Funnily enough that issue is about rcc not making dependencies available to the build system, but I believe it actually started doing so somewhere down the track. I did a little work on it a couple of years back but regrettably never followed up [1].

[0] https://github.com/mesonbuild/meson/blob/27c01dff01832fa9d0c... [1] https://github.com/mesonbuild/meson/issues/97#issuecomment-3...

Unless Meson gets to support all the toolchains and OS supported by CMake, and the existing plugins eco system it is non starter.

Meson will most likely be kept as the GTK, GNOME build system.

Non-sense. It only needs to support the toolchains and OSes people actually want, by no stretch of the imagination does it have to support all of the same stuff CMake does. It is not a non-starter for many.

It is the build system of lots of projects already, not just GNOME but also freedesktop stuff too, libinput is one.

That is not how adoption at scale works.

Everyone on C++ community is gravitating towards CMake, Conan and vcpkg, they won't migrate to something else now.

Yes it is. CMake was adopted long before it supported everything it supports today.

Let us know when Meson gets first class support on Eclipse CDT, Clion, Qt Creator, KDevelop, Android Studio, VSCode and Visual Studio.

OK, ping me in 3 years.

Surely. I have seen too many carriages drive by.

Actually, there’s already an excellent Meson plugin for VSCode, and a pretty good one for Qt Creator, so I hope you’re a fan of losing bets :p

So far, everyone that has bet against CMake has lost to overtake it.

Scons, qmake, waf, premake, Gradle C++, MSBuild, ...

I know which horse to bet on.

Scons and qmake both predate CMake. It’s weird to characterize them as trying to “overtake” CMake; that’s not really a thing I think happened. As a matter of fact, there was a time when qmake was a better option; it was a fairly complete Makefile generator that had good support for subprojects and out of tree builds.

Then again, this is all irrelevant. What these projects all had to offer at one point is historical. Qt has to leave qmake because of qmake. During that time they evaluated many options, including a fairly innovative design called qbs - Qt Build System. In the end, the choice of CMake makes a lot of sense, as it is clearly the best available choice that had ecosystem support. Qt is unlikely to switch build systems again soon.

This entire argument has happened before, but instead of meson vs CMake it was CMake vs autoconf. The thing is, I suspect CMake will still exist even as Meson inevitably continues to gain traction, just as autoconf (unfortunately) exists today.

There is, in fact, room for more C++ build systems, and almost definitely room for better interop. (Meson and CMake have some interop today.)

> Also, cmake has become the de facto solution for open source libraries, not sure why they would select something else

It is definitely not the de facto solution for open source. After using CMake as a C++ developer, I will never use CMake again. I simply don't have time to write a program in a slow, stringly-typed, ad hoc DSL just to build my actual program.

At work, we use cmake; i've never seen a technology with so little 'traction'; I know one guy who is good with cmake all the other (including me) avoid touching this thing as much as possible!

Sigh, I remember debugging a complex makefile generation issue (it took me 3 days, two guys before me failed) and it didn't make me hate the make tool, I just wished it had better tracing/debugging but cmake that's a different story..

> It is definitely not the de facto solution for open source. After using CMake as a C++ developer, I will never use CMake again.

most projects are using CMake nowadays : https://www.jetbrains.com/research/devecosystem-2018/cpp/

Most _C++_ projects are using CMake, but it has virtually no adoption among other mainstream languages, and given most open source is not C++, CMake definitely doesn't have any strong share of the open source market.

My comment was about Qt, which is a C++ project. Of course CMake doesn’t have any market share outside of C++, but in the C++ world it has become the main build tool for open source projects. Even Microsoft is now spending their time supporting CMake via both Visual Studio and vcpkg.

Please[0] is a Bazel-like build system written in Go.

[0] http://please.build/

But, unlike Bazel, it doesn't run on Windows, which is a big limitation in enterprises.

That, and it doesn't actually support the multitude of weird things that Bazel does.

How’s Bazel documentation these days for people who haven’t worked at Google?

In April 2018, I tried to do a quick port of a tiny gsl::span-using toy app to absl::span, but I gave up, because Abseil wanted me to build my app using Bazel and Bazel docs seemed to assume that I already have some context that Googlers would have but I hadn’t.

(I emphasize that this was a _quick_ attempt at a toy program and not about making a serious time investment to learn a tool for a serious project.)

This is a legitimate criticism I hear even from new Google employees; Bazel's documentation is pretty hard to read unless you already know a lot of its core concepts, unfortunately.

gn had the same problem when I looked at it in 2017. I was working with breakpad at the time (a Chromium subproject), and unfortunately they had only partially ported some platforms to gn, and I didn't feel like plumbing in 3 different build systems for the 3 different platforms. Long story short, gn was painful without the internal documentation Googlers presumably had.

What about non-monorepos? At my company we have quite a few large-ish projects (consisting of ~5 published artifacts, LoC doesn't matter for my question, but maybe 100k per project?), but the problem is that we want different organizations to have access to different subsets of our projects... so we don't have a mono-repo?

It it possible to do some sort of conglomeration where we could bazel-all-the-things and still publish separate artifacts (jar's, deb's, etc.) for different purposes?

You could import each thing as `http_archive`s or `git_repository` or `local_repository`s and use visibility to manage what projects are allowed to use what

Just trying to understand... so we would have ~5 (going by my original example figure) Bazel builds and then use "imports of published things" to have projects that depend on each other work?

You would have 5 workspaces. How they work together is sort of up to you.

If you use `local_repository`, then you can link whatever's checked out from version control together. This can be dangerous since there's nothing that enforces what version of what works with what, but it's helpful for example, if you want to beta test a new version of a ruleset in a workspace that consumes it.

If you want to be strict about publishing artifacts and versioning, then `http_archive` is the way to go. You can choose your publishing schedule and other workspaces can independently manage which versions of the published artifacts they want to use.

`git_repository` is a middle ground if you don't want the hassle of publishing versioned artifacts, since it lets each workspace reference specific commit SHAs for the things they import.

Thanks. Things may have changed since the last time I investigated Bazel -- I'll read up on this stuff.

I'm actually currently leaning towards something like Nix or Guix just because it really encompasses everything...

Nix is good for inter-package dependencies, while Bazel is good for internal dependencies. You could use Bazel to build your packages for Nix, and import your other projects into your BUILD files with https://github.com/tweag/rules_nixpkgs.

I really want to use it but anything beyond the simple example is difficult to migrate to. I must have read the toolchain pages 10x times but still setting up a custom compiler or even different JDK setup is confusing.

From what I have heard, the toolchain story is being worked on heavily. There are some legacy reasons why toolchains were designed the way they were (mostly due to the fact that Google essentially only uses one compiler for each language, and certain toolchains were implemented natively instead of as plugins), but the confusing semantics are improving! Maybe give it another shot in 6-12 months.

If android switches to bazel now that bazel is 1.0 does that mean we can switch from a completely undocumented tool to a poorly documented tool?

Bazel documentation is fine from what I've seen. Moreover if you ask questions on Stack Overflow, members of the Bazel team will often respond (kudos especially to László for his support on SO).

> completely undocumented tool

Gradle is far from "completely undocumented". Unless you mean Android Gradle Plugin, which surprisingly _is_ somewhat documented, although the docs are tough to accidentaly just stumble by

I was actually refering to Soong not Gradle. Soong has been replacing makefiles since android 7. Given that incremental builds are more of a "hopeful aspiration" rather than "something that actually works" in Android 6, I can't blame them too much for doing a major reworking.

many bigots here dismiss bazel out of hand for being written in java.

the binary ships it own JRE and starts instantly (uses a server), why is java an issue exactly??

I've been using it for over almost two years on a 5 languages iOS/Android project, and then on everything I could. Congrats to the team!

Nice! I've been considering trying out Bazel for an Android project but didn't get around to it yet. How is the IDE support (Android Studio and/or IntelliJ)? And how is actual Android support in Bazel, for example in terms of feature modules, app bundles, DataBinding (that I guess is supported as a regular annotation processor) and third party plugins like Firebase? How much of a dealbreaker is it if I have to use a complex 3rd party Gradle plugin that I wouldn't want to rewrite for Bazel?

I've looked at Bazel some time ago but it was lacking full Swift support. There's some mentions to Swift scattered across the docs but some of them lead to 404 pages[1].

Has anyone gotten a Swift + Objective-C project running on Bazel or should I stay with Facebook's Buck?

[1]: https://github.com/bazelbuild/rules_apple/blob/master/doc/ru...

Edit: Found the correct links for the Swift/Apple support docs.



We (Zenly) are, it works well!

My company uses Java and Javascript, our build system is Maven. It starts the npm/yarn build process. We have hundreds of projects in it.

I don't like the tooks, but would it be worth the trouble to change to Bazel?

Bazel has good support for both languages. My personal experience is with the JavaScript + Rollup (or TypeScript + Rollup) rules for Bazel.

There are rules for Bazel + Yarn integration and although I use them, I’m not really qualified to give an opinion. With the TypeScript projects I work on, you have an ordinary yarn.lock and some extra Bazel rules in WORKSPACE to install the packages, and then you use some Bazel rules for TypeScript + Rollup to create bundled JavaScript files. Bazel’s TypeScript rules execute much faster than running tsc from the command line because someone took the time to figure out how to keep a hot copy of tsc running that Bazel can send commands to.

My personal experience is that you can do a gradual migration by working bottom-up, starting with the components that have no dependencies. There is a learning curve to it if you are the one writing rules, and it can take a while to get the hang of it / find where the good resources are.

I would evaluate it on an estimated cost/benefit basis. The primary benefit for large projects is build times. If you can quantify how much time your developers are spending waiting for builds, and estimate how much time you can cut off with Bazel’s shared caches, you can guess how quickly it will pay off or whether it will pay off at all.

If nothing else, you can find a leaf dependency somewhere in your projects and write some quick BUILD files for it. That shouldn’t take very long, if you have a half-day or day to spare.

I just posted about recent changes we made in the Bazel JavaScript layer. It's easier to migrate now that you can continue running whatever tools with the same config as Maven is doing. For example, if the `ts_library` rule doesn't work for you, it's possible to just use `tsc` (with the caveat as you point out, it will be slower) https://dev.to/jakeherringbone/layering-in-bazel-for-web-389...

That, combined with Maven->Bazel being a common migration path for Java project, should mean that your costs to change are incremental.

Another benefit for large projects is test execution time. If you can parallelize all your testing on a farm of machines you can bring your CI time down by an order of magnitude.

What helped you push through the steep learning curve when you started? Asking cause I was excited recently to try Bazel specifically for a Typescript monorepo, and the learning curve crushed me.

To be honest I had written some custom build systems and spent extensive time working on the problem. So a lot of the Bazel features made me go “aha” because they solved a problem I recognized—but the features seem weird if you don’t recognize the problem you’re solving.

What helps is to understand the separate phases of the Bazel build process and look at other Bazel repositories.

To me it’s not as bad as e.g. the mess I remember going through with TypeScript back when you had to write a lot of your own types for libraries you used.

Thanks for the reply.

A bunch of the problems I had might be due to out of date js tools, since apparently Bazel has been breaking compatibility a lot pre 1.0. Things I pull from the docs of the main js/ts bazel libraries are more likely to error than work. Will wait a while for things to catch up now that it's 1.0 and check back in next year or something.

Thanks again.

I started by going through the Java tutorial[1] to get a feel for it. It's fairly approachable and you don't actually need any Java knowledge.

Once you familiarize yourself with the concept of rules and targets, you can follow the instructions for whichever ruleset you want to use. The official one for JS is rules_nodejs[2]

Another thing that really helped me cement my understanding of Bazel was to deep dive into Starlark and write my own rules. The examples repo[3] is a great resource for that

[1] https://docs.bazel.build/versions/0.29.1/tutorial/java.html

[2] https://github.com/bazelbuild/rules_nodejs

[3] https://github.com/bazelbuild/examples/tree/master/rules

Give the title of this thread, don't you mean https://docs.bazel.build/versions/1.0.0/tutorial/java.html ?

Oh, yeah, good catch

I wonder how / if Bazel will integrate wit Google Cloud Build. IMO that should be an end goal for a tool like Bazel. At a first glance, they are (highly) incompatible. 1. Bazel's statefulness is a core feature while GCB is stateless. 2. They both need to control things holistically to work (or you lose advantages otherwise).

So in order to benefit from the two, it would likely require GCB to work with Bazel natively and also to have some powerful state abstractions in order to optimize builds in a distributed fashion.

There's a hosted Bazel in an early access preview. Tensorflow uses it.

Thank you! Would love to try that. How to access pre-release?

Just use it in your cloudbuild.yaml file: https://github.com/GoogleCloudPlatform/cloud-builders/tree/m...

The only contact I’ve had with bazel is via Tensorflow (more specifically tf lite). Last I tried it was not possible to set up bazel to output a static library. Has this changed?

It should be:


IIRC, transitive C++ libraries are coming sometime this year.

I am going to be hated and downvoted for that but let's go...

Bazel, like Bucks and other, try to bring on table a build system / deployment system that is multi-language, multi-platform and developer oriented. A holy Graal that many developer ( like me ) looked for decade and that many (large) organizations more or less tried to do one day (and most failed)

It is a good idea. It is a required tool to improve productivity. However, if the idea is good on paper, in the implementation, Bazel is damn wrong.

- Bazel is centered around "mono-repo" culture, making it much harder to integrated with multi-source, multi-repo, multi-version projects like many of us have. If I have no doubt that it is great at Google, the external world is not google.

- Bazel is made in JAVA, requires the JVM and this is a problem. That make Bazel not a "light" tool easy to deploy in a fresh VM or in a container.

- Bazel mix the concepts Build System ( like Make, ant, co ) and Deployment System like ( rpm, pkgsrc, etc). That makes Bazel pretty hard to integrate with projects that have existing build system, and almost impossible to integrate INSIDE an other Deployment System (usual package manager, deployment pipeline). The problem that Bazel faces with some languages ( python, go ) is a cause of that.

- Bazel venerates and follows the cult of "DO NOT INSTALL": compile and execute in workspace, there is no "make install", not installation phase. If "convenient" in mono-repo, this is often a nightmare because the boundary between components can be easily violated... and you finish by having many project that use internal headers or interface.

- Bazel makes mandatory ( almost ) to have internet to compile. This is a problem, a major problem in many organization (like mine) where downloading random source and binary from the Web is not acceptable for security reasons. Try to run Bazel in a sandbox.... and cry.

- Related to what I said before, Bazel mixes Build system and Deployment system. Doing so, it makes the same mistake that many "language specific" package manager and make uselessly hard to depend on an already installed, local library / component.

- And finally, last but not least... The options.... Bazel throw away 30 years of conventions / naming from the ( GNU / BSD world ) to create its own.... That make the learning curve difficult... Specially with a (up to recently) very sparse and outdated documentation.

I have no doubt that inside Google or Facebook, Bazel or Bucks are amazing.But they have been released too late for the external world in my mind.

Nowadays platform independant package managers like spack (https://github.com/spack/spack), Nix (https://nixos.org/nix/), GUIX (http://guix.gnu.org/) gives 95% of the advantages of Bazel without the pains of it.

1) bazel can be built with a self contained bundled paired down jvm, and possibly an executable using GraalVM but I don’t know anyone who tried that yet.

2) bazel can do offline compiles, and is actually built to run in sandboxes

3) bazel can act as a build system only and delegate to externa package managers

Look at rules_nodejs for example and the managed_directories + yarn_install/npm_install rule

4) depend on already installed stuff. Toolchains already provide a way to do this, and the android_sdk_repository rule literally requires preinstallation.

Seems to me your making a lot of claims about bazel without having used it.

> Seems to me your making a lot of claims about bazel without having used it

I lost literally weeks of my life to make Bazel work in external build systems for tensorflow and others. And I am apparently not the only one : https://archive.fosdem.org/2018/schedule/event/how_to_make_p...

Then I don't allow you to tell me "not having used it".

All the answers you gave are "fixes" that have been made after the community complained. Fixes that were sometimes not even documented.

Bazel is based on Blaze, which is Google's internal mono repo build system. But Angular, Kubernetes, Tensorflow, et al, are open source projects, Angular in particular, needs to work with NodeJS and Typescript ecosystems, which almost immediately demands dealing with node_modules dependencies if Bazel is to be used with it. You see to imply this was a case of Bazel folks somehow being blind sided by these requirements, but it's more a case that step 1 was to make a version of Blaze that works outside Google, step 2 was to make it work for non-monorepos and non-single-version dependencies, and step 3 was to make it integrate with alien package management ecosystems.

Actual support for the things you claimed is quite good now, and that's my point. You posted a number of claims that are now no longer true as of version 1.0. Yeah, a year ago, things were bad, but then again, it was a project at beta level in rapid development, and pretty much for early adopters.

I’m maintaining an example polyglot Bazel-built monorepo if anyone wants to follow Bazel’s progress.

I aim to follow best practices and mark out any undocumented gotchas.


What is it? What can I use this for?

Polyglot build system (a la make), designed by Google for large code bases with:

* Assured reproducibility via sandboxing

* Distributed caching

* Distributed execution (experimental)

* Support for long-lived worker processes

* Static analysis of build dependencies

* Uniform CLI for builds and tests

Build definitions and extensions are written in Starlark, a subset of Python.

If you work in a large repo, especially one with multiple languages, you should be interested in Bazel.

Many of Google's OSS projects now build with Bazel (TensorFlow, K8s, Angular).

Buck and Pants are similar tools by Facebook and Twitter respectively, inspired by Bazel's closed source predecessor Blaze.

It’s a build system. That might not be very exciting to you, but for large projects and multi-language projects it is usually much better than other build systems.

It is designed to help give you fast distributed builds with high shared cache hit rates, reproducible builds, and a lot of tools to analyze dependencies.

Personally, I like using it any time a project starts using multiple languages, protobufs, or generated sources.

One big pain point for bazel is installation on Windows, it needs msys2 and need it on the default path. Just a pain to setup. I was thinking of proposing bazel in my org, but after my try on work laptop I abandoned the idea.

It doesn't require msys2 anymore, so you might want to try again.

Does anyone have experience building graalvm native images with Bazel? Or quarkus more specifically?

There are Bazel rules for Graal: https://github.com/andyscott/rules_graal

Am I correct in thinking that Bazel is a fantastic fit for golang with modules?

In theory, yes. In practice... you will need to either a) force everyone in your team to never use native Go tools ever, or b) cobble together third-party tools to bridge the two worlds. Start with https://github.com/bazelbuild/bazel-gazelle and see how deep the rabbit hole goes.

It's possible, but it's not trivial, and not nearly as smooth as using the Go tools in the first place. Unless you have a hair-on-fire problem dealing with cross-language dependencies or insanely long build times, it will probably not be a good use of your time.

Yeah, I think the heuristic should be "Am I forced to migrate to bazel to solve real problems my company is having?"


"Can I migrate because bazel is the new thing"

Is it any easier to port to different operating systems than earlier versions ?

The story would be much promising if Google bought github.

Unfortunately, bazel's future is significantly complicated because of the lack of a repo partner in the ecosystem.

In short, a true single repo experience with bazel would be easily 10x the current one.

Is there something to gain from Basel on small projects?

Not as much. But quality cache invalidation and high level project descriptions are always nice.

Brazil blaze and bazel, someone should tell the story


Bazel is here to stay. Sooo many companies are jumping on this bandwagon [1].

Bazel is better than Maven, Pants... you name it.

Think of this like Protobuf or Kubernetes. It's an open source tool used to build things at scale. There will be lots of users and contributors. A cottage industry will spring up, and this will grow well beyond Google.

[1] https://github.com/bazelbuild/bazel/wiki/Bazel-Users

If only Android team cared to use it instead of Gradle.

Android is probably the only platform where almost at every conference there is a regular talk on how to improve build times.

Not even C++ conferences talk so much about build times.

I find gradle is something like doing thing diligently what should not be done in first place. It seems clear though Oracle/OpenJDK have no interest in putting some fucking options in `javac` itself to compile project/module or even package. Users must wade through maven/gradle crap every time even to compile pure Java projects.

No idea what you're talking about here.

I used to compile Java projects since 1996 until 2004 pretty alright.

And Maven runs circles around Gradle's performance, without requiring a background daemon sucking up at least 2GB, minimum.

Gradle is the only reason Groovy is still somehow relevant and it has to thank Google's Android team for it.

As for the Oracle/OpenJDK remark, it is an open source project, under the GPL with Classpath exception license, it is up to the ones that keep saying that Oracle doesn't do a good job to contribute.

The cottage industry is already springing up. There are a few firms offering Bazel consulting and training. https://bazel.build/experts.html

There is also a considerable work done by Angular team to integrate it on front end builds.

It’s OSS software not a SaaS.

If people use it there will be no deprecation date. You can just fork and continue.

Bazel is too complex to make this practical for most people. It would be like forking Go.

However, it's unlikely to be abandoned.

I’m sorry if that sounds harsh, but that’s at best a naive view. In practice you cannot just fork the codebase and continue maintaining it, that has a high cost that a lot of organization cannot afford. It’s technically true that you can fork the project but in practice forking something that big doesn’t really happen without a big player leading the fork (or some motivated people with enough time and money to dedicate to the fork, which is possible but unlikely).

Hence the “if people use it” qualifier.

Companies relying on Bazel would step in and support it. We’ve seen this play out plenty of times before, e.g. Hudson to Jenkins or OpenOffice to LibreOffice, where a popular OSS product loses a corporate sponsor and continues.

You’re ignoring all those times where that didn’t actually happen. It seems quite risky for an organization to rely on the idea “you can just fork and continue the development yourself in case of issues”. That’s not what happens in practice, taking over the development of something like bazel has a non-negligeable cost. Now for sure you can find some (cherry-picked IMHO) success stories but that’s the exception as far as I can tell.

Hundreds of developers around the Googleplex rejoice.

Isn’t Bazel based off another tool they use internally?

Yes. My guess is that over a long enough period of time, the internal build system will migrate to Bazel. But I don’t think it will happen soon. Large migrations are always a pain, but Bazel does make it easier (since you can isolate dependencies very well and migrate bottom-up).

I don’t work there, but I have talked to Googlers working on build systems.

Bazel is just the open source name of the internal Google build system, it is the same codebase.

From what I understand, the two tools have some very significant differences at this point. They may be the same codebase in a broad sense, and may share a lot of code, but I bet they are using something like Copybara to synchronize them.


I want to try this with a monorepo codebase now but every time I try to get started I feel overwhelmed.

I'm trying to sell my team on it, but none of us has experience with bazel. I'm trying to figure out how to do a small poc where we migrate one intermediate thing in the monorepo to bazel and try to prove out how it can take over everything.

I have personally done this.

- You are going to screw things up a couple times. The documentation for Bazel is not always clear, but you can usually find examples or explanations of complicated stuff on forums somewhere.

- Start with a leaf dependency, something in your codebase that doesn’t depend on anything else. Then work your way up. Just write a WORKSPACE + BUILD.bazel in your root, and then put a BUILD.bazel in the directory for the dependency you are going to work on.

- Look at examples like Tensor Flow, especially for how to handle third-party dependencies (although Tensor Flow is going to do it in a more complicated way).

- Migrate your tests as you go.

- Just run "bazel build //...:all" or "bazel test //...:all" from your project root as you go to make sure things don’t break.

- Undoubtedly it will take a while to develop a good mental model of how Bazel works. For fun, try looking around in the output directories, or look at the sandboxes it creates.

I appreciate your help and was curious if you could field one more specific question:

What if I wanted to fork tensorflow and integrate my project into it, would I need the tensorflow fork as something like a git submodule in my main repo, or is there a way to tell bazel there's another repo somewhere else and where it fits into the graph?

Yes: the internal tool is Blaze

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact