8 ms·

Bazel 8.0

137 points by oftenwrong 2y ago

ashishb 2y ago
I like layering Makefile on top of language-specific build tools like Go, Poetry, cargo, gradle and yarn. Has anyone seen an advantage of using bazel? The learning curve is an issue as well. Blaze (inside Google) was pretty good but not only because of common builds but also because of artifacts that were shared across developers.
- pgmike 2y ago
  I personally like using it for my monorepo projects. At least for Java the Maven dependencies for all of the services get defined in a single top-level MODULE.bazel file which prevents drift in the dependency versions I'm using across the project. I'm sure it's possible to do the exact same thing in <NAME ANOTHER BUILD TOOL>, but I found it to be easy enough in Bazel without the configuration being overly verbose.
  - cmckn 2y ago
    No other tool needed, this is usually solved in Maven projects with a “bill of materials” POM. There’s an example in the Maven docs: https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#bill-of-materials-bom-poms https://maven.apache.org/guides/introduction/introduction-to...
- jeffbee 2y ago
  It has that "worst, except for all the others" quality. It's getting much easier to use with bzlmod, at least for the C++ stuff and especially if you have no weird requirements. I also strongly prefer its toolchain model over others, having recently needed to convert a large C++ project from host GNU tools to hermetic LLVM tools. From time to time I ponder converting projects to cmake but I no longer perceive any advantages.
  - ashishb 2y ago
    Any advantage for using Bazel for newer languages that have a standardized dependency setup like Go or Rust? Or even for JS/TS, where almost everyone uses npm/pnpm/yarn?
    - jeffbee 2y ago
      I personally don't use it for Go or Rust, so maybe the answer is "no"? There's no native builder for C++ so the utility of Bazel for that language is more apparent.
    - phinnaeus 2y ago
      Talk about Bazel in the context of a specific language is missing the point. Bazel is for huge monorepos that aren't based on a single language so that you can have a common interface for all your code in the monorepo regardless of what language individual bits are written in.
    - liuliu 2y ago
      If you have mixed-language needs, yes. But if not, standardized setup is far better with IDE integration (mostly, VS code LSP etc).
    - mrdomino_ 2y ago
      Other folks have said “mixed languages” and “monorepos” and I agree; I’ll also add that it incorporates steps like building Docker/OCI images, tarballs, potentially even deb/rpm packages all in one place, and with the same sandboxing and deterministic reproducibility as you get with the base language rules. Like, “I want an OCI image with these contents” gets you an image with the exact files bit-for-bit from the same source, every time. At least aspirationally.
    - kldx 2y ago
      Cargo workspaces do not support compiling two subprojects for two different targets in a single build invocation. The workaround was (still is?) to use each subproject as a standalone project just for builds. The dependency between these targets is now expressed by the Makefile invoking cargo. This also breaks IDE integrations because your IDE does not know how to ask cargo to pick the right target for each workspace member. Bazel makes this consistent with platforms [0]. I have not used bazel with Rust but it worked fine for my multi-target C++ builds. [0] https://bazel.build/extending/platforms https://bazel.build/extending/platforms
- lihaoyi 2y ago
  Bazel does a lot of non-obvious things that other build tools don't, that become important in larger codebases (100-10,000 active developers): - Automatic caching of everything. In most build tools, caching is opt in, so the core build/compile steps usually end up getting cached but everything else is not and ends up being wastefully recomputed all the time. In Bazel, caching is the default so everything is cached. Even tests are cached so if you run a test twice on the same code and inputs (transitively), the second time it is skipped - Dependency based test selection: You can use bazel query to determine the possible targets and tests affected by a code change, allowing you to trivially set up CI to only run tests downstream of a PR diff and skip unrelated ones. Any large codebase that doesn't use Bazel ends up re-inventing this manually, e.g. consider this code in apache/spark that re-implements this in a Python script that wraps `mvn` or `sbt` (which are build tools that do not provide this functionality) https://github.com/apache/spark/blob/290b4b31bae2e02b648d2c5ef61183f337b18f8f/dev/sparktestsupport/modules.py#L108-L126 https://github.com/apache/spark/blob/290b4b31bae2e02b648d2c5... - Automatic sandboxing of your build steps in CGroup/NS containers, to ensure your build steps do not make use of un-declared files. In most build tools, this kind of mistake results in confusing nondeterministic parallelism and cache invalidation problems down the road, where e.g. your build step may rely on a file on disk but not realize it needs to re-compute when the file changes. In Bazel, these mis-configurations result in a deterministic error up front - At my last job we extended these cgroups to limit CPU/Memory usage as well, which eliminates the noisy neighbour problem and ensures a build step or test gets the same compute footprint whether run alone during development or 96x parallel on a CI worker (https://github.com/bazelbuild/bazel/pull/21322 https://github.com/bazelbuild/bazel/pull/21322). Otherwise it's common for tests to pass when run during manual development then timeout or OOM when run in CI under resource pressure due to other tests hogging the CPU or RAM - Built in support for seamless shared caches: e.g. I compile something on my laptop, you download it to your laptop for usage. This also applies to tests: e.g. if TestFoo was run in CI on master, if i pull master and run all tests without changing the Foo code, TestFoo is skipped and uses the CI result - Built in support for shared compute clusters: e.g. I compile stuff and it automatically happens in the cloud on 96 core machines, or i run a lot of tests (e.g. after a big refactor) on my laptop and it automatically gets farmed out to run 1024x parallel on the cluster (which despite running faster shouldn't cost any more than running 1x parallel, since it costs 1024x as much per-second but should finish in 1024x fewer seconds!) - Support for deep integration between multiple languages: e.g. building a Go binary and Rust library which are both used in a Python executable, which is then tested using a Bash script and deployed as part of a Java backend server. Most build tools support one language well and others not at all, Bazel supports them all (not perfect but adequately) If you never hit these needs, you don't need Bazel, and you probably shouldn't use it because it is ferociously complicated. But if you do hit these needs, most other build tools simply do not cut it at all. We're trying to support some of these use cases and provide a simpler alternative in https://mill-build.org https://mill-build.org, but Bazel really is a high bar to reach in terms of features that support large monorepos
- eddd-ddde 2y ago
  For languages like go and rust there's not a lot of added benefits. For C++ tho it's much easier in my opinion. You can also manage dependencies through it.
SuperSandro2000 2y ago
Bazel is the most distro hostile build system I know.
- elromulous 2y ago
  Could you elaborate?
- orf 2y ago
  Cool? Is that supposed to be a downside?
  - dietr1ch 2y ago
    On one hand, exactly, anything that's grabbed from the distro is actually an issue to reproducibility, but I guess it also means there could be some shim to make installing things easier (assuming `apt/pacman/whatever install blaze` isn't enough to get going, which I guess may be the case since bootstrapping toolchains might not be straightforward and needs configuration/smashing-your-head-against-the-wall for each language, especially if it's not already supported like C++/Java).
- klysm 2y ago
  Hostile to distros how?
dieortin 2y ago
I would encourage anyone in doubt to give Bazel a try, especially for C/C++ projects. I find it to be very robust nowadays, and much saner than CMake or Makefiles. Even if you don’t have a huge monorepo, the reproducibility and speed are great.
- throwup238 2y ago
  Anyone know if it can replace CMake in desktop Qt projects?
- bambambazooka 2y ago
  I really really like the concepts of Bazel, but it feels like you have to jump through a lot of hoops to use it in a non-monorepo setup :/ I hope to find the time to build a PoC one day…
adamgordonbell 2y ago
Here's my guide on when to use bazel, that I wrote back when I was at Earthly. I think it's still very representative. https://earthly.dev/blog/bazel-build/ https://earthly.dev/blog/bazel-build/
- jdlshore 2y ago
  That’s a great article. Thanks for sharing it.
- HackerQED 2y ago
  great article. Read it 1 year ago and still very informative.
- goodklopp 2y ago
  Thanks for sharing. Ended up reading a lot about build systems.
conroy 2y ago
Did a crazy thing and adopted Bazel for our two person company. Took a while to get everything working, but the average build takes about 30 seconds to run, across Go / Rust / TypeScript.
aschleck 2y ago
Bazel makes easy things hard and impossible things possible
- arunix 2y ago
  Perl had a similar motto ("Making Easy Things Easy and Hard Things Possible") https://books.google.com.sb/books?id=3Fc4DQAAQBAJ https://books.google.com.sb/books?id=3Fc4DQAAQBAJ
malkia 2y ago
I've been using it to package the opentelemetry-c++ sdk for us, with single command: bazel run make_otel_sdk it compiles debug, fastdebug, release, then it places the .dll, include/ folders, etc. into single zip. I'm also invoking sentry-cli to obtain all source code used and also place them in https://github.com/malkia/opentelemetry-cpp/actions/runs/12055826824/job/33617036685#step:17:64 https://github.com/malkia/opentelemetry-cpp/actions/runs/120... it produced this https://github.com/malkia/opentelemetry-cpp/releases/tag/v1.18.0-dll.1 https://github.com/malkia/opentelemetry-cpp/releases/tag/v1.... At work in a top level .bazelrc I have caching on the intranet, but on the github I'm relying (not ideal) on the actions/cache through disk_cache blobs - it works, but storing back all "blobs" makes it a bit worse than expected.
- jeffbee 2y ago
  Neat setup. I would point out that this shows some of the problems with the way bazel is managed as a project. You had to fix your build for bazel 8, which isn't great. But the bigger issue is you were forced to adopt rules_cc, which because of their weird adherence to "semantic versioning" means you are now subject to unannounced breaking changes, because rules_cc hasn't hit 1.0. Rules_pkg went through the same problem: it was built in to bazel, then they moved it out into rules_pkg 0.0.1, then the maintainer of rules_pkg felt free to break everything multiple times, mostly for unjustifiable aesthetic reasons.
  - malkia 2y ago
    Yup! I'm using this project as playground to get better at Bazel, and also to evaluate for myself - "is this really a tool I can use at work" - "Not yet, because someone else must be mad as me to love it to use it, instead of use it to solve problems" - especially on Windows :)
deleted 2y ago
[deleted]
tommiegannert 2y ago
> The WORKSPACE mechanism is now disabled by default. > Bzlmod, the new way to manage external dependencies, is turned on by default since Bazel 7.0. > We are aiming to remove WORKSPACE in Bazel 9 completely. Oh, that's good to know. Sorry I wasn't paying attention, M.

137 points by oftenwrong 2y ago

I like layering Makefile on top of language-specific build tools like Go, Poetry, cargo, gradle and yarn. Has anyone seen an advantage of using bazel? The learning curve is an issue as well. Blaze (inside Google) was pretty good but not only because of common builds but also because of artifacts that were shared across developers.

pgmike 2y ago
I personally like using it for my monorepo projects. At least for Java the Maven dependencies for all of the services get defined in a single top-level MODULE.bazel file which prevents drift in the dependency versions I'm using across the project. I'm sure it's possible to do the exact same thing in <NAME ANOTHER BUILD TOOL>, but I found it to be easy enough in Bazel without the configuration being overly verbose.
- cmckn 2y ago
  No other tool needed, this is usually solved in Maven projects with a “bill of materials” POM. There’s an example in the Maven docs: https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#bill-of-materials-bom-poms https://maven.apache.org/guides/introduction/introduction-to...
jeffbee 2y ago
It has that "worst, except for all the others" quality. It's getting much easier to use with bzlmod, at least for the C++ stuff and especially if you have no weird requirements. I also strongly prefer its toolchain model over others, having recently needed to convert a large C++ project from host GNU tools to hermetic LLVM tools. From time to time I ponder converting projects to cmake but I no longer perceive any advantages.
- ashishb 2y ago
  Any advantage for using Bazel for newer languages that have a standardized dependency setup like Go or Rust? Or even for JS/TS, where almost everyone uses npm/pnpm/yarn?
  - jeffbee 2y ago
    I personally don't use it for Go or Rust, so maybe the answer is "no"? There's no native builder for C++ so the utility of Bazel for that language is more apparent.
  - phinnaeus 2y ago
    Talk about Bazel in the context of a specific language is missing the point. Bazel is for huge monorepos that aren't based on a single language so that you can have a common interface for all your code in the monorepo regardless of what language individual bits are written in.
  - liuliu 2y ago
    If you have mixed-language needs, yes. But if not, standardized setup is far better with IDE integration (mostly, VS code LSP etc).
  - mrdomino_ 2y ago
    Other folks have said “mixed languages” and “monorepos” and I agree; I’ll also add that it incorporates steps like building Docker/OCI images, tarballs, potentially even deb/rpm packages all in one place, and with the same sandboxing and deterministic reproducibility as you get with the base language rules. Like, “I want an OCI image with these contents” gets you an image with the exact files bit-for-bit from the same source, every time. At least aspirationally.
  - kldx 2y ago
    Cargo workspaces do not support compiling two subprojects for two different targets in a single build invocation. The workaround was (still is?) to use each subproject as a standalone project just for builds. The dependency between these targets is now expressed by the Makefile invoking cargo. This also breaks IDE integrations because your IDE does not know how to ask cargo to pick the right target for each workspace member. Bazel makes this consistent with platforms [0]. I have not used bazel with Rust but it worked fine for my multi-target C++ builds. [0] https://bazel.build/extending/platforms https://bazel.build/extending/platforms
lihaoyi 2y ago
Bazel does a lot of non-obvious things that other build tools don't, that become important in larger codebases (100-10,000 active developers): - Automatic caching of everything. In most build tools, caching is opt in, so the core build/compile steps usually end up getting cached but everything else is not and ends up being wastefully recomputed all the time. In Bazel, caching is the default so everything is cached. Even tests are cached so if you run a test twice on the same code and inputs (transitively), the second time it is skipped - Dependency based test selection: You can use bazel query to determine the possible targets and tests affected by a code change, allowing you to trivially set up CI to only run tests downstream of a PR diff and skip unrelated ones. Any large codebase that doesn't use Bazel ends up re-inventing this manually, e.g. consider this code in apache/spark that re-implements this in a Python script that wraps `mvn` or `sbt` (which are build tools that do not provide this functionality) https://github.com/apache/spark/blob/290b4b31bae2e02b648d2c5ef61183f337b18f8f/dev/sparktestsupport/modules.py#L108-L126 https://github.com/apache/spark/blob/290b4b31bae2e02b648d2c5... - Automatic sandboxing of your build steps in CGroup/NS containers, to ensure your build steps do not make use of un-declared files. In most build tools, this kind of mistake results in confusing nondeterministic parallelism and cache invalidation problems down the road, where e.g. your build step may rely on a file on disk but not realize it needs to re-compute when the file changes. In Bazel, these mis-configurations result in a deterministic error up front - At my last job we extended these cgroups to limit CPU/Memory usage as well, which eliminates the noisy neighbour problem and ensures a build step or test gets the same compute footprint whether run alone during development or 96x parallel on a CI worker (https://github.com/bazelbuild/bazel/pull/21322 https://github.com/bazelbuild/bazel/pull/21322). Otherwise it's common for tests to pass when run during manual development then timeout or OOM when run in CI under resource pressure due to other tests hogging the CPU or RAM - Built in support for seamless shared caches: e.g. I compile something on my laptop, you download it to your laptop for usage. This also applies to tests: e.g. if TestFoo was run in CI on master, if i pull master and run all tests without changing the Foo code, TestFoo is skipped and uses the CI result - Built in support for shared compute clusters: e.g. I compile stuff and it automatically happens in the cloud on 96 core machines, or i run a lot of tests (e.g. after a big refactor) on my laptop and it automatically gets farmed out to run 1024x parallel on the cluster (which despite running faster shouldn't cost any more than running 1x parallel, since it costs 1024x as much per-second but should finish in 1024x fewer seconds!) - Support for deep integration between multiple languages: e.g. building a Go binary and Rust library which are both used in a Python executable, which is then tested using a Bash script and deployed as part of a Java backend server. Most build tools support one language well and others not at all, Bazel supports them all (not perfect but adequately) If you never hit these needs, you don't need Bazel, and you probably shouldn't use it because it is ferociously complicated. But if you do hit these needs, most other build tools simply do not cut it at all. We're trying to support some of these use cases and provide a simpler alternative in https://mill-build.org https://mill-build.org, but Bazel really is a high bar to reach in terms of features that support large monorepos
eddd-ddde 2y ago
For languages like go and rust there's not a lot of added benefits. For C++ tho it's much easier in my opinion. You can also manage dependencies through it.

SuperSandro2000 2y ago

Bazel is the most distro hostile build system I know.

elromulous 2y ago
Could you elaborate?
orf 2y ago
Cool? Is that supposed to be a downside?
- dietr1ch 2y ago
  On one hand, exactly, anything that's grabbed from the distro is actually an issue to reproducibility, but I guess it also means there could be some shim to make installing things easier (assuming `apt/pacman/whatever install blaze` isn't enough to get going, which I guess may be the case since bootstrapping toolchains might not be straightforward and needs configuration/smashing-your-head-against-the-wall for each language, especially if it's not already supported like C++/Java).
klysm 2y ago
Hostile to distros how?

dieortin 2y ago

I would encourage anyone in doubt to give Bazel a try, especially for C/C++ projects. I find it to be very robust nowadays, and much saner than CMake or Makefiles. Even if you don’t have a huge monorepo, the reproducibility and speed are great.

throwup238 2y ago
Anyone know if it can replace CMake in desktop Qt projects?
bambambazooka 2y ago
I really really like the concepts of Bazel, but it feels like you have to jump through a lot of hoops to use it in a non-monorepo setup :/ I hope to find the time to build a PoC one day…

adamgordonbell 2y ago

Here's my guide on when to use bazel, that I wrote back when I was at Earthly. I think it's still very representative. https://earthly.dev/blog/bazel-build/ https://earthly.dev/blog/bazel-build/

jdlshore 2y ago
That’s a great article. Thanks for sharing it.
HackerQED 2y ago
great article. Read it 1 year ago and still very informative.
goodklopp 2y ago
Thanks for sharing. Ended up reading a lot about build systems.

conroy 2y ago

Did a crazy thing and adopted Bazel for our two person company. Took a while to get everything working, but the average build takes about 30 seconds to run, across Go / Rust / TypeScript.

aschleck 2y ago

Bazel makes easy things hard and impossible things possible

arunix 2y ago
Perl had a similar motto ("Making Easy Things Easy and Hard Things Possible") https://books.google.com.sb/books?id=3Fc4DQAAQBAJ https://books.google.com.sb/books?id=3Fc4DQAAQBAJ

malkia 2y ago

I've been using it to package the opentelemetry-c++ sdk for us, with single command: bazel run make_otel_sdk it compiles debug, fastdebug, release, then it places the .dll, include/ folders, etc. into single zip. I'm also invoking sentry-cli to obtain all source code used and also place them in https://github.com/malkia/opentelemetry-cpp/actions/runs/12055826824/job/33617036685#step:17:64 https://github.com/malkia/opentelemetry-cpp/actions/runs/120... it produced this https://github.com/malkia/opentelemetry-cpp/releases/tag/v1.18.0-dll.1 https://github.com/malkia/opentelemetry-cpp/releases/tag/v1.... At work in a top level .bazelrc I have caching on the intranet, but on the github I'm relying (not ideal) on the actions/cache through disk_cache blobs - it works, but storing back all "blobs" makes it a bit worse than expected.

jeffbee 2y ago
Neat setup. I would point out that this shows some of the problems with the way bazel is managed as a project. You had to fix your build for bazel 8, which isn't great. But the bigger issue is you were forced to adopt rules_cc, which because of their weird adherence to "semantic versioning" means you are now subject to unannounced breaking changes, because rules_cc hasn't hit 1.0. Rules_pkg went through the same problem: it was built in to bazel, then they moved it out into rules_pkg 0.0.1, then the maintainer of rules_pkg felt free to break everything multiple times, mostly for unjustifiable aesthetic reasons.
- malkia 2y ago
  Yup! I'm using this project as playground to get better at Bazel, and also to evaluate for myself - "is this really a tool I can use at work" - "Not yet, because someone else must be mad as me to love it to use it, instead of use it to solve problems" - especially on Windows :)

deleted 2y ago

[deleted]

tommiegannert 2y ago

> The WORKSPACE mechanism is now disabled by default. > Bzlmod, the new way to manage external dependencies, is turned on by default since Bazel 7.0. > We are aiming to remove WORKSPACE in Bazel 9 completely. Oh, that's good to know. Sorry I wasn't paying attention, M.