6 ms·

Bazel can be clunky, but not having some bazel equivalent can have very significant costs that are easy to get accustomed to or overlook. Things like engineers

by dub 4y ago

Bazel can be clunky, but not having some bazel equivalent can have very significant costs that are easy to get accustomed to or overlook. Things like engineers losing time wondering why their node dependencies weren't correctly installed, or dealing with a pre-commit check that reminds them they didn't manually regenerate the generated files, or having humans write machine-friendly configuration that's not actually human-friendly because there's no easy way to introduce custom file transformations during the build. Bazel doesn't spark joy for me and I wouldn't say I look forward to using it, but personally I would still always choose it for a codebase that's going to have multiple developers and last a long time. It's vastly easier to go with bazel from the beginning than to wish you could switch to it and realize people have already introduced a million circular dependencies and it's going to be a multi-month or multi-year process to migrate to it.

trasz 4y ago
>there's no easy way to introduce custom file transformations during the build. Every real-world build system I’ve seen provides that functionality. In particular a standard make(1) can do it just fine.
- dub 4y ago
  make(1) has no native support for giving each build rule its own sandboxed view of the filesystem like bazel does. If I could have a wish to upgrade file transformation with make(1), I'd probably want a widely-available, standard, simple command to make a rule-specific virtual filesystem that overlays a configurable read-only/copy-on-write view of selected existing files or directories behind a writable rule-specific output directory.
  - trasz 4y ago
    Why would you want it to mess with sandboxes? It’s a build system. There are other mechanisms for sandboxing, no need to reinvent the wheel. Basel is as non-standard as it gets - essentially yet another Google’s case of NIH - but apart from that, how is an ad-hoc single-use pseudo-language better than reusing standard mechanism? To me it’s just a bad design.
    - joshuamorton 4y ago
      Are you calling bazel a nonstandard single use pseudo language, but make a standard tool? That's just an argument from tradition. And you want sandboxing because that's what gets you good caching. The value of bazel is never having to run make clean because artifacts aren't correctly being built from cache. Having no distinction between clean and incremental builds is really nice.
      - trasz 4y ago
        That is an argument from tradition - which you yourself have brought up, calling Bazel a "standard way". >The value of bazel is never having to run make clean because artifacts aren't correctly being built from cache. You can get that with make(1) too, check out FreeBSD's META_MODE for one example. And it didn't require reinventing the wheel.
        joshuamorton 4y ago
        > which you yourself have brought up, calling Bazel a "standard way". I didn't do any such thing. My point is simply that make and bazel are similarly "nonstandard single use pseudo languages". In many ways, bazel is superior to make from a language perspective (it resembles other languages more closely, being a dialect of python, and avoids the loadbearing tab issue), so I think I could make the argument that bazel is in many ways less nonstandard, but make is certainly more common than bazel, so it could go either way. > You can get that with make(1) too, check out FreeBSD's META_MODE for one example. This suffers from the same issues that natural make does (notably the whole mtime thing). See https://apenwarr.ca/log/20181113 https://apenwarr.ca/log/20181113 for a much better explanation than I can provide as to why make's entire model of "change" is irreparably broken, and why hash (+sandbox!) based approaches (which bazel and redo and nix and cargo and nearly every other modern build tool use) are far superior. > And it didn't require reinventing the wheel. You call inventing a new syscall to not-even-fully fix a limitation of the tool not reinventing the wheel? Like I guess its not, it's just like building a weird grand shrine around the broken wheel. It's far worse. I don't want to need to change my operating system to have make work better, but still worse than the alternatives. That's simply not a compelling argument.
reissbaker 4y ago
In my experience, Bazel is a net negative for most teams. Pretty much every JS engineer is familiar with npm; a tiny fraction are familiar with Bazel. Ditto with pip, cargo, etc. And it doesn't solve the hard part of the build process, which is distributed builds. Most of the user-perceptible value of Blaze is making builds fast by farming them out to a zillion machines — that's why it's called "blaze," because it's fast! — and Bazel doesn't do that for you. And it's clunky, and you need to teach every new hire how to use it. The juice just isn't worth the squeeze. Just write an adapter in your build infra for the well known tools and be done with it. You'll get much more value putting work into something else, like code review tools, testing, dev environments, staging...
- dilyevsky 4y ago
  There are many remote build backends for bazel available - some open source and some proprietary (EngFlow and Google’s RBE). Even without a build farm it will still be a huge performance boost due to caching - something make and co (recommended in sibling thread) can not do along with a bunch of other stuff that bazel does for you.
  - reissbaker 4y ago
    I've heard the "huge performance boost" argument before, and in my experience it's often kinda marginal, or even negative because the native build tools have been optimized for their specific task. (Sure, I'll give you that it's better than Make, but is it faster than the Go compiler?) I wasn't familiar with EngFlow; it looks like it's a startup that raised seed funding less than a year ago. I think what you're referencing with "Google RBE" is a Google Cloud project now rebranded to "Cloud Builds" — which supports the underlying native tools like npm, pip, etc without requiring you to switch to Bazel. Bazel is better than Make for building large C/C++ projects (although it's hardly the only game in town for "better than Make"). But aside from that use case, in my experience it's not really worth the hassle. You can get most of the benefits you want without using it, and people are already going to be familiar with the tools native to the ecosystems they work in like pip, npm, etc.
EdwardDiego 4y ago
Given how Borg unleashed on the world as K8s came to dominate orchestration, Bazel's lack of similar uptake outside of the Google walled garden is indicative that it's not solving problems for non-Google teams significantly better than existing build tools. Look at how much pain many companies have gone through to move to K8s from existing infrastructure, there is perceived value driving that. Bazel lacks that perception of value.

by dub 4y ago

trasz 4y ago

>there's no easy way to introduce custom file transformations during the build. Every real-world build system I’ve seen provides that functionality. In particular a standard make(1) can do it just fine.

dub 4y ago
make(1) has no native support for giving each build rule its own sandboxed view of the filesystem like bazel does. If I could have a wish to upgrade file transformation with make(1), I'd probably want a widely-available, standard, simple command to make a rule-specific virtual filesystem that overlays a configurable read-only/copy-on-write view of selected existing files or directories behind a writable rule-specific output directory.
- trasz 4y ago
  Why would you want it to mess with sandboxes? It’s a build system. There are other mechanisms for sandboxing, no need to reinvent the wheel. Basel is as non-standard as it gets - essentially yet another Google’s case of NIH - but apart from that, how is an ad-hoc single-use pseudo-language better than reusing standard mechanism? To me it’s just a bad design.
  - joshuamorton 4y ago
    Are you calling bazel a nonstandard single use pseudo language, but make a standard tool? That's just an argument from tradition. And you want sandboxing because that's what gets you good caching. The value of bazel is never having to run make clean because artifacts aren't correctly being built from cache. Having no distinction between clean and incremental builds is really nice.
    - trasz 4y ago
      That is an argument from tradition - which you yourself have brought up, calling Bazel a "standard way". >The value of bazel is never having to run make clean because artifacts aren't correctly being built from cache. You can get that with make(1) too, check out FreeBSD's META_MODE for one example. And it didn't require reinventing the wheel.
      - joshuamorton 4y ago
        > which you yourself have brought up, calling Bazel a "standard way". I didn't do any such thing. My point is simply that make and bazel are similarly "nonstandard single use pseudo languages". In many ways, bazel is superior to make from a language perspective (it resembles other languages more closely, being a dialect of python, and avoids the loadbearing tab issue), so I think I could make the argument that bazel is in many ways less nonstandard, but make is certainly more common than bazel, so it could go either way. > You can get that with make(1) too, check out FreeBSD's META_MODE for one example. This suffers from the same issues that natural make does (notably the whole mtime thing). See https://apenwarr.ca/log/20181113 https://apenwarr.ca/log/20181113 for a much better explanation than I can provide as to why make's entire model of "change" is irreparably broken, and why hash (+sandbox!) based approaches (which bazel and redo and nix and cargo and nearly every other modern build tool use) are far superior. > And it didn't require reinventing the wheel. You call inventing a new syscall to not-even-fully fix a limitation of the tool not reinventing the wheel? Like I guess its not, it's just like building a weird grand shrine around the broken wheel. It's far worse. I don't want to need to change my operating system to have make work better, but still worse than the alternatives. That's simply not a compelling argument.

reissbaker 4y ago

In my experience, Bazel is a net negative for most teams. Pretty much every JS engineer is familiar with npm; a tiny fraction are familiar with Bazel. Ditto with pip, cargo, etc. And it doesn't solve the hard part of the build process, which is distributed builds. Most of the user-perceptible value of Blaze is making builds fast by farming them out to a zillion machines — that's why it's called "blaze," because it's fast! — and Bazel doesn't do that for you. And it's clunky, and you need to teach every new hire how to use it. The juice just isn't worth the squeeze. Just write an adapter in your build infra for the well known tools and be done with it. You'll get much more value putting work into something else, like code review tools, testing, dev environments, staging...

dilyevsky 4y ago
There are many remote build backends for bazel available - some open source and some proprietary (EngFlow and Google’s RBE). Even without a build farm it will still be a huge performance boost due to caching - something make and co (recommended in sibling thread) can not do along with a bunch of other stuff that bazel does for you.
- reissbaker 4y ago
  I've heard the "huge performance boost" argument before, and in my experience it's often kinda marginal, or even negative because the native build tools have been optimized for their specific task. (Sure, I'll give you that it's better than Make, but is it faster than the Go compiler?) I wasn't familiar with EngFlow; it looks like it's a startup that raised seed funding less than a year ago. I think what you're referencing with "Google RBE" is a Google Cloud project now rebranded to "Cloud Builds" — which supports the underlying native tools like npm, pip, etc without requiring you to switch to Bazel. Bazel is better than Make for building large C/C++ projects (although it's hardly the only game in town for "better than Make"). But aside from that use case, in my experience it's not really worth the hassle. You can get most of the benefits you want without using it, and people are already going to be familiar with the tools native to the ecosystems they work in like pip, npm, etc.

EdwardDiego 4y ago

Given how Borg unleashed on the world as K8s came to dominate orchestration, Bazel's lack of similar uptake outside of the Google walled garden is indicative that it's not solving problems for non-Google teams significantly better than existing build tools. Look at how much pain many companies have gone through to move to K8s from existing infrastructure, there is perceived value driving that. Bazel lacks that perception of value.