8 ms·

Bazel – Correct, reproducible, fast builds for everyone

625 points by drivebyubnt 11y ago

ngd 11y ago
This is an open sourcing of Google's internal build tool. I know it as Blaze, which Bazel is an anagram of. Many files in the source have references to Blaze.
pron 11y ago
How does it compare with Java 9's sjavac (http://stackoverflow.com/a/26424760/750563 http://stackoverflow.com/a/26424760/750563)? EDIT: I fully understand that this is a build tool for multiple languages. But its raison d'etre is speed. So I'm asking what techniques does Bazel use to accelerate builds and how do they differ from those used by sjavac, which is also designed to accelerate builds of huge projects?
- Sphax 11y ago
  I don't think they're even related, Bazel is a general build tool, sjavac looks like a smarter Java compiler ?
  - pron 11y ago
    ... that exploits parallelism and caching (and a hot VM) to accelerate build of huge projects, and supports build clusters.
    - untothebreach 11y ago
      huge Java projects...why would you compare a general build tool to a language-specific one?
      - pron 11y ago
        What difference does that make? I am not saying you could replace this tool with sjavac, but as they both tout speed as their main feature (and justification), I am wondering whether they both employ the same techniques or different ones.
    - skj 11y ago
      Inside Google we do that as well, but it's actually distinct from bazel. Bazel is a build language, and I believe they provide a reference implementation. It's not unimaginable (and as I mentioned, it has been done internally) to use caching, incremental, distributed builds. In fact, it was originally designed with those goals in mind.
- hanwenn 11y ago
  I work on Bazel. Bazel also builds other languages, such as C++ and Objective-C. We do invoke the Java compiler through a wrapper of our own. We think we can make that work as a daemon process to benefit from a hot JVM, but haven't gotten round to that.
  - pron 11y ago
    Do you also use timestamps like sjavac or some other mechanism, like hashing?
    - hanwenn 11y ago
      Bazel uses checksums to determine if builds are up-to-date, but shortcuts the checksumming if metadata (timestamp, filesize) have not changed between builds.
      - pron 11y ago
        Your FAQ says this: Bazel configuration files are much more structured than Gradle's, letting Bazel understand exactly what each action does. This allows for more parallelism and better reproducibility. Could you please elaborate on that (i.e. with regards to both parallelism and reproducibility)?
        hanwenn 11y ago
        I'm not a Gradle expert, so take this with a grain of salt. Gradle a single-threaded execution in some parts (I believe it may be configuration?), because the build-rules are written in a full blown language with access to the filesystem, and internal parts of Gradle. In Bazel, rules have to declare inputs and outputs, and this can be enforced with sandboxing. This allows to predict that two rules do not interfere with each other, so we know we can run them in parallel. Our extension language disallows direct access to the file system, and also forbids access to other sources of non-determinism, such as hash tables and the clock.
        11y ago
habosa 11y ago
Working at Google, Blaze is one of the technologies that amazes me most. Any engineer can build any Google product from source on any machine just by invoking a Blaze command. I may not want to build GMail from source (could take a while) but it's awesome to know that I can. I think this could be hugely useful to very large open source projects (like databases or operating systems) that may be intimidating for contributors to build and test.
- middleclick 11y ago
  So the builds are reproducible automatically?
  - hanwenn 11y ago
    See http://bazel.io/docs/FAQ.html http://bazel.io/docs/FAQ.html, "Will Bazel make my builds reproducible automatically? For Java and C++ binaries, yes, assuming you do not change the toolchain. If you have build steps that involve custom recipes (eg. executing binaries through a shell script inside a rule), you will need to take some extra care: Do not use dependencies that were not declared. Sandboxed execution (–spawn_strategy=sandboxed, only on Linux) can help find undeclared dependencies. Avoid storing timestamps in generated files. ZIP files and other archives are especially prone to this. Avoid connecting to the network. Sandboxed execution can help here too. Avoid processes that use random numbers, in particular, dictionary traversal is randomized in many programming languages."
    - hansjorg 11y ago
      When you say Java, does that include Android? I see that Android is supported, but couldn't find anything about reproducibility. Reproducible Android builds would be very interesting.
      - esrauch 11y ago
        http://bazel.io/docs/roadmap.html http://bazel.io/docs/roadmap.html It looks like they only have iOS and not Android in this first release, but they are planning on adding Android support ~June of this year.
latkin 11y ago
Correct, reproducible, fast builds for everyone not running Windows
- izacus 11y ago
  Not really sure why this is getting downvoted - it's kind of an important detail when choosing a build system if you have to do multiplatform deployment.
  - istvan__ 11y ago
    I am pretty sure it has nothing to do with the average population being biased.
  - melling 11y ago
    I didn't downvote him but it sounds like "he's driving angry". He does work for that "evil" company in Redmond. :-). Maybe he can help make F# a great cross-platform language and increase the goodwill? Microsoft did an incredible job with F# but it really only runs well on Windows.
    - MichaelGG 11y ago
      What particular problems on non-Windows do you have with F#? I've been running F# applications in telecom, in production, on Mono, for several years. F# doesn't really seem to be the factor there at all, it's just general .NET support. In fact, F# can do a bit better than C#, as F# actually includes a static linker.
      - melling 11y ago
        Strange, most blogs that I've read discuss the pain involved. http://spin.atomicobject.com/2014/06/06/f-sharp-mono-unix/ http://spin.atomicobject.com/2014/06/06/f-sharp-mono-unix/ I guess it's the same with any new product or language: when there are a lot more great testimonials than cries of pain, then it's actually safe to use that product.
pacala 11y ago
A couple of questions: * If I have a Maven-based project with heavy reliance on pre-built jars from Maven Central, what's the recipe to port it to Bazel? * Related, if I have multiple github repos, say a couple open source libraries and a couple private repos, what's a good recipe in conjunction to Bazel?
- deleted 11y ago
  [deleted]
- kchod 11y ago
  Check out http://bazel.io/docs/build-encyclopedia.html#maven_jar http://bazel.io/docs/build-encyclopedia.html#maven_jar. In the root of your build, specify the jars you want from maven and then add them as dependencies in your BUILD files. The first time you run "bazel build", they'll be downloaded and cached from then on. It's somewhat limited in functionality at the moment, but should work for basic "download and depend on a jar". For multiple Github repos, use http://bazel.io/docs/build-encyclopedia.html#http_archive http://bazel.io/docs/build-encyclopedia.html#http_archive or http://bazel.io/docs/build-encyclopedia.html#new_http_archive http://bazel.io/docs/build-encyclopedia.html#new_http_archiv... (depending on if it's a Bazel repository or not). Let us know if you have any questions or issues!
  - pacala 11y ago
    Thanks for the tips. I'm super-hyped that blaze was open sourced, it is one of the best systems I've ever had the pleasure to work with. A couple more questions :) * Any pointers for adding Scala (sbt?) support? I'd start here: http://bazel.io/docs/skylark/rules.html http://bazel.io/docs/skylark/rules.html. * Suppose I develop using multiple repos and http_archive. I'd like to make changes both to a library and to a project that depends on it simultaneously, without committing the library patches to master github repo just yet. Is there a way to configure the http_archive, let's say by saying "bazel --mode=local", and have it customize the remote archive http to use a different url (say, my github's fork instead of the master github) for that build?
    - 11y ago
Zariel 11y ago
Is this the tool that Google uses to build its Golang source? Or is that something else which is not available?
- jwcrux 11y ago
  You can build golang from source pretty easily. If I remember right, it's just downloading the tarball and running ./all.bash or something like that.
- kchod 11y ago
  The Golang source code for the server code at google is built with this tool. The rules that accomplish this are rather complex due to their interactions with our C++ libraries, and predates the open source "Go" tool. The experience with the Google internal rules, motivated some of the choices in the "go" tool, I believe. If you're interested, hanwen wrote a bunch rules with similar semantics as the internal rules, see https://github.com/google/bazel/tree/master/base_workspace/examples/go/ https://github.com/google/bazel/tree/master/base_workspace/e... . It would be nice to make these semantics match the external ones better, but it requires us to open up more tooling, so people won't need to write BUILD files.
  - zzzhao 11y ago
    In what cases would using Bazel make sense to build Go projects? If they're extremely large? If they have a lot of dependencies on code in other languages? If you need sophisticated build/release tooling? BTW, thanks for the release! Will have a fun time digging through this over the next few days. I heard some murmurs that Blaze was going to be open sourced from around the watercooler but didn't think it'd be so soon.
    - hanwenn 11y ago
      I guess if you want to integrate Go tools with builds in other languages. If you are using pure Go for your entire ecosystem, there is not much point in using Bazel, as the "go" tool is very capable for that scenario.
  - 11y ago
yarapavan 11y ago
Surprisingly, significant parts of the code is not open source. According to this page, http://bazel.io/docs/governance.html http://bazel.io/docs/governance.html, Is Bazel developed fully in the open? Unfortunately not. We have a significant amount of code that is not open source; in terms of rules, only ~10% of the rules are open source at this point. We did an experiment where we marked all changes that crossed the internal and external code bases over the course of a few weeks, only to discover that a lot of our changes still cross both code bases.
- deleted 11y ago
  [deleted]
- Symmetry 11y ago
  Do they mean that 10% of the original Blaze rules are now open source or that 10% of the Bazel rules they've released are open source?
  - DannyBee 11y ago
    The former.
- spankalee 11y ago
  I don't think you're interpreting that section quite right. That section is talking about whether or not Bazel is fully _developed_ in the open, and the answer is "Unfortunately not". What they mean is that changes to the internal source of Blaze often involve changes to both the open sourced part, which is Bazel, and the closed parts, which are additional rules that are neither open sourced, nor included in Bazel (Blaze has about 5x as many rules as Bazel). It's best to make atomic changes, so rather than split the changes, review and submit the open source changes externally, and the closed rules changes internally (which would complicate reviews, testing, syncing and rollbacks), then pull in the external changes, they submit these cross-code-base changes internally, then dump the change into the external repo. The next paragraph on that page makes it clear that the code is open, even if not all of the development process is. To be clear, all of Bazel is open source and the source is available here: https://github.com/google/bazel https://github.com/google/bazel
cies 11y ago
What would be needed to get this to work with Haskell? I read in the "Getting started": > You can now create your own targets and compose them. So does this mean it is a replacement for `make`? => Yes Found the answer here: http://bazel.io/docs/FAQ.html http://bazel.io/docs/FAQ.html
- kchod 11y ago
  If you're interested in adding rules for a new language, check out Skylark: http://bazel.io/docs/skylark/concepts.html http://bazel.io/docs/skylark/concepts.html.
zobzu 11y ago
I had a bit of a read but I didn't find where it explains (code or doc) how it achieves reproducible builds. It seems like a stricter, huge make-like harness (in fact it reminds me of the mozilla firefox python build system a bit). It's not bad by any means, but it seems like to me it doesn't "magically" fix the "be reproducible" problem at all (which is what it seem to claim) Am I missing something?
- lberki 11y ago
  You are absolutely correct: Bazel by itself does not make your builds reproducible. If a tool calls rand() or bakes the current time into its output, reproducibility goes out of the window. What Bazel does, however, is to make it possible to run build steps in a sandbox (although the current one is kinda leaky) so that your build is isolated from the environment and thus behaves in the same way on any computer. It also tracks dependencies correctly so that it knows when a specific action needs to be re-run. This makes it possible to diagnose non-reproducible build steps easily. At Google, the hit rate of our distributed build cache usually floats around 99%, and this would be impossible without reproducible build steps.
  - jrockway 11y ago
    Does work done by Debian to make Linux packages build reproducibly help Bazel? https://wiki.debian.org/ReproducibleBuilds https://wiki.debian.org/ReproducibleBuilds Would Bazel help with the remaining long tail of packages in Debian?
- nevir 11y ago
  You're right - it doesn't magically solve build reproducibility. Bazel pushes you towards a build configuration where you have to describe (in a terse way) the entire dependency graph of what is being built. It allows Bazel to be smart about where in the graph things are stale. If you run a script that outputs intermediate files, Bazel needs to know about that scripts inputs and outputs. And it works better if it knows them ahead of time.
- skybrian 11y ago
  It's not magic; you have to work at it. (For example, make sure that zip doesn't put timestamps in the file.) But it's designed so that code generators should act as pure functions from input files to output files, and many generators actually are, especially the built in ones. If you do this then the build system will help you. Writing generators to run this way is kind of a pain, actually, sort of like writing code to run in a sandbox. Also, the generators themselves must be checked in, and often built from source. But we consider the results worth it.
setheron 11y ago
If i'm sticking to primarily Java; is there a benefit to using Bazel as opposed to Maven / Gradle / Sbt ?
- astral303 11y ago
  At first impression, unless you have a single gigantic source code base, unlikely. From their FAQ: >> "Gradle: Bazel configuration files are much more structured than Gradle's, letting Bazel understand exactly what each action does. This allows for more parallelism and better reproducibility" The value of "more parallelism" depends on the complexity of your Java source code base. I can easily imagine why this extra structure can lead to more parallelism. However, I am not buying "better reproducibility" without justification or explanation. I've had very reproducible Maven builds for years (and I don't see how Gradle would be different). So I would love to know which aspects are improved upon with this structure, if someone could expand or explain. Finally, I'm very wary of "much more structure". The worst thing about Maven is its extreme insistence on structure and schema and very specific architecture of your build tasks and components. In contrast, with Gradle, you can freely shape your build scripts to reflect the "build architecture" of your source tree in a minimal, maintainable way. Furthermore, when your application's needs change, refactoring your build is far easier in Gradle, thanks to its internal-DSL style (the build script is code). If the structure isn't "free", you pay for structure with reduced build script development speed. For Google, it's a tradeoff worth having with that massive source tree.
  - ulfjack 11y ago
    I work on Bazel. We've put a bunch of work into making sure that we know about every file that goes into the Java compilation, and if any of them changes (and only then) do we recompile. Within Google, we use a form of sandboxing to enforce that. You're also right that it isn't free - we have reason to believe that larger projects and larger teams will see benefits from using Bazel. Use your best judgement.
  - asuffield 11y ago
    blaze is nothing remotely like the wall of cruft that maven forces you to climb for everything you do. I would describe it as "almost entirely unlike maven".
thechao 11y ago
I've been burned by so many build tools over the years. I've finally settled (for C/++/asm) on the combination of Make + ccache: I build a _very_ paranoid Makefile that recompiles everything if it feels like anything changes. For instance, every rule that compiles a C/++ file is invoked if _any_ header/inc/template file changes. I let ccache do the precise timestamp/check-sum based analysis. The result is that (for large builds < 10MMLOC) I rarely wait for more than a few hundred milliseconds on incremental, _and_ I have confidence that I never miscompile. I just wish that I had a high-performance replacement for linking that was cross-platform (deterministic mode for ar), and for non-C/++ flows. Writing a deterministic ar is about 20 lines of C-code, but then I have to bake that into the tool in awkward ways. For generalized flows, I've looked at fabricate.py as a ccache replacement, but the overhead of spinning up the Python VM always nukes performance.
- beagle3 11y ago
  > I build a _very_ paranoid Makefile that recompiles everything if it feels like anything changes. Do you have some kind of way to verify that your makefile dependencies conform to your source dependencies? Is clang/gcc tracking sufficient for your use case? What about upgrading the compiler itself, does your makefile depend on that? If so, how? Have you considered tup[0]? Or djb-redo[1]? Both seem infinitely better than Make if you are paranoid. tup even claims to work on Windows, although I have no idea how they do that (or what the slowdown is like). Personally, I'm in the old Unix camp of many-small-executables, non of which goes over 1M statically linked (modern "small"), so it's rarely more than 3 secs to rebuild an executable from scratch. > (deterministic mode for ar) Why do you care about ar determinism? Shouldn't it be ld determinism you are worried about? [0] http://gittup.org/tup/ http://gittup.org/tup/ [1] https://github.com/apenwarr/redo https://github.com/apenwarr/redo
  - thechao 11y ago
    > Do you have some kind of way to verify that your makefile dependencies conform to your source dependencies? Nope. I explicitly use a conservative approximation—this guarantees correctness, over speed. Building everything every time with a clean tree is where I begin; I start optimizing after that. > Is clang/gcc tracking sufficient for your use case? What about upgrading the compiler itself, does your makefile depend on that? If so, how? Self-rewriting Makefiles (to consume the .d files), combined with the cleaning necessary for them, become a large technical debt—especially given the complexity of the Makefile needed to generate them. Modern CCen just aren't capable of this. Perhap Doug Gregor's module system will land in C21/C++21, and we'll see some good, then. > Have you considered tup[0]? Or djb-redo[1]? Yes. They are both don't provide significantly better correctness guarantees combined with sufficiently better performance to justify the cost to porting to older Unixen. (This is a consensus opinion at my shop; I, personally, enjoy tup.) > Why do you care about ar determinism? Shouldn't it be ld determinism you are worried about? Determinism let's me cache *.o/a/so/dylib/exe/whatnot without getting false-positives due to time-stamp changes and owner/group permissions in the obj/ar files (see ar(1)). ld is deterministic under all the CCen I use by setting the moral-equivalent of -frandom-seed.
malkia 11y ago
Oh, but my favourite option "blaze menu" is missing :)
- asuffield 11y ago
  Huh. I never knew that was there. I'll remember this next time I'm around Charleston.
mashraf 11y ago
Is Google departing from just throwing white papers over the wall and let community figure out the implementation details? blaze white paper was dropped a while ago and there are already two clones in Pants and Buck at Twitter and FB. It would be interesting to see how far off clones are from original implementation.
- cbgb 11y ago
  Do you have a link to that white paper? A quick search on their research site doesn't really yield any results.
  - kchod 11y ago
    I'm a developer on Bazel, and AFAIK there is no white paper. We definitely don't want to "throw it over the wall," we're going to try to push more and more development into the open over time.
    - cbgb 11y ago
      Ah, OK, thanks. I do get a kick out of reading various Google white papers (GFS, Spanner, and the Multi-Paxos implementation for Chubby come to mind), so I'd definitely be interested in the prospect of a future white paper :).
    - bruckie 11y ago
      There's not a white paper, but there's this series of posts: http://google-engtools.blogspot.com/2011/06/build-in-cloud-accessing-source-code.html http://google-engtools.blogspot.com/2011/06/build-in-cloud-a... (and an accompanying presentation)
rquirk 11y ago
What is this lameness? https://github.com/google/bazel/tree/master/third_party https://github.com/google/bazel/tree/master/third_party - why not use gradle repos to download jars with known hashes? Sticking all those jars in the git repo is just... well, I expected better from Google.
- MichaelGG 11y ago
  Well keeping dependencies in source means no third party dependency at build time, right?
  - rquirk 11y ago
    Right... but this will be an anchor on adoption. I can see why e.g. the Android build system does the same thing since it's all off in its own world anyway. I doubt you'd be popular with Linux distro packagers if you required bazel for some C library.
    - ulfjack 11y ago
      We realize that not everyone wants to check in all of their dependencies, and that's cool too. Check out the build encyclopedia for rules to talk to 'plain http' and maven repositories.
- sarnowski 11y ago
  Tbh, if your company relies on this software, I would also make sure that it cannot just vanish - and thats the most efftive solution. Artifacts can disappear from the internet and you don't know if the downloaded stuff is still the same as before. Especially, if you look outside of the maven ecosystem, but even there you have to rely on apache and their partners. An outage can mean that you cannot deploy critical bugfixes to your platform.
  - adambatkin 11y ago
    This is why you should have repository manager like JFrog Artifactory or Sonatype Nexus which can transparently proxy third-party repositories (like Maven Central).
eisar 11y ago
Made by the arrogant Google assholes? No thanks. I'll stick to pants. Disclaimer: I work for TAGA (The Arrogant Google Assholes)
shmerl 11y ago
> Why doesn't Google use …? Make, Ninja: These tools give very exact control over what commands get invoked to build files, but it's up to the user to write rules that are correct. > Users interact with Bazel on a higher level. For example, it has built-in rules for "Java test", "C++ binary", and notions such as "target platform" and "host platform". The rules have been battle tested to be foolproof. But does it give the optional custom level of control that for example CMake + Ninja provide? Or it's only high level rules?
- blinks 11y ago
  http://bazel.io/docs/skylark/concepts.html http://bazel.io/docs/skylark/concepts.html You can [at least internally] define custom rules to handle pretty much anything, in almost-but-not-quite-python.
zerr 11y ago
Fast - compared to what?
jibu 11y ago
Maven doesn't work so well when there are loads of small self contained 'micro-libraries' (yes, sub-projects, but they are so involved to set up they almost defeat the purpose). Was considering pants -- which doesnt seem like it has great adoption? -- but this seems like its substantially more fully featured. Presumably will also make opensourcing internal projects easier. That can't be a bad thing :)
- spullara 11y ago
  WRT to Java support: Since it doesn't appear to generate poms or publish to maven repositories it doesn't seem very useful on the open source part of things. It seems explicitly for generating internal, proprietary software from a monolithic source tree. I would have much rather seen the incremental compiler and jar generator integrated to maven than replacing the entire build system.
  - alblue 11y ago
    Actually Maven 3.3 was released recently which has a smart builder for building separate parts in parallel, and using Takari plugins you can use the Eclipse complier which is parallelising in itself. See http://takari.io http://takari.io for more details.
jacquesm 11y ago
Getting rid of the timestamps in jar files is a huge improvement. I really hate it that when I recompile some huge java project I can't run a checksum on the jar to verify that the build is identical to a previous run (or when being dumped into some project that my current source tree is an accurate reflection of what is running in production).
nchelluri 11y ago
I worked at Ning for a couple of years (http://www.ning.com/ http://www.ning.com/) and the internal codename of our create-your-own social network was Bazel. When I first saw the headline I thought they'd open-sourced it.
pjjw 11y ago
Any reason the python support was ripped out? I've got my suspicions about not wanting/not being able to properly release the python packaging method in use internally, but I'm curious if I'd be tilting at windmills to try and get it to output pexes.
- fridek 11y ago
  The same question about JS. Closure Compiler never made much sense for me without blaze.
- DannyBee 11y ago
  I suspect the reason was: "They need to start with something and go from there". So they started with the use cases likely to be the most popular. Additionally, there are definitely cases where the implementations of rules at Google are a morass, and rather than dump it on the open source community, it makes more sense to clean them up when they get rebuilt.
forrestthewoods 11y ago
Will there ever be Windows support?
forrestthewoods 11y ago
Will there ever be Windows support?
deleted 11y ago
[deleted]
deleted 11y ago
[deleted]
hbhakhra 11y ago
This seems very promising. Does anyone know if this would this work with the OSGI framework?
w4tson 11y ago
It's another impressive feat from Google and reading the comments I've kind of established that 1. Binaries are checked in to source 2. It's more structured than Gradle 3. It's for very large code bases 5. It's nix only But... 1. We've already had the "chuck it in a lib directory" approach. The distributed approach maven/ivy etc seems to be working for the millions of developers out there who just have to get through the end of the day without production going up in flames. I suppose it's like moving a portion maven central into your code base. Checked in. Feels very odd, and kinda against one of the pillars of JVM: Maven. Love it or hate it it's one of most mature build/repository types out there. npm, bower anyone? 2. Got to agree with astral303. This isn't really something to shout about. Better reproducibility? Gradle/SBT have had incremental builds for quite a while. We all know there's no silver bullet, if you don't declare your inputs and outputs to gradle/blaze tasks or seed with random values then you're only going to get unrepoduceable builds. 3. Very large, I get that. 4. Very large code bases tend to enterprise systems. Enterprise systems tend to have a plethora of platforms/OSs so it being nix only is a drawback. However I suppose that if in charge of 10MLOC code base then I could mandate nix only builds? However in my experience they also tend to gravitate towards standards that seem to have longevity. I'm yet to give it a go so I'll reserve final judgement. However I will say that I do wonder how far we'd be if Googles through their brightest minds at and worked with Maven/Gradle/SBT etc to scale their builds. (Yes I realise it's multi-lang - so is gradle). Perhaps the whole community would benefit from performance benefits. Anyway hats off Google guys. It looks impressive and no doubt I'll jumping all over it in 12 months. In the mean time I'm off to go read up on Angular 2.0, or Typescript or ES6 or ES7 or whatever else I need* to know to get me through the day. Really I'm just jealous I don't have 10MLOC code base :D
- cromwellian 11y ago
  I don't know about Bazel, but Blaze doesn't "check in binaries". Build artifacts are cached, but not "checked in". The problem with maven and gradle is that their build actions/plugins can have have unobservable side effects. This approach is more 'pure functional'. You have rules which take inputs, run actions, produce outputs and memoize them. If inputs don't change, then you use memoized outputs and don't run the action. As long as your actions produce observable side effects in the outputs (and don't produce side effects which are not part of the outputs, but product state which depended upon in some manner), then you can do a lot of optimizations on this graph. In my experience with maven and gradle, they are way way slower, and that's on relatively small projects
danneu 11y ago
The "b"-with-leaves-sprouting-from-it logo is also used by http://beanstalkapp.com/ http://beanstalkapp.com/
cromwellian 11y ago
If only our code search and code review systems were public too.
- solomatov 11y ago
  BTW, do you have blaze build for gwt? ant seems unwieldy for me.
  - bruckie 11y ago
    Yes.
  - cromwellian 11y ago
    Internally to google, gwt_application, gwt_module, gwt_test is a built-in rule. GWT itself is built with blaze internally (not ant) as well.
    - solomatov 11y ago
      Do you have plans to open this stuff?
bubersson 11y ago
Wohoo! This is awesome :)
toolslive 11y ago
depends what you mean with reproducible: build a jar twice, and its md5sum will change because there are timestamps in the archive.
frownie 11y ago
From the FAQ : Multi-language support: Bazel supports Java, Objective-C and C++ out of the box, and can be extended to support arbitrary programming languages. c'mon, not even the Go language from Google itself ?
brooksbp 11y ago
Will GYP/GN be deprecated in favor of Bazel? What, if any, does the convergence among these projects look like longevity-wise?
mikojava 11y ago
Here's the Gradle Team's perspective on Bazel https://www.gradle.org/gradle-team-perspective-on-bazel/ https://www.gradle.org/gradle-team-perspective-on-bazel/

625 points by drivebyubnt 11y ago

ngd 11y ago

This is an open sourcing of Google's internal build tool. I know it as Blaze, which Bazel is an anagram of. Many files in the source have references to Blaze.

pron 11y ago

How does it compare with Java 9's sjavac (http://stackoverflow.com/a/26424760/750563 http://stackoverflow.com/a/26424760/750563)? EDIT: I fully understand that this is a build tool for multiple languages. But its raison d'etre is speed. So I'm asking what techniques does Bazel use to accelerate builds and how do they differ from those used by sjavac, which is also designed to accelerate builds of huge projects?

Sphax 11y ago
I don't think they're even related, Bazel is a general build tool, sjavac looks like a smarter Java compiler ?
- pron 11y ago
  ... that exploits parallelism and caching (and a hot VM) to accelerate build of huge projects, and supports build clusters.
  - untothebreach 11y ago
    huge Java projects...why would you compare a general build tool to a language-specific one?
    - pron 11y ago
      What difference does that make? I am not saying you could replace this tool with sjavac, but as they both tout speed as their main feature (and justification), I am wondering whether they both employ the same techniques or different ones.
  - skj 11y ago
    Inside Google we do that as well, but it's actually distinct from bazel. Bazel is a build language, and I believe they provide a reference implementation. It's not unimaginable (and as I mentioned, it has been done internally) to use caching, incremental, distributed builds. In fact, it was originally designed with those goals in mind.
hanwenn 11y ago
I work on Bazel. Bazel also builds other languages, such as C++ and Objective-C. We do invoke the Java compiler through a wrapper of our own. We think we can make that work as a daemon process to benefit from a hot JVM, but haven't gotten round to that.
- pron 11y ago
  Do you also use timestamps like sjavac or some other mechanism, like hashing?
  - hanwenn 11y ago
    Bazel uses checksums to determine if builds are up-to-date, but shortcuts the checksumming if metadata (timestamp, filesize) have not changed between builds.
    - pron 11y ago
      Your FAQ says this: Bazel configuration files are much more structured than Gradle's, letting Bazel understand exactly what each action does. This allows for more parallelism and better reproducibility. Could you please elaborate on that (i.e. with regards to both parallelism and reproducibility)?
      - hanwenn 11y ago
        I'm not a Gradle expert, so take this with a grain of salt. Gradle a single-threaded execution in some parts (I believe it may be configuration?), because the build-rules are written in a full blown language with access to the filesystem, and internal parts of Gradle. In Bazel, rules have to declare inputs and outputs, and this can be enforced with sandboxing. This allows to predict that two rules do not interfere with each other, so we know we can run them in parallel. Our extension language disallows direct access to the file system, and also forbids access to other sources of non-determinism, such as hash tables and the clock.
        11y ago

habosa 11y ago

Working at Google, Blaze is one of the technologies that amazes me most. Any engineer can build any Google product from source on any machine just by invoking a Blaze command. I may not want to build GMail from source (could take a while) but it's awesome to know that I can. I think this could be hugely useful to very large open source projects (like databases or operating systems) that may be intimidating for contributors to build and test.

middleclick 11y ago
So the builds are reproducible automatically?
- hanwenn 11y ago
  See http://bazel.io/docs/FAQ.html http://bazel.io/docs/FAQ.html, "Will Bazel make my builds reproducible automatically? For Java and C++ binaries, yes, assuming you do not change the toolchain. If you have build steps that involve custom recipes (eg. executing binaries through a shell script inside a rule), you will need to take some extra care: Do not use dependencies that were not declared. Sandboxed execution (–spawn_strategy=sandboxed, only on Linux) can help find undeclared dependencies. Avoid storing timestamps in generated files. ZIP files and other archives are especially prone to this. Avoid connecting to the network. Sandboxed execution can help here too. Avoid processes that use random numbers, in particular, dictionary traversal is randomized in many programming languages."
  - hansjorg 11y ago
    When you say Java, does that include Android? I see that Android is supported, but couldn't find anything about reproducibility. Reproducible Android builds would be very interesting.
    - esrauch 11y ago
      http://bazel.io/docs/roadmap.html http://bazel.io/docs/roadmap.html It looks like they only have iOS and not Android in this first release, but they are planning on adding Android support ~June of this year.

latkin 11y ago

Correct, reproducible, fast builds for everyone not running Windows

izacus 11y ago
Not really sure why this is getting downvoted - it's kind of an important detail when choosing a build system if you have to do multiplatform deployment.
- istvan__ 11y ago
  I am pretty sure it has nothing to do with the average population being biased.
- melling 11y ago
  I didn't downvote him but it sounds like "he's driving angry". He does work for that "evil" company in Redmond. :-). Maybe he can help make F# a great cross-platform language and increase the goodwill? Microsoft did an incredible job with F# but it really only runs well on Windows.
  - MichaelGG 11y ago
    What particular problems on non-Windows do you have with F#? I've been running F# applications in telecom, in production, on Mono, for several years. F# doesn't really seem to be the factor there at all, it's just general .NET support. In fact, F# can do a bit better than C#, as F# actually includes a static linker.
    - melling 11y ago
      Strange, most blogs that I've read discuss the pain involved. http://spin.atomicobject.com/2014/06/06/f-sharp-mono-unix/ http://spin.atomicobject.com/2014/06/06/f-sharp-mono-unix/ I guess it's the same with any new product or language: when there are a lot more great testimonials than cries of pain, then it's actually safe to use that product.

pacala 11y ago

A couple of questions: * If I have a Maven-based project with heavy reliance on pre-built jars from Maven Central, what's the recipe to port it to Bazel? * Related, if I have multiple github repos, say a couple open source libraries and a couple private repos, what's a good recipe in conjunction to Bazel?

deleted 11y ago
[deleted]
kchod 11y ago
Check out http://bazel.io/docs/build-encyclopedia.html#maven_jar http://bazel.io/docs/build-encyclopedia.html#maven_jar. In the root of your build, specify the jars you want from maven and then add them as dependencies in your BUILD files. The first time you run "bazel build", they'll be downloaded and cached from then on. It's somewhat limited in functionality at the moment, but should work for basic "download and depend on a jar". For multiple Github repos, use http://bazel.io/docs/build-encyclopedia.html#http_archive http://bazel.io/docs/build-encyclopedia.html#http_archive or http://bazel.io/docs/build-encyclopedia.html#new_http_archive http://bazel.io/docs/build-encyclopedia.html#new_http_archiv... (depending on if it's a Bazel repository or not). Let us know if you have any questions or issues!
- pacala 11y ago
  Thanks for the tips. I'm super-hyped that blaze was open sourced, it is one of the best systems I've ever had the pleasure to work with. A couple more questions :) * Any pointers for adding Scala (sbt?) support? I'd start here: http://bazel.io/docs/skylark/rules.html http://bazel.io/docs/skylark/rules.html. * Suppose I develop using multiple repos and http_archive. I'd like to make changes both to a library and to a project that depends on it simultaneously, without committing the library patches to master github repo just yet. Is there a way to configure the http_archive, let's say by saying "bazel --mode=local", and have it customize the remote archive http to use a different url (say, my github's fork instead of the master github) for that build?
  - 11y ago

Zariel 11y ago

Is this the tool that Google uses to build its Golang source? Or is that something else which is not available?

jwcrux 11y ago
You can build golang from source pretty easily. If I remember right, it's just downloading the tarball and running ./all.bash or something like that.
kchod 11y ago
The Golang source code for the server code at google is built with this tool. The rules that accomplish this are rather complex due to their interactions with our C++ libraries, and predates the open source "Go" tool. The experience with the Google internal rules, motivated some of the choices in the "go" tool, I believe. If you're interested, hanwen wrote a bunch rules with similar semantics as the internal rules, see https://github.com/google/bazel/tree/master/base_workspace/examples/go/ https://github.com/google/bazel/tree/master/base_workspace/e... . It would be nice to make these semantics match the external ones better, but it requires us to open up more tooling, so people won't need to write BUILD files.
- zzzhao 11y ago
  In what cases would using Bazel make sense to build Go projects? If they're extremely large? If they have a lot of dependencies on code in other languages? If you need sophisticated build/release tooling? BTW, thanks for the release! Will have a fun time digging through this over the next few days. I heard some murmurs that Blaze was going to be open sourced from around the watercooler but didn't think it'd be so soon.
  - hanwenn 11y ago
    I guess if you want to integrate Go tools with builds in other languages. If you are using pure Go for your entire ecosystem, there is not much point in using Bazel, as the "go" tool is very capable for that scenario.
- 11y ago

yarapavan 11y ago

Surprisingly, significant parts of the code is not open source. According to this page, http://bazel.io/docs/governance.html http://bazel.io/docs/governance.html, Is Bazel developed fully in the open? Unfortunately not. We have a significant amount of code that is not open source; in terms of rules, only ~10% of the rules are open source at this point. We did an experiment where we marked all changes that crossed the internal and external code bases over the course of a few weeks, only to discover that a lot of our changes still cross both code bases.

deleted 11y ago
[deleted]
Symmetry 11y ago
Do they mean that 10% of the original Blaze rules are now open source or that 10% of the Bazel rules they've released are open source?
- DannyBee 11y ago
  The former.
spankalee 11y ago
I don't think you're interpreting that section quite right. That section is talking about whether or not Bazel is fully _developed_ in the open, and the answer is "Unfortunately not". What they mean is that changes to the internal source of Blaze often involve changes to both the open sourced part, which is Bazel, and the closed parts, which are additional rules that are neither open sourced, nor included in Bazel (Blaze has about 5x as many rules as Bazel). It's best to make atomic changes, so rather than split the changes, review and submit the open source changes externally, and the closed rules changes internally (which would complicate reviews, testing, syncing and rollbacks), then pull in the external changes, they submit these cross-code-base changes internally, then dump the change into the external repo. The next paragraph on that page makes it clear that the code is open, even if not all of the development process is. To be clear, all of Bazel is open source and the source is available here: https://github.com/google/bazel https://github.com/google/bazel

cies 11y ago

What would be needed to get this to work with Haskell? I read in the "Getting started": > You can now create your own targets and compose them. So does this mean it is a replacement for `make`? => Yes Found the answer here: http://bazel.io/docs/FAQ.html http://bazel.io/docs/FAQ.html

kchod 11y ago
If you're interested in adding rules for a new language, check out Skylark: http://bazel.io/docs/skylark/concepts.html http://bazel.io/docs/skylark/concepts.html.

zobzu 11y ago

I had a bit of a read but I didn't find where it explains (code or doc) how it achieves reproducible builds. It seems like a stricter, huge make-like harness (in fact it reminds me of the mozilla firefox python build system a bit). It's not bad by any means, but it seems like to me it doesn't "magically" fix the "be reproducible" problem at all (which is what it seem to claim) Am I missing something?

lberki 11y ago
You are absolutely correct: Bazel by itself does not make your builds reproducible. If a tool calls rand() or bakes the current time into its output, reproducibility goes out of the window. What Bazel does, however, is to make it possible to run build steps in a sandbox (although the current one is kinda leaky) so that your build is isolated from the environment and thus behaves in the same way on any computer. It also tracks dependencies correctly so that it knows when a specific action needs to be re-run. This makes it possible to diagnose non-reproducible build steps easily. At Google, the hit rate of our distributed build cache usually floats around 99%, and this would be impossible without reproducible build steps.
- jrockway 11y ago
  Does work done by Debian to make Linux packages build reproducibly help Bazel? https://wiki.debian.org/ReproducibleBuilds https://wiki.debian.org/ReproducibleBuilds Would Bazel help with the remaining long tail of packages in Debian?
nevir 11y ago
You're right - it doesn't magically solve build reproducibility. Bazel pushes you towards a build configuration where you have to describe (in a terse way) the entire dependency graph of what is being built. It allows Bazel to be smart about where in the graph things are stale. If you run a script that outputs intermediate files, Bazel needs to know about that scripts inputs and outputs. And it works better if it knows them ahead of time.
skybrian 11y ago
It's not magic; you have to work at it. (For example, make sure that zip doesn't put timestamps in the file.) But it's designed so that code generators should act as pure functions from input files to output files, and many generators actually are, especially the built in ones. If you do this then the build system will help you. Writing generators to run this way is kind of a pain, actually, sort of like writing code to run in a sandbox. Also, the generators themselves must be checked in, and often built from source. But we consider the results worth it.

setheron 11y ago

If i'm sticking to primarily Java; is there a benefit to using Bazel as opposed to Maven / Gradle / Sbt ?

astral303 11y ago
At first impression, unless you have a single gigantic source code base, unlikely. From their FAQ: >> "Gradle: Bazel configuration files are much more structured than Gradle's, letting Bazel understand exactly what each action does. This allows for more parallelism and better reproducibility" The value of "more parallelism" depends on the complexity of your Java source code base. I can easily imagine why this extra structure can lead to more parallelism. However, I am not buying "better reproducibility" without justification or explanation. I've had very reproducible Maven builds for years (and I don't see how Gradle would be different). So I would love to know which aspects are improved upon with this structure, if someone could expand or explain. Finally, I'm very wary of "much more structure". The worst thing about Maven is its extreme insistence on structure and schema and very specific architecture of your build tasks and components. In contrast, with Gradle, you can freely shape your build scripts to reflect the "build architecture" of your source tree in a minimal, maintainable way. Furthermore, when your application's needs change, refactoring your build is far easier in Gradle, thanks to its internal-DSL style (the build script is code). If the structure isn't "free", you pay for structure with reduced build script development speed. For Google, it's a tradeoff worth having with that massive source tree.
- ulfjack 11y ago
  I work on Bazel. We've put a bunch of work into making sure that we know about every file that goes into the Java compilation, and if any of them changes (and only then) do we recompile. Within Google, we use a form of sandboxing to enforce that. You're also right that it isn't free - we have reason to believe that larger projects and larger teams will see benefits from using Bazel. Use your best judgement.
- asuffield 11y ago
  blaze is nothing remotely like the wall of cruft that maven forces you to climb for everything you do. I would describe it as "almost entirely unlike maven".

thechao 11y ago

I've been burned by so many build tools over the years. I've finally settled (for C/++/asm) on the combination of Make + ccache: I build a _very_ paranoid Makefile that recompiles everything if it feels like anything changes. For instance, every rule that compiles a C/++ file is invoked if _any_ header/inc/template file changes. I let ccache do the precise timestamp/check-sum based analysis. The result is that (for large builds < 10MMLOC) I rarely wait for more than a few hundred milliseconds on incremental, _and_ I have confidence that I never miscompile. I just wish that I had a high-performance replacement for linking that was cross-platform (deterministic mode for ar), and for non-C/++ flows. Writing a deterministic ar is about 20 lines of C-code, but then I have to bake that into the tool in awkward ways. For generalized flows, I've looked at fabricate.py as a ccache replacement, but the overhead of spinning up the Python VM always nukes performance.

beagle3 11y ago
> I build a _very_ paranoid Makefile that recompiles everything if it feels like anything changes. Do you have some kind of way to verify that your makefile dependencies conform to your source dependencies? Is clang/gcc tracking sufficient for your use case? What about upgrading the compiler itself, does your makefile depend on that? If so, how? Have you considered tup[0]? Or djb-redo[1]? Both seem infinitely better than Make if you are paranoid. tup even claims to work on Windows, although I have no idea how they do that (or what the slowdown is like). Personally, I'm in the old Unix camp of many-small-executables, non of which goes over 1M statically linked (modern "small"), so it's rarely more than 3 secs to rebuild an executable from scratch. > (deterministic mode for ar) Why do you care about ar determinism? Shouldn't it be ld determinism you are worried about? [0] http://gittup.org/tup/ http://gittup.org/tup/ [1] https://github.com/apenwarr/redo https://github.com/apenwarr/redo
- thechao 11y ago
  > Do you have some kind of way to verify that your makefile dependencies conform to your source dependencies? Nope. I explicitly use a conservative approximation—this guarantees correctness, over speed. Building everything every time with a clean tree is where I begin; I start optimizing after that. > Is clang/gcc tracking sufficient for your use case? What about upgrading the compiler itself, does your makefile depend on that? If so, how? Self-rewriting Makefiles (to consume the .d files), combined with the cleaning necessary for them, become a large technical debt—especially given the complexity of the Makefile needed to generate them. Modern CCen just aren't capable of this. Perhap Doug Gregor's module system will land in C21/C++21, and we'll see some good, then. > Have you considered tup[0]? Or djb-redo[1]? Yes. They are both don't provide significantly better correctness guarantees combined with sufficiently better performance to justify the cost to porting to older Unixen. (This is a consensus opinion at my shop; I, personally, enjoy tup.) > Why do you care about ar determinism? Shouldn't it be ld determinism you are worried about? Determinism let's me cache *.o/a/so/dylib/exe/whatnot without getting false-positives due to time-stamp changes and owner/group permissions in the obj/ar files (see ar(1)). ld is deterministic under all the CCen I use by setting the moral-equivalent of -frandom-seed.

malkia 11y ago

Oh, but my favourite option "blaze menu" is missing :)

asuffield 11y ago
Huh. I never knew that was there. I'll remember this next time I'm around Charleston.

mashraf 11y ago

Is Google departing from just throwing white papers over the wall and let community figure out the implementation details? blaze white paper was dropped a while ago and there are already two clones in Pants and Buck at Twitter and FB. It would be interesting to see how far off clones are from original implementation.

cbgb 11y ago
Do you have a link to that white paper? A quick search on their research site doesn't really yield any results.
- kchod 11y ago
  I'm a developer on Bazel, and AFAIK there is no white paper. We definitely don't want to "throw it over the wall," we're going to try to push more and more development into the open over time.
  - cbgb 11y ago
    Ah, OK, thanks. I do get a kick out of reading various Google white papers (GFS, Spanner, and the Multi-Paxos implementation for Chubby come to mind), so I'd definitely be interested in the prospect of a future white paper :).
  - bruckie 11y ago
    There's not a white paper, but there's this series of posts: http://google-engtools.blogspot.com/2011/06/build-in-cloud-accessing-source-code.html http://google-engtools.blogspot.com/2011/06/build-in-cloud-a... (and an accompanying presentation)

rquirk 11y ago

What is this lameness? https://github.com/google/bazel/tree/master/third_party https://github.com/google/bazel/tree/master/third_party - why not use gradle repos to download jars with known hashes? Sticking all those jars in the git repo is just... well, I expected better from Google.

MichaelGG 11y ago
Well keeping dependencies in source means no third party dependency at build time, right?
- rquirk 11y ago
  Right... but this will be an anchor on adoption. I can see why e.g. the Android build system does the same thing since it's all off in its own world anyway. I doubt you'd be popular with Linux distro packagers if you required bazel for some C library.
  - ulfjack 11y ago
    We realize that not everyone wants to check in all of their dependencies, and that's cool too. Check out the build encyclopedia for rules to talk to 'plain http' and maven repositories.
sarnowski 11y ago
Tbh, if your company relies on this software, I would also make sure that it cannot just vanish - and thats the most efftive solution. Artifacts can disappear from the internet and you don't know if the downloaded stuff is still the same as before. Especially, if you look outside of the maven ecosystem, but even there you have to rely on apache and their partners. An outage can mean that you cannot deploy critical bugfixes to your platform.
- adambatkin 11y ago
  This is why you should have repository manager like JFrog Artifactory or Sonatype Nexus which can transparently proxy third-party repositories (like Maven Central).

eisar 11y ago

Made by the arrogant Google assholes? No thanks. I'll stick to pants. Disclaimer: I work for TAGA (The Arrogant Google Assholes)

shmerl 11y ago

> Why doesn't Google use …? Make, Ninja: These tools give very exact control over what commands get invoked to build files, but it's up to the user to write rules that are correct. > Users interact with Bazel on a higher level. For example, it has built-in rules for "Java test", "C++ binary", and notions such as "target platform" and "host platform". The rules have been battle tested to be foolproof. But does it give the optional custom level of control that for example CMake + Ninja provide? Or it's only high level rules?

blinks 11y ago
http://bazel.io/docs/skylark/concepts.html http://bazel.io/docs/skylark/concepts.html You can [at least internally] define custom rules to handle pretty much anything, in almost-but-not-quite-python.

zerr 11y ago

Fast - compared to what?

jibu 11y ago

Maven doesn't work so well when there are loads of small self contained 'micro-libraries' (yes, sub-projects, but they are so involved to set up they almost defeat the purpose). Was considering pants -- which doesnt seem like it has great adoption? -- but this seems like its substantially more fully featured. Presumably will also make opensourcing internal projects easier. That can't be a bad thing :)

spullara 11y ago
WRT to Java support: Since it doesn't appear to generate poms or publish to maven repositories it doesn't seem very useful on the open source part of things. It seems explicitly for generating internal, proprietary software from a monolithic source tree. I would have much rather seen the incremental compiler and jar generator integrated to maven than replacing the entire build system.
- alblue 11y ago
  Actually Maven 3.3 was released recently which has a smart builder for building separate parts in parallel, and using Takari plugins you can use the Eclipse complier which is parallelising in itself. See http://takari.io http://takari.io for more details.

jacquesm 11y ago

Getting rid of the timestamps in jar files is a huge improvement. I really hate it that when I recompile some huge java project I can't run a checksum on the jar to verify that the build is identical to a previous run (or when being dumped into some project that my current source tree is an accurate reflection of what is running in production).

nchelluri 11y ago

I worked at Ning for a couple of years (http://www.ning.com/ http://www.ning.com/) and the internal codename of our create-your-own social network was Bazel. When I first saw the headline I thought they'd open-sourced it.

pjjw 11y ago

Any reason the python support was ripped out? I've got my suspicions about not wanting/not being able to properly release the python packaging method in use internally, but I'm curious if I'd be tilting at windmills to try and get it to output pexes.

fridek 11y ago
The same question about JS. Closure Compiler never made much sense for me without blaze.
DannyBee 11y ago
I suspect the reason was: "They need to start with something and go from there". So they started with the use cases likely to be the most popular. Additionally, there are definitely cases where the implementations of rules at Google are a morass, and rather than dump it on the open source community, it makes more sense to clean them up when they get rebuilt.

forrestthewoods 11y ago

Will there ever be Windows support?

forrestthewoods 11y ago

Will there ever be Windows support?

deleted 11y ago

[deleted]

deleted 11y ago

[deleted]

hbhakhra 11y ago

This seems very promising. Does anyone know if this would this work with the OSGI framework?

w4tson 11y ago

It's another impressive feat from Google and reading the comments I've kind of established that 1. Binaries are checked in to source 2. It's more structured than Gradle 3. It's for very large code bases 5. It's nix only But... 1. We've already had the "chuck it in a lib directory" approach. The distributed approach maven/ivy etc seems to be working for the millions of developers out there who just have to get through the end of the day without production going up in flames. I suppose it's like moving a portion maven central into your code base. Checked in. Feels very odd, and kinda against one of the pillars of JVM: Maven. Love it or hate it it's one of most mature build/repository types out there. npm, bower anyone? 2. Got to agree with astral303. This isn't really something to shout about. Better reproducibility? Gradle/SBT have had incremental builds for quite a while. We all know there's no silver bullet, if you don't declare your inputs and outputs to gradle/blaze tasks or seed with random values then you're only going to get unrepoduceable builds. 3. Very large, I get that. 4. Very large code bases tend to enterprise systems. Enterprise systems tend to have a plethora of platforms/OSs so it being nix only is a drawback. However I suppose that if in charge of 10MLOC code base then I could mandate nix only builds? However in my experience they also tend to gravitate towards standards that seem to have longevity. I'm yet to give it a go so I'll reserve final judgement. However I will say that I do wonder how far we'd be if Googles through their brightest minds at and worked with Maven/Gradle/SBT etc to scale their builds. (Yes I realise it's multi-lang - so is gradle). Perhaps the whole community would benefit from performance benefits. Anyway hats off Google guys. It looks impressive and no doubt I'll jumping all over it in 12 months. In the mean time I'm off to go read up on Angular 2.0, or Typescript or ES6 or ES7 or whatever else I need* to know to get me through the day. Really I'm just jealous I don't have 10MLOC code base :D

cromwellian 11y ago
I don't know about Bazel, but Blaze doesn't "check in binaries". Build artifacts are cached, but not "checked in". The problem with maven and gradle is that their build actions/plugins can have have unobservable side effects. This approach is more 'pure functional'. You have rules which take inputs, run actions, produce outputs and memoize them. If inputs don't change, then you use memoized outputs and don't run the action. As long as your actions produce observable side effects in the outputs (and don't produce side effects which are not part of the outputs, but product state which depended upon in some manner), then you can do a lot of optimizations on this graph. In my experience with maven and gradle, they are way way slower, and that's on relatively small projects

danneu 11y ago

The "b"-with-leaves-sprouting-from-it logo is also used by http://beanstalkapp.com/ http://beanstalkapp.com/

cromwellian 11y ago

If only our code search and code review systems were public too.

solomatov 11y ago
BTW, do you have blaze build for gwt? ant seems unwieldy for me.
- bruckie 11y ago
  Yes.
- cromwellian 11y ago
  Internally to google, gwt_application, gwt_module, gwt_test is a built-in rule. GWT itself is built with blaze internally (not ant) as well.
  - solomatov 11y ago
    Do you have plans to open this stuff?

bubersson 11y ago

Wohoo! This is awesome :)

toolslive 11y ago

depends what you mean with reproducible: build a jar twice, and its md5sum will change because there are timestamps in the archive.

frownie 11y ago

From the FAQ : Multi-language support: Bazel supports Java, Objective-C and C++ out of the box, and can be extended to support arbitrary programming languages. c'mon, not even the Go language from Google itself ?

brooksbp 11y ago

Will GYP/GN be deprecated in favor of Bazel? What, if any, does the convergence among these projects look like longevity-wise?

mikojava 11y ago

Here's the Gradle Team's perspective on Bazel https://www.gradle.org/gradle-team-perspective-on-bazel/ https://www.gradle.org/gradle-team-perspective-on-bazel/