6 ms·

The main driver of success in either model is in the tooling and practices invested in it to make it work in an organization. Google is successful with their mo

The main driver of success in either model is in the tooling and practices invested in it to make it work in an organization. Google is successful with their monorepo because they have invested in building (blaze), source control (piper, code search), and commit to always developing on HEAD. Multirepo is currently easier for most companies because most public tooling (git, package manager) is built around multirepos. One place I see multirepos fall over is awful dependency management practices internally and in open source. Many dependencies quickly become outdated and are not updated in cadence, slowing down writers and consumers. Better tooling can help here but an organization needs real discipline to stay on top of things.

alexnewman 7y ago
Google uses no versioned libraries?
- chrisseaton 7y ago
  I think they do, but there is only ever one single version in use - the version in the repo.
  - minxomat 7y ago
    Yes, and that prevents duplication of work when it's time to upgrade: https://github.com/microsoft/TypeScript/issues/33272 https://github.com/microsoft/TypeScript/issues/33272
  - user5994461 7y ago
    If so, they would have a massive problem upgrading libraries like numpy because there are too many and too big breaking changes between releases.
    - deleted 7y ago
      [deleted]
    - jumpingmice 7y ago
      Inside Google's main repo there are different build targets for libraries with incompatible API changes that are too difficult to fix all at once, e.g. there might be numpy_1_8 and numpy_1_10 separately. Python at Google muddled along for years without numpy at all so it's not like anyone would be seriously harmed by having an old release in the repo.
- Cedricgc 7y ago
  Libraries internal to Google are kept at the latest and consumers are updated to use the latest APIs. 3rd party libraries are checking into the monorepo at a specific version and everyone uses the same version.
  - kyrra 7y ago
    It's documented here https://opensource.google/docs/thirdparty/ https://opensource.google/docs/thirdparty/
    - deleted 7y ago
      [deleted]
username90 7y ago
I wonder why nobody have made a good public monorepo offering similar to what Google have internally. Would probably be a hit at many companies since it fixes so many issues related to working in very large teams.
- dmoy 7y ago
  At large enough scale, it causes a lot of problems and breaks like every other dev tool requiring a lot of work to get it back together. That said, there are some open source pieces to help. Facebook open sourced their mercurial stuffs so you can get version control at scale (and before then you just use perforce). Google open sourced bazel. Google open sourced some parts of the underlying infra behind code search, but not enough to really work properly. And of course lower level there's a plethora of reasonable db offerings, etc. It would still require a lot of glue though.
  - username90 7y ago
    It is just that Googles tooling around code works really well together. Code search to view code and directory based history so you aren't swamped by others commits, tap to run all unit tests all the time but with sectioned projects so you don't run all tests on every presubmit, sponge to gather every test log ever (even for the tests you run locally) so you can link full test logs to coworkers when you have problems (also note that the logs source file links are actual links into code search), critique for easy versioned code reviews where you can diff code between any sets of comments so you can see its evolution and running presubmit checks and sponge links for tests, blaze to make a structured directory dependency management system to make partial checkouts and distributed cached builds work well. I'd like a set of tightly coupled tool like that working outside of Google, but I guess it might be a just a dream, it is a bit too big of a project.
    - djhaskin987 7y ago
      > directory based commits This whole thread is interesting because subversion is exactly this, and works with large code bases. We used to have these told and we moved away from them.
gnachman 7y ago
When I started at Google the tooling was not very good, and monorepo was pretty painful. They used perforce, and it simply couldn't keep up. Commits could take minutes. Code review was also unbearably slow. Blaze didn't exist yet; just before I started they had tools that generated million+ line makefiles that everyone hated. So yeah, you need good tooling, but even Google didn't have it for the first ~10 years of its existence.
- tomrod 7y ago
  I went to graduate school with a guy who ended up on Google's infrastructure team. He'd previously worked as head or lead dev for subversion (a little hazy on the details). We worked on a small project where we put together statistical measures for codebases. It was a lot of fun, even if the infrastructure was out of my wheelhouse at the time. Folks that can manage billion-line codebases are on a whole different level I think. I wonder sometimes how many folks like that there are. EDIT: Looks like he left for a bit and is now back. Good on him!
OneMoreGoogler 7y ago
You know Google's monorepo is successful because they have so many of them! google3, Android, Chrome browser, ChromeOS... Kidding aside, my point is Google recognizes obvious boundaries between e.g. their web stuff and android, and organizes their code accordingly.

alexnewman 7y ago

Google uses no versioned libraries?

chrisseaton 7y ago
I think they do, but there is only ever one single version in use - the version in the repo.
- minxomat 7y ago
  Yes, and that prevents duplication of work when it's time to upgrade: https://github.com/microsoft/TypeScript/issues/33272 https://github.com/microsoft/TypeScript/issues/33272
- user5994461 7y ago
  If so, they would have a massive problem upgrading libraries like numpy because there are too many and too big breaking changes between releases.
  - deleted 7y ago
    [deleted]
  - jumpingmice 7y ago
    Inside Google's main repo there are different build targets for libraries with incompatible API changes that are too difficult to fix all at once, e.g. there might be numpy_1_8 and numpy_1_10 separately. Python at Google muddled along for years without numpy at all so it's not like anyone would be seriously harmed by having an old release in the repo.
Cedricgc 7y ago
Libraries internal to Google are kept at the latest and consumers are updated to use the latest APIs. 3rd party libraries are checking into the monorepo at a specific version and everyone uses the same version.
- kyrra 7y ago
  It's documented here https://opensource.google/docs/thirdparty/ https://opensource.google/docs/thirdparty/
  - deleted 7y ago
    [deleted]

username90 7y ago

I wonder why nobody have made a good public monorepo offering similar to what Google have internally. Would probably be a hit at many companies since it fixes so many issues related to working in very large teams.

dmoy 7y ago
At large enough scale, it causes a lot of problems and breaks like every other dev tool requiring a lot of work to get it back together. That said, there are some open source pieces to help. Facebook open sourced their mercurial stuffs so you can get version control at scale (and before then you just use perforce). Google open sourced bazel. Google open sourced some parts of the underlying infra behind code search, but not enough to really work properly. And of course lower level there's a plethora of reasonable db offerings, etc. It would still require a lot of glue though.
- username90 7y ago
  It is just that Googles tooling around code works really well together. Code search to view code and directory based history so you aren't swamped by others commits, tap to run all unit tests all the time but with sectioned projects so you don't run all tests on every presubmit, sponge to gather every test log ever (even for the tests you run locally) so you can link full test logs to coworkers when you have problems (also note that the logs source file links are actual links into code search), critique for easy versioned code reviews where you can diff code between any sets of comments so you can see its evolution and running presubmit checks and sponge links for tests, blaze to make a structured directory dependency management system to make partial checkouts and distributed cached builds work well. I'd like a set of tightly coupled tool like that working outside of Google, but I guess it might be a just a dream, it is a bit too big of a project.
  - djhaskin987 7y ago
    > directory based commits This whole thread is interesting because subversion is exactly this, and works with large code bases. We used to have these told and we moved away from them.

gnachman 7y ago

When I started at Google the tooling was not very good, and monorepo was pretty painful. They used perforce, and it simply couldn't keep up. Commits could take minutes. Code review was also unbearably slow. Blaze didn't exist yet; just before I started they had tools that generated million+ line makefiles that everyone hated. So yeah, you need good tooling, but even Google didn't have it for the first ~10 years of its existence.

tomrod 7y ago
I went to graduate school with a guy who ended up on Google's infrastructure team. He'd previously worked as head or lead dev for subversion (a little hazy on the details). We worked on a small project where we put together statistical measures for codebases. It was a lot of fun, even if the infrastructure was out of my wheelhouse at the time. Folks that can manage billion-line codebases are on a whole different level I think. I wonder sometimes how many folks like that there are. EDIT: Looks like he left for a bit and is now back. Good on him!

OneMoreGoogler 7y ago

You know Google's monorepo is successful because they have so many of them! google3, Android, Chrome browser, ChromeOS... Kidding aside, my point is Google recognizes obvious boundaries between e.g. their web stuff and android, and organizes their code accordingly.