10 ms·

Google Feedback on TypeScript 3.5

396 points by dkns 7y ago

psv1 7y ago
Not a TypeScript user, but what really stood out to me is that Google are using a monorepo.
- bad_user 7y ago
  Google is using the monorepo ;-)
- Dobbs 7y ago
  Google is pretty notorious for this. It is one of the reasons behind the old golang GOPATH setup, and one of the reasons it took so long for the Go taking so long to get modules.
  - jmillikin 7y ago
    I'm not sure that's true. The layout of Go code in Google's monorepo is not at all similar to GOPATH, and patterns that are common within google3 (such as multiple Go packages in one directory) are fundamentally incompatible with the Go build system. As an ex-Googler, I still strongly prefer the google3 style and am annoyed when open-source Go tooling can't deal with it properly.
    - nih0 7y ago
      what is google3 style?
      - papln 7y ago
        I think it's the mentioned > multiple Go packages in one directory as opposed to having a 1:1 package:dir mapping
- heavenlyhash 7y ago
  Be careful not to mistake Google's use of a monorepo with consideration of whether $DAYJOB or $FOSSPROJECT should use a monorepo. Google has a lot of tooling and some very thoroughly-considered and reinforced policies and cultures around their use of a monorepo. Trying to use a monorepo without those tools and ingrained policies may not be very likely to lead to similar results.
  - arlk 7y ago
    for reference https://ai.google/research/pubs/pub45424 https://ai.google/research/pubs/pub45424
  - rkangel 7y ago
    The default should be keeping code together, and justifying why you are splitting it up, not the other way around. While what you said is true, the codebase of most organisations is not large enough that they run into those scale constraints for a long time. Instead, if you split up your code you immediately get organisational headaches of managing changes across multiple codebases. If you keep it together, the scale challenges can be managed later, if they occur (c.f. YAGNI). If you are splitting, codebases should be split across organisational boundaries, not technical ones. If you look at the FOSS world, the obvious conclusion is that each library is in its own repo, and that is partly true. In practice what is happening is that each OSS project is its own organisational team, and so it lives together. If you have parts of your company on different continents working on different things (and maybe you need different access) then splitting up may make sense. Otherwise, you're probably just making your life harder for little to no gain.
    - heavenlyhash 7y ago
      I think I'd more or less agree with that. For example, one of the questions I generally ask in a meeting room that's considering this topic, and has for example microservices in flight, is: "Are you willing to put in the work to make Service A support more than one version at a time in Service B?", and if not, then that's a very (very) strong indicator that the level of coupling and lack of organizational boundaries will result in pain from anything but a monorepo. I prefer to split things up when possible. When it does fit, there's many benefits. There's also the concern that building too much culture and tooling that presumes a lack of splitting of repos can become its own form of trap which becomes increasingly hard to navigate out of, even if you later want to, as the situation self-iterates. But I agree that splitting things out needs justification, and the "default" stance should include that.
- apalmer 7y ago
  that stood out to me too... not the monorepo really, but the fact its a monorepo of a billion lines of code. seems almost impossible to maintain when you have a dependency change at the lowest levels that affects numerous projects.If i am understanding google was in a situation where they had to update every project that used typescript inside the entire company at the same time, that seems untenable.
  - yoz-y 7y ago
    Mostly it means that you have to be really thoughtful when introducing a breaking change to a low level library.
    - papln 7y ago
      Or you can YOLO / Leroy Jenkins the change and let everyone else fix the breakage you create, or see if they demand a rollback.
      - yoz-y 7y ago
        They will not only demand a rollback but also do it. Besides, low level changes at this scale require a copious amount of approvals.
  - larzang 7y ago
    Whether it's a monorepo or not doesn't change the fact that you have a billion lines of code to maintain, but at least this way they're forced to make changes consistently. The alternative would be doing it piecemeal with a dozen versions of dependencies propagating due to fragmentation, which is way way worse.
  - forgot-my-pw 7y ago
    Here's an article from a maintainer of a framework library (Angular): https://medium.com/@Jakeherringbone/you-too-can-love-the-monorepo-d95d1d6fcebe https://medium.com/@Jakeherringbone/you-too-can-love-the-mon... Something like this might only be possible due to their tooling and test coverage. So when you change something, you immediately get alerted of broken tests.
- kyrra 7y ago
  Another interesting bit with the monorepo is the one-version-policy: https://opensource.google.com/docs/thirdparty/oneversion/ https://opensource.google.com/docs/thirdparty/oneversion/ That's why the typescript upgrade was so hard for them. We (attempt) to enforce a single version of a library/toolchain to be checked into the codebase at any given time. You can have multiple in during an upgrade, but it's highly discouraged.
  - tylerhou 7y ago
    This is also why Google says to test everything; even minor version upgrades can have unexpected behavioral changes. Without tests, these might break your project without warning.
  - papln 7y ago
    On the contrary, having multiple-versions could have made the upgrade much worse, by deferring compatibility problems from submit time to deploy time.
    - kyrra 7y ago
      It's a trade-off. As a user of a third-party library, I would like to upgrade it to get new functionality. But there are breaking changes in the update. So to upgrade I would have to fix all users of the library to upgrade. While this is better overall for the codebase, it can put a lot of work on others for a not well maintained third-party library. Something like TS has people that help keep it updated. But for something more obscure, it'll be on someone else who cares enough to put in the work.
- slig 7y ago
  How does that work on a nodejs project? I understand that they only have one version of the TypeScript compiler for all projects inside the monorepo, so that means there's only one huge package.json inside with all the packages used by every project inside the repo?
  - jagthebeetle 7y ago
    The lingua franca for Google's building needs is (more or less) bazel, where you say target /a/b/c depends on /dep/v1_1, /dep/xyz, /common/foo, etc.). (There is a filesystem-like hierarchy parallel to, but not necessarily the same as the corresponding repository's directory layout.) Bazel is extensible via rules [1], so if you really wanted to use NodeJS on your team, you might create a `nodejs_binary` rule that put everything in the right directory and ran some NodeJS packager on it. You'd probably not put it into production. Also, third-party code lives in a single third-party directory, so yes, internal users could pull down code they wanted (and for which there wasn't a satisfactory internal version already) into that directory: https://opensource.google.com/docs/thirdparty/ https://opensource.google.com/docs/thirdparty/ [1]: https://docs.bazel.build/versions/0.29.0/skylark/rules.html https://docs.bazel.build/versions/0.29.0/skylark/rules.html
tannhaeuser 7y ago
Is there a way to migrate jsdoc-annotated JavaScript code over to TS, and is TS's minifier as good as Google's closure-compiler yet?
- smt88 7y ago
  I don't know of prod-quality tools to migrate from jsdoc to TS. TS does have comment-level typing, so you can get the benefits without transpiling anything (similar to Flow). TS pointedly does not minify output. It does the opposite: try to generate code that a human might have written. It's very easy to incorporate TS into a Babel or Webpack build pipeline though, so you can use a purpose-built minifier of your choice.
- vbitz 7y ago
  Visual Studio Code has a "Code Fix" that can automatically copies JSDoc into TypeScript annotations. This can be done to an entire file at once. TypeScript does not come with a minifier. The code it produces is compatible with the target version of JavaScript (e.g. compile ES6 modules into CommonJS) but unminified.
  - tannhaeuser 7y ago
    I see. closure-compiler's "advanced" mode takes advantage of jsdoc type info for minifying so I guess generic syntactical minification won't compress as much for the time being.
    - evmar 7y ago
      We (Google TS team) maintain a tool[1] that transforms TS types into jsdoc types for the purpose of feeding them into 'advanced' mode. We also (to answer the grandparent question) maintain a tool[2] that converts Closure-annotated TS into JS. (Why both ways? We transition JS->TS and check it in as the code the user works with, while we use the TS->JS one within the compiler at optimization time.) [1] https://github.com/angular/tsickle https://github.com/angular/tsickle [2] 'gents', in this repo https://github.com/angular/clutz https://github.com/angular/clutz
andreigaspar 7y ago
So happy to see the attention Typescript is getting lately. Absolutely love the language, and have been using it since it got released.
- umvi 7y ago
  I tried using it with Angular, but it doesn't seem to help much. For example, if you have something like: <button (click)="login(email, pass)" /> And then a TypeScript function like: login(email: string, pass: string) { } TypeScript can't help you at all here because all the typing is determined at runtime by Angular. Even if `email` is a number or a boolean, no problem, it will just happily pass it in. What benefit, then, does TypeScript provide? I understand it's compile-time guarantees, but how does that help if the types are coming in from HTML land which the TypeScript compiler doesn't examine at all?
  - olmo 7y ago
    Glad to see someone pints this out. That’s the main reason I use React + Typescrip. JSX is an extension of javascript and can be fully checked while any template language is a custom invention that it’s hardly toolable.
  - threecreepio 7y ago
    For your template code in angular, there won't be any significant benefit to using typescript. It could be worse than not having typescript at all, since you can add type annotations to your 'login'-function that don't match up with reality. That's not really a typescript issue though, and it works great with libraries that don't use string templates. From what I've seen the angular community hasn't really prioritized a typecheck-able templating language.
    - dmix 7y ago
      Nor does Vue. The focus is on typing the reactive data that feeds the templates which should be sufficient. Plenty of things like HTML form elements take numbers or strings just fine, as it all outputs to strings in the end. Additionally, by breaking up stuff into smaller composable components there should be enough gating and typing layers, at least with Vue/Vuex that's the case. If it was just plopping straight into the elements 1-to-1 that might be a different story.
vallode 7y ago
Issues like these are the reason we can thrive as a community. Not playing a blame-game and offering direct feedback to a wonderful open source project. Trying to get my own company to get more involved in feedback to the open source tools we use as I think it is so extremely respectful and encouraging!
jannes 7y ago
I love TypeScript. I started using it around v1.0. Microsoft has hit some gold with it. When I first started using it I had lots of `any` in my code (like the Google employee is describing here). But over time it really starts being extremely clean.
- radicalbyte 7y ago
  I started around 0.7/0.8, it was already such a fantastic product back then. It's just fantastic, Javascript on steroids.
  - Rychard 7y ago
    I don't have a significant amount of experience with TypeScript, but even from my limited experience I agree that it's a fantastic product. That said, at $JOB we (unfortunately) have a fair amount of production code written in TypeScript 0.9 which nobody has ever upgraded, and simply won't compile on a more recent version. It's been that way for years now, and every attempt to bring it up to date has been met with failure. It may be an overly broad request, but I'd be very interested if anyone had any suggestions for how we might go about performing such an upgrade in an iterative manner. My understanding is that pre-1.0, TypeScript went through a number of breaking changes (as would be expected of any pre-release software), but I've never found a complete list of what those breaking changes were.
    - pas 7y ago
      So, because it currently "works", it should "safely" compile by adding many `any`s. (No Implicit Any, and other strictness flags set to false.) Then it's a matter of cleaning them up. But usually, at least in my experience, the problem is the libraries and their API, which used to be very much full of any in those dark times.
    - jannes 7y ago
      You can see a list of all changes in the "What's New in TypeScript" document on GitHub: https://github.com/Microsoft/TypeScript/wiki/What's-new-in-TypeScript https://github.com/Microsoft/TypeScript/wiki/What's-new-in-T... And a list of breaking changes here: https://github.com/microsoft/TypeScript/wiki/Breaking-Changes https://github.com/microsoft/TypeScript/wiki/Breaking-Change... They both go back until v1.1. For older changes you can check the blog: https://devblogs.microsoft.com/typescript/announcing-typescript-1-0/ https://devblogs.microsoft.com/typescript/announcing-typescr... https://devblogs.microsoft.com/typescript/announcing-typescript-0-9-5/ https://devblogs.microsoft.com/typescript/announcing-typescr... Anyway, it's hard to say what you need to do to get your code to compile on the latest TypeScript version without knowing what some common compile errors are in your code. Here are some guesses: 1. The way you obtain type definitions for third-party packages has changed. It used to work with /// <reference> and tools like nuget or TSD (TypeScript definition manager). But now it works with npm in the @types namespace and some npm packages even include definitions themselves now. Of course, updating to the latest type definitions would mean that you have to upgrade to the latest versions of the third-party libraries as well. Otherwise you have to find a way to keep working with the old definitions for the respective library versions. 2. Pre-1.5 TypeScript had something called "internal modules" which were renamed to "namespaces" and are generally discouraged now. Hopefully your code is not using that feature. https://www.typescriptlang.org/docs/handbook/namespaces.html https://www.typescriptlang.org/docs/handbook/namespaces.html 3. tsconfig.json. I don't remember when they introduced it, but you likely need to create this file and tweak the settings
Stay_frostJebel 7y ago
This write-up led me to Evan Martins blog. What a gold mine. Digestible writing with a peek inside of Google's internals. http://neugierig.org/software/blog/ http://neugierig.org/software/blog/
heavenlyhash 7y ago
> (I might suggest the underlying problem in this code is relying on inference too much, but the threshold for "too much" is difficult to communicate to users.) This is a very outstandingly interesting line out the whole writeup. I like the writeup in its entirety for being very balanced and thoughtful, but this line in particular really stands out to me as worth more thought for anyone interested in language and type system design. Inference is great. Except... when it's not. When it's "too much". When it starts making breaking changes appear "too distantly". It's an interesting topic to reflect upon, because the inference isn't making a breakage; just shifting around where the breakage appears. (And this makes it hard to ever direct criticism at an inference mechanism!) But this kind of shifting-around of the breakage appearance can be a drastic impact on the ergonomics of handling it: how early it's detected, how close to the important change site tooling will be able to point the coder, etc. That's important. And almost everything about it involves the word "too" -- which means the area is incredibly subjective, and requires some kind of norm-building... while not necessarily providing any clear place for that norm-building to center around. I don't have a point here other than to say this is interesting to reflect on. I suspect the last chapter on type inference systems has not yet been written. Can an inference system be designed such that it naturally restrains "too" much use of it?
- the_duke 7y ago
  > because the inference isn't making a breakage; just shifting around where the breakage appears That's a very valuable insight, and a concern in a lot of languages with advanced inference! The problem is exacerbated in Typescript though because it is fundamentally a practical, evolved layer over Javascript, with an unsound type system that likes to bail out to any.
- jimbo1qaz 7y ago
  In my experience with type inference (quite a bit of mypy, now learning rust), inferring local variable types based on what's assigned to it is not very confusing, as long as me or the IDE can figure out what type is being assigned. But when a function return value or variable's generic argument is determined by where the value is used, I tend to have more trouble (since one type can propagate from one function, to its argument, to the function which generated the argument).
bsaul 7y ago
A little off-topic: I'm currently trying to find a way to write (type safe) business logic once, then reuse it pretty much everywhere (mobile / web / desktop),and it seems to me that typescript has become the only option. Javascript runtime is present everywhere, and can interface with anything. Does someone knows of another alternative (viable right now, or in the coming months) ? I know llvm can theoretically target any platform, including wasm, but how painfull is it in practice ? Can you write a line of code that does a network request then expect it to run as it is on the browser and on mobile platforms ?
- GiorgioG 7y ago
  C# (with .NET Core 3.0 officially launching on the 23rd of September, Blazor (WASM) support is included.) I have been playing around with Blazor a bit with the preview releases and I have to say it's pretty slick.
  - stefano 7y ago
    Did they improve the download size?
    - cutler 7y ago
      No and on that basis it's unusable. However that's because it downloads the whole runtime. Eventually MS intend to load only your compiled code.
      - mattferderer 7y ago
        You can avoid that issue with the Server Side Blazor concept for business side portions of your application. Depending on the application you can get extremely far with just using HTTP requests to an API for anything business logic related. A lot of apps over complicate themselves by trying to force a JS framework on the front end with no benefit to the user. I seem to prefer using non-SPAs these days over SPAs because very few places do them well. Google being a terrible SPA developer. - Edit - I forgot to mention Elixir's Phoenix has LiveView which is similar to Server Side Blazor.
brlewis 7y ago
I'm really curious how many lines of TypeScript are in use at Google. I bet most JS is still Closure.
- deleted 7y ago
  [deleted]
msoad 7y ago
filter(Boolean) is a bad idea anyways. I was bitten by this before.
- y2bd 7y ago
  Why do you say so?
  - papln 7y ago
    One reason: This isn't TS-specific; it's common in Python too: coercing to Bool has language semantics (for whatever language you are in) which often don't match the application semantics of your program. Application programmers don't (and shouldn't have to, but for the language's over-eager coercions) always think about the boolean semantics of all their objects. In particular, None and empty/zero object are both False in Python, and Python style/linters push you to avoid explicit comparison to None, which gets weird when your application wants to treat empty objects as True because they have differen semantics from None. (For example, in a security function, None may mean no-op / fallback to default, but Empty might mean "Reject all".
    - WorldMaker 7y ago
      Another reason that is TS-specific is that of JS functions can be highly variadic in the number of arguments and a lot of subtle runtime bugs can be found in blindly passing arguments without checking their count. If filter changes from returning only one thing to say two (for instance, an index count), Boolean may produce a runtime error for having too many arguments, may silently ignore extra arguments, may interpret an extra argument as changing the behavior, or some combination of all three depending on strictness versus compatibility level, executing browser, phase of the moon, etc. Boolean itself I've not had trouble with, but things like map(parseInt) is the big one that bites a lot of junior developers all the time in TS/JS. (parseInt takes an optional second argument for radix, so in cases where the second argument returned by map is an index count, which is likely, you get it parsing in base-0, base-1, base-2, … which is almost never something you'd do intentionally.)
lxe 7y ago
Almost every single Flow version upgrade is like this — every new version brings a slew of errors due to Flow’s continuous movement away from practicality towards “soundness”.
- vicapow 7y ago
  I’m not sure I agree. I’ve found updating Flow in a rather large monorepo a relatively straightforward process. The changes are usually rather small because the team releases every two weeks. They also manage the update internally within their company’s monorepo so they’ll usually find out about these types of unexpected behavior changes before the community does. That said, we do rely on automated error excludes (similar to eslint-ignore-next-line) for things that cannot be fixed with codemods. Those errors were always there it’s just now you know about them. Better to stem the bleeding by updating the type checker to the latest version.
  - lxe 7y ago
    > I’ve found updating Flow in a rather large monorepo a relatively straightforward process. I'm not sure you're being entirely honest here, as we both know through feedback that flow updates in the said monorepo are one of the most burdensome processes for its contributors. As a maintainer/owner of a monorepo experience, it's crucial to maintain empathy and honesty of how processes such as dependency and tooling updates affect (and are perceived by) its users.
    - vicapow 7y ago
      Apologies, lxe. I can tell you're frustrated. If the the Flow upgrade process has been burdensome to others, I'd really like to know. I've always thought we've done a good job of updating internally on the platform team and that it hasn't been a concern for end users. We did run into some pain points in changing configuration options around the unnecessary optional-chaining lint rules but that wasn't related to updating the Flow version. I think you may be conflating those.
  - lhorie 7y ago
burtonator 7y ago
Typescript is absolutely amazing. I've been working with it for the last 8 months. https://getpolarized.io/ https://getpolarized.io/ and the source is here: https://github.com/burtonator/polar-bookshelf https://github.com/burtonator/polar-bookshelf I could have NOT made as much progress just by using JS directly. When you have a large code-base and you're trying to make progress as fast as possible refactoring is needed and the strict typing is invaluable. Honestly, the MAIN issue with TS are issues around webpack + typings for 3rd party and older modules. I'd say 85% of the code I want to use already has types but when they don't it's frustrating to have to pause and write my own types. I have 20 years of Java experience. Used it since 1.0 and for the most part have been unhappy with everything else. I've decided that Node + Typescript is by far the most productive environment for me to switch to. I can code elegant apps with both front and backends and I get strict typing. Could NOT have made so much progress without TS.
- patates 7y ago
  I'm writing this as a developer who writes a lot of C# and Typescript: Have you tried C#? Even though I sometimes miss the flexibility of Typescript when writing in it, I love the reliability. Maybe it's no as battle-tested as Java (especially with all the rewrites recently), but feels like it's 99.9% there.
  - htgb 7y ago
    Not GP, but as a developer working with TypeScript and C#, I respectfully disagree regarding the type system. There are many things I like better in C# compared to JavaScript, but I feel overly constrained by the type system of C# way more often, and most notable is the lack of discriminated unions (aka sum types etc). You can say that something has this and that, but not that it is this or that.
    - andrejk 7y ago
      I agree, as a mostly C# developer for years, then Typescript (Node, Angular and React) for the last couple of years, when I go back to C# projects, I feel like I'm doing a lot of work for the compiler. And really elegant use of the TS type system doesn't translate well into C# many times, forcing me to write boilerplate. I've been eyeing F# for more elegant managed code, but that's going to take me a little more up-front investment to get productive.
jbverschoor 7y ago
Google sure has a lot of comments these days
hnsocks 7y ago
Testing shadow ban: google and Facebook sucks more than socks time to ban me
dmix 7y ago
> but any time someone saves a Selection into a member variable, they ended up writing down whatever type TS inferred at that time, e.g. > mySel: d3.Selection<HTMLElement, {}, null, undefined>; I'm curious how Google and others approach adopting Typescript gradually, as I'm pretty new to it, I'm assuming it goes like: The programmer converts code to Typescript and when they come across return types they copy the inferred type and add it to the codebase directly wherever possible. I'm assuming just as a matter of using (untyped) libraries you need to rely on the output of Typescript in order to try have every return typed. So the biggest problem seems to be how TS infers things changed meaning you can't always trust what you copied as staying consistent, even if the source library doesn't change itself. That's always something to keep in mind for overhead.
- gmoot 7y ago
  The explicit type is optional. It's not that it's required to copy out whatever the compiler inferred, it's just that lots of people do it to be explicit.
  - dmix 7y ago
    All types are optional in typescript. I was just curious about Google's approach to it and if they had some standard practice of always copying the inferred type to get a great amount of coverage.
    - lonelappde 7y ago
      Copying the inferred type is the same as copying your runtime outputs into your tests. It's a statement that you believe the result is correct and it's on you to make that judgement call. It's not a policy to blindly copy everything, because that defeats the purpose of type checking and testing, turning your tests into " verify that nothing changed", not "verify that the system behaves as intended".
      - dmix 7y ago
        Thanks, thats typically how I approach it as well, it's good to hear it written out. I try not to append copied types without fully understanding their structure either. But sometimes you just need to just trust it to solve a problem. There's plenty of blog posts and docs on advanced types but I'm interested in the day-to-day best practical approaches people are taking adopting it. I should look around for some literature or talks on the subject...
_the_inflator 7y ago
It is very funny to read this from Google, since we talked about similar problems with Google regarding Angular upgrades, where features were modified that Google considered not used/no use cases known, while we were relying on these. From my point of view Core members of any super large project (like React, Angular, TypeScript) are limited by design in what they perceive as their target audience and their use cases. This is simply a matter of fact: even as a core dev you cannot know how every dev uses your product. So this is some sort of left-pad moment for TypeScript.
juliendc 7y ago
I've recently built a Node server with TypeScript and it's a joy to use with external libraries when the types are available. It's such a time saver to not have to guess which method to call with which arguments (I've had only experience with dynamic languages before). Some libraries don't have types or they are outdated but it was a minority. With the experience I've found that most of the type errors are actually between the backend and the frontend in web applications. It's still hard to fully type the entire flow from the database calls with the ORM to the objects manipulation in the frontend. How are you dealing with that? We used Nexus with GraphQL but it was still a bit cumbersome.
- deleted 7y ago
  [deleted]
- gregplaysguitar 7y ago
  We use https://graphql-code-generator.com/ https://graphql-code-generator.com/ in combination with some scripts which update the graphql schema at build time - works great. Using Apollo on the front end. Having types straddle the client/server divide is a huge win

396 points by dkns 7y ago

psv1 7y ago

Not a TypeScript user, but what really stood out to me is that Google are using a monorepo.

bad_user 7y ago
Google is using the monorepo ;-)
Dobbs 7y ago
Google is pretty notorious for this. It is one of the reasons behind the old golang GOPATH setup, and one of the reasons it took so long for the Go taking so long to get modules.
- jmillikin 7y ago
  I'm not sure that's true. The layout of Go code in Google's monorepo is not at all similar to GOPATH, and patterns that are common within google3 (such as multiple Go packages in one directory) are fundamentally incompatible with the Go build system. As an ex-Googler, I still strongly prefer the google3 style and am annoyed when open-source Go tooling can't deal with it properly.
  - nih0 7y ago
    what is google3 style?
    - papln 7y ago
      I think it's the mentioned > multiple Go packages in one directory as opposed to having a 1:1 package:dir mapping
heavenlyhash 7y ago
Be careful not to mistake Google's use of a monorepo with consideration of whether $DAYJOB or $FOSSPROJECT should use a monorepo. Google has a lot of tooling and some very thoroughly-considered and reinforced policies and cultures around their use of a monorepo. Trying to use a monorepo without those tools and ingrained policies may not be very likely to lead to similar results.
- arlk 7y ago
  for reference https://ai.google/research/pubs/pub45424 https://ai.google/research/pubs/pub45424
- rkangel 7y ago
  The default should be keeping code together, and justifying why you are splitting it up, not the other way around. While what you said is true, the codebase of most organisations is not large enough that they run into those scale constraints for a long time. Instead, if you split up your code you immediately get organisational headaches of managing changes across multiple codebases. If you keep it together, the scale challenges can be managed later, if they occur (c.f. YAGNI). If you are splitting, codebases should be split across organisational boundaries, not technical ones. If you look at the FOSS world, the obvious conclusion is that each library is in its own repo, and that is partly true. In practice what is happening is that each OSS project is its own organisational team, and so it lives together. If you have parts of your company on different continents working on different things (and maybe you need different access) then splitting up may make sense. Otherwise, you're probably just making your life harder for little to no gain.
  - heavenlyhash 7y ago
    I think I'd more or less agree with that. For example, one of the questions I generally ask in a meeting room that's considering this topic, and has for example microservices in flight, is: "Are you willing to put in the work to make Service A support more than one version at a time in Service B?", and if not, then that's a very (very) strong indicator that the level of coupling and lack of organizational boundaries will result in pain from anything but a monorepo. I prefer to split things up when possible. When it does fit, there's many benefits. There's also the concern that building too much culture and tooling that presumes a lack of splitting of repos can become its own form of trap which becomes increasingly hard to navigate out of, even if you later want to, as the situation self-iterates. But I agree that splitting things out needs justification, and the "default" stance should include that.
apalmer 7y ago
that stood out to me too... not the monorepo really, but the fact its a monorepo of a billion lines of code. seems almost impossible to maintain when you have a dependency change at the lowest levels that affects numerous projects.If i am understanding google was in a situation where they had to update every project that used typescript inside the entire company at the same time, that seems untenable.
- yoz-y 7y ago
  Mostly it means that you have to be really thoughtful when introducing a breaking change to a low level library.
  - papln 7y ago
    Or you can YOLO / Leroy Jenkins the change and let everyone else fix the breakage you create, or see if they demand a rollback.
    - yoz-y 7y ago
      They will not only demand a rollback but also do it. Besides, low level changes at this scale require a copious amount of approvals.
- larzang 7y ago
  Whether it's a monorepo or not doesn't change the fact that you have a billion lines of code to maintain, but at least this way they're forced to make changes consistently. The alternative would be doing it piecemeal with a dozen versions of dependencies propagating due to fragmentation, which is way way worse.
- forgot-my-pw 7y ago
  Here's an article from a maintainer of a framework library (Angular): https://medium.com/@Jakeherringbone/you-too-can-love-the-monorepo-d95d1d6fcebe https://medium.com/@Jakeherringbone/you-too-can-love-the-mon... Something like this might only be possible due to their tooling and test coverage. So when you change something, you immediately get alerted of broken tests.
kyrra 7y ago
Another interesting bit with the monorepo is the one-version-policy: https://opensource.google.com/docs/thirdparty/oneversion/ https://opensource.google.com/docs/thirdparty/oneversion/ That's why the typescript upgrade was so hard for them. We (attempt) to enforce a single version of a library/toolchain to be checked into the codebase at any given time. You can have multiple in during an upgrade, but it's highly discouraged.
- tylerhou 7y ago
  This is also why Google says to test everything; even minor version upgrades can have unexpected behavioral changes. Without tests, these might break your project without warning.
- papln 7y ago
  On the contrary, having multiple-versions could have made the upgrade much worse, by deferring compatibility problems from submit time to deploy time.
  - kyrra 7y ago
    It's a trade-off. As a user of a third-party library, I would like to upgrade it to get new functionality. But there are breaking changes in the update. So to upgrade I would have to fix all users of the library to upgrade. While this is better overall for the codebase, it can put a lot of work on others for a not well maintained third-party library. Something like TS has people that help keep it updated. But for something more obscure, it'll be on someone else who cares enough to put in the work.
slig 7y ago
How does that work on a nodejs project? I understand that they only have one version of the TypeScript compiler for all projects inside the monorepo, so that means there's only one huge package.json inside with all the packages used by every project inside the repo?
- jagthebeetle 7y ago
  The lingua franca for Google's building needs is (more or less) bazel, where you say target /a/b/c depends on /dep/v1_1, /dep/xyz, /common/foo, etc.). (There is a filesystem-like hierarchy parallel to, but not necessarily the same as the corresponding repository's directory layout.) Bazel is extensible via rules [1], so if you really wanted to use NodeJS on your team, you might create a `nodejs_binary` rule that put everything in the right directory and ran some NodeJS packager on it. You'd probably not put it into production. Also, third-party code lives in a single third-party directory, so yes, internal users could pull down code they wanted (and for which there wasn't a satisfactory internal version already) into that directory: https://opensource.google.com/docs/thirdparty/ https://opensource.google.com/docs/thirdparty/ [1]: https://docs.bazel.build/versions/0.29.0/skylark/rules.html https://docs.bazel.build/versions/0.29.0/skylark/rules.html

tannhaeuser 7y ago

Is there a way to migrate jsdoc-annotated JavaScript code over to TS, and is TS's minifier as good as Google's closure-compiler yet?

smt88 7y ago
I don't know of prod-quality tools to migrate from jsdoc to TS. TS does have comment-level typing, so you can get the benefits without transpiling anything (similar to Flow). TS pointedly does not minify output. It does the opposite: try to generate code that a human might have written. It's very easy to incorporate TS into a Babel or Webpack build pipeline though, so you can use a purpose-built minifier of your choice.
vbitz 7y ago
Visual Studio Code has a "Code Fix" that can automatically copies JSDoc into TypeScript annotations. This can be done to an entire file at once. TypeScript does not come with a minifier. The code it produces is compatible with the target version of JavaScript (e.g. compile ES6 modules into CommonJS) but unminified.
- tannhaeuser 7y ago
  I see. closure-compiler's "advanced" mode takes advantage of jsdoc type info for minifying so I guess generic syntactical minification won't compress as much for the time being.
  - evmar 7y ago
    We (Google TS team) maintain a tool[1] that transforms TS types into jsdoc types for the purpose of feeding them into 'advanced' mode. We also (to answer the grandparent question) maintain a tool[2] that converts Closure-annotated TS into JS. (Why both ways? We transition JS->TS and check it in as the code the user works with, while we use the TS->JS one within the compiler at optimization time.) [1] https://github.com/angular/tsickle https://github.com/angular/tsickle [2] 'gents', in this repo https://github.com/angular/clutz https://github.com/angular/clutz

andreigaspar 7y ago

So happy to see the attention Typescript is getting lately. Absolutely love the language, and have been using it since it got released.

umvi 7y ago
I tried using it with Angular, but it doesn't seem to help much. For example, if you have something like: <button (click)="login(email, pass)" /> And then a TypeScript function like: login(email: string, pass: string) { } TypeScript can't help you at all here because all the typing is determined at runtime by Angular. Even if `email` is a number or a boolean, no problem, it will just happily pass it in. What benefit, then, does TypeScript provide? I understand it's compile-time guarantees, but how does that help if the types are coming in from HTML land which the TypeScript compiler doesn't examine at all?
- olmo 7y ago
  Glad to see someone pints this out. That’s the main reason I use React + Typescrip. JSX is an extension of javascript and can be fully checked while any template language is a custom invention that it’s hardly toolable.
- threecreepio 7y ago
  For your template code in angular, there won't be any significant benefit to using typescript. It could be worse than not having typescript at all, since you can add type annotations to your 'login'-function that don't match up with reality. That's not really a typescript issue though, and it works great with libraries that don't use string templates. From what I've seen the angular community hasn't really prioritized a typecheck-able templating language.
  - dmix 7y ago
    Nor does Vue. The focus is on typing the reactive data that feeds the templates which should be sufficient. Plenty of things like HTML form elements take numbers or strings just fine, as it all outputs to strings in the end. Additionally, by breaking up stuff into smaller composable components there should be enough gating and typing layers, at least with Vue/Vuex that's the case. If it was just plopping straight into the elements 1-to-1 that might be a different story.

vallode 7y ago

Issues like these are the reason we can thrive as a community. Not playing a blame-game and offering direct feedback to a wonderful open source project. Trying to get my own company to get more involved in feedback to the open source tools we use as I think it is so extremely respectful and encouraging!

jannes 7y ago

I love TypeScript. I started using it around v1.0. Microsoft has hit some gold with it. When I first started using it I had lots of `any` in my code (like the Google employee is describing here). But over time it really starts being extremely clean.

radicalbyte 7y ago
I started around 0.7/0.8, it was already such a fantastic product back then. It's just fantastic, Javascript on steroids.
- Rychard 7y ago
  I don't have a significant amount of experience with TypeScript, but even from my limited experience I agree that it's a fantastic product. That said, at $JOB we (unfortunately) have a fair amount of production code written in TypeScript 0.9 which nobody has ever upgraded, and simply won't compile on a more recent version. It's been that way for years now, and every attempt to bring it up to date has been met with failure. It may be an overly broad request, but I'd be very interested if anyone had any suggestions for how we might go about performing such an upgrade in an iterative manner. My understanding is that pre-1.0, TypeScript went through a number of breaking changes (as would be expected of any pre-release software), but I've never found a complete list of what those breaking changes were.
  - pas 7y ago
    So, because it currently "works", it should "safely" compile by adding many `any`s. (No Implicit Any, and other strictness flags set to false.) Then it's a matter of cleaning them up. But usually, at least in my experience, the problem is the libraries and their API, which used to be very much full of any in those dark times.
  - jannes 7y ago
    You can see a list of all changes in the "What's New in TypeScript" document on GitHub: https://github.com/Microsoft/TypeScript/wiki/What's-new-in-TypeScript https://github.com/Microsoft/TypeScript/wiki/What's-new-in-T... And a list of breaking changes here: https://github.com/microsoft/TypeScript/wiki/Breaking-Changes https://github.com/microsoft/TypeScript/wiki/Breaking-Change... They both go back until v1.1. For older changes you can check the blog: https://devblogs.microsoft.com/typescript/announcing-typescript-1-0/ https://devblogs.microsoft.com/typescript/announcing-typescr... https://devblogs.microsoft.com/typescript/announcing-typescript-0-9-5/ https://devblogs.microsoft.com/typescript/announcing-typescr... Anyway, it's hard to say what you need to do to get your code to compile on the latest TypeScript version without knowing what some common compile errors are in your code. Here are some guesses: 1. The way you obtain type definitions for third-party packages has changed. It used to work with /// <reference> and tools like nuget or TSD (TypeScript definition manager). But now it works with npm in the @types namespace and some npm packages even include definitions themselves now. Of course, updating to the latest type definitions would mean that you have to upgrade to the latest versions of the third-party libraries as well. Otherwise you have to find a way to keep working with the old definitions for the respective library versions. 2. Pre-1.5 TypeScript had something called "internal modules" which were renamed to "namespaces" and are generally discouraged now. Hopefully your code is not using that feature. https://www.typescriptlang.org/docs/handbook/namespaces.html https://www.typescriptlang.org/docs/handbook/namespaces.html 3. tsconfig.json. I don't remember when they introduced it, but you likely need to create this file and tweak the settings

Stay_frostJebel 7y ago

This write-up led me to Evan Martins blog. What a gold mine. Digestible writing with a peek inside of Google's internals. http://neugierig.org/software/blog/ http://neugierig.org/software/blog/

heavenlyhash 7y ago

> (I might suggest the underlying problem in this code is relying on inference too much, but the threshold for "too much" is difficult to communicate to users.) This is a very outstandingly interesting line out the whole writeup. I like the writeup in its entirety for being very balanced and thoughtful, but this line in particular really stands out to me as worth more thought for anyone interested in language and type system design. Inference is great. Except... when it's not. When it's "too much". When it starts making breaking changes appear "too distantly". It's an interesting topic to reflect upon, because the inference isn't making a breakage; just shifting around where the breakage appears. (And this makes it hard to ever direct criticism at an inference mechanism!) But this kind of shifting-around of the breakage appearance can be a drastic impact on the ergonomics of handling it: how early it's detected, how close to the important change site tooling will be able to point the coder, etc. That's important. And almost everything about it involves the word "too" -- which means the area is incredibly subjective, and requires some kind of norm-building... while not necessarily providing any clear place for that norm-building to center around. I don't have a point here other than to say this is interesting to reflect on. I suspect the last chapter on type inference systems has not yet been written. Can an inference system be designed such that it naturally restrains "too" much use of it?

the_duke 7y ago
> because the inference isn't making a breakage; just shifting around where the breakage appears That's a very valuable insight, and a concern in a lot of languages with advanced inference! The problem is exacerbated in Typescript though because it is fundamentally a practical, evolved layer over Javascript, with an unsound type system that likes to bail out to any.
jimbo1qaz 7y ago
In my experience with type inference (quite a bit of mypy, now learning rust), inferring local variable types based on what's assigned to it is not very confusing, as long as me or the IDE can figure out what type is being assigned. But when a function return value or variable's generic argument is determined by where the value is used, I tend to have more trouble (since one type can propagate from one function, to its argument, to the function which generated the argument).

bsaul 7y ago

A little off-topic: I'm currently trying to find a way to write (type safe) business logic once, then reuse it pretty much everywhere (mobile / web / desktop),and it seems to me that typescript has become the only option. Javascript runtime is present everywhere, and can interface with anything. Does someone knows of another alternative (viable right now, or in the coming months) ? I know llvm can theoretically target any platform, including wasm, but how painfull is it in practice ? Can you write a line of code that does a network request then expect it to run as it is on the browser and on mobile platforms ?

GiorgioG 7y ago
C# (with .NET Core 3.0 officially launching on the 23rd of September, Blazor (WASM) support is included.) I have been playing around with Blazor a bit with the preview releases and I have to say it's pretty slick.
- stefano 7y ago
  Did they improve the download size?
  - cutler 7y ago
    No and on that basis it's unusable. However that's because it downloads the whole runtime. Eventually MS intend to load only your compiled code.
    - mattferderer 7y ago
      You can avoid that issue with the Server Side Blazor concept for business side portions of your application. Depending on the application you can get extremely far with just using HTTP requests to an API for anything business logic related. A lot of apps over complicate themselves by trying to force a JS framework on the front end with no benefit to the user. I seem to prefer using non-SPAs these days over SPAs because very few places do them well. Google being a terrible SPA developer. - Edit - I forgot to mention Elixir's Phoenix has LiveView which is similar to Server Side Blazor.

brlewis 7y ago

I'm really curious how many lines of TypeScript are in use at Google. I bet most JS is still Closure.

deleted 7y ago
[deleted]

msoad 7y ago

filter(Boolean) is a bad idea anyways. I was bitten by this before.

y2bd 7y ago
Why do you say so?
- papln 7y ago
  One reason: This isn't TS-specific; it's common in Python too: coercing to Bool has language semantics (for whatever language you are in) which often don't match the application semantics of your program. Application programmers don't (and shouldn't have to, but for the language's over-eager coercions) always think about the boolean semantics of all their objects. In particular, None and empty/zero object are both False in Python, and Python style/linters push you to avoid explicit comparison to None, which gets weird when your application wants to treat empty objects as True because they have differen semantics from None. (For example, in a security function, None may mean no-op / fallback to default, but Empty might mean "Reject all".
  - WorldMaker 7y ago
    Another reason that is TS-specific is that of JS functions can be highly variadic in the number of arguments and a lot of subtle runtime bugs can be found in blindly passing arguments without checking their count. If filter changes from returning only one thing to say two (for instance, an index count), Boolean may produce a runtime error for having too many arguments, may silently ignore extra arguments, may interpret an extra argument as changing the behavior, or some combination of all three depending on strictness versus compatibility level, executing browser, phase of the moon, etc. Boolean itself I've not had trouble with, but things like map(parseInt) is the big one that bites a lot of junior developers all the time in TS/JS. (parseInt takes an optional second argument for radix, so in cases where the second argument returned by map is an index count, which is likely, you get it parsing in base-0, base-1, base-2, … which is almost never something you'd do intentionally.)

lxe 7y ago

Almost every single Flow version upgrade is like this — every new version brings a slew of errors due to Flow’s continuous movement away from practicality towards “soundness”.

vicapow 7y ago
I’m not sure I agree. I’ve found updating Flow in a rather large monorepo a relatively straightforward process. The changes are usually rather small because the team releases every two weeks. They also manage the update internally within their company’s monorepo so they’ll usually find out about these types of unexpected behavior changes before the community does. That said, we do rely on automated error excludes (similar to eslint-ignore-next-line) for things that cannot be fixed with codemods. Those errors were always there it’s just now you know about them. Better to stem the bleeding by updating the type checker to the latest version.
- lxe 7y ago
  > I’ve found updating Flow in a rather large monorepo a relatively straightforward process. I'm not sure you're being entirely honest here, as we both know through feedback that flow updates in the said monorepo are one of the most burdensome processes for its contributors. As a maintainer/owner of a monorepo experience, it's crucial to maintain empathy and honesty of how processes such as dependency and tooling updates affect (and are perceived by) its users.
  - vicapow 7y ago
    Apologies, lxe. I can tell you're frustrated. If the the Flow upgrade process has been burdensome to others, I'd really like to know. I've always thought we've done a good job of updating internally on the platform team and that it hasn't been a concern for end users. We did run into some pain points in changing configuration options around the unnecessary optional-chaining lint rules but that wasn't related to updating the Flow version. I think you may be conflating those.
- lhorie 7y ago

burtonator 7y ago

Typescript is absolutely amazing. I've been working with it for the last 8 months. https://getpolarized.io/ https://getpolarized.io/ and the source is here: https://github.com/burtonator/polar-bookshelf https://github.com/burtonator/polar-bookshelf I could have NOT made as much progress just by using JS directly. When you have a large code-base and you're trying to make progress as fast as possible refactoring is needed and the strict typing is invaluable. Honestly, the MAIN issue with TS are issues around webpack + typings for 3rd party and older modules. I'd say 85% of the code I want to use already has types but when they don't it's frustrating to have to pause and write my own types. I have 20 years of Java experience. Used it since 1.0 and for the most part have been unhappy with everything else. I've decided that Node + Typescript is by far the most productive environment for me to switch to. I can code elegant apps with both front and backends and I get strict typing. Could NOT have made so much progress without TS.

patates 7y ago
I'm writing this as a developer who writes a lot of C# and Typescript: Have you tried C#? Even though I sometimes miss the flexibility of Typescript when writing in it, I love the reliability. Maybe it's no as battle-tested as Java (especially with all the rewrites recently), but feels like it's 99.9% there.
- htgb 7y ago
  Not GP, but as a developer working with TypeScript and C#, I respectfully disagree regarding the type system. There are many things I like better in C# compared to JavaScript, but I feel overly constrained by the type system of C# way more often, and most notable is the lack of discriminated unions (aka sum types etc). You can say that something has this and that, but not that it is this or that.
  - andrejk 7y ago
    I agree, as a mostly C# developer for years, then Typescript (Node, Angular and React) for the last couple of years, when I go back to C# projects, I feel like I'm doing a lot of work for the compiler. And really elegant use of the TS type system doesn't translate well into C# many times, forcing me to write boilerplate. I've been eyeing F# for more elegant managed code, but that's going to take me a little more up-front investment to get productive.

jbverschoor 7y ago

Google sure has a lot of comments these days

hnsocks 7y ago

Testing shadow ban: google and Facebook sucks more than socks time to ban me

dmix 7y ago

> but any time someone saves a Selection into a member variable, they ended up writing down whatever type TS inferred at that time, e.g. > mySel: d3.Selection<HTMLElement, {}, null, undefined>; I'm curious how Google and others approach adopting Typescript gradually, as I'm pretty new to it, I'm assuming it goes like: The programmer converts code to Typescript and when they come across return types they copy the inferred type and add it to the codebase directly wherever possible. I'm assuming just as a matter of using (untyped) libraries you need to rely on the output of Typescript in order to try have every return typed. So the biggest problem seems to be how TS infers things changed meaning you can't always trust what you copied as staying consistent, even if the source library doesn't change itself. That's always something to keep in mind for overhead.

gmoot 7y ago
The explicit type is optional. It's not that it's required to copy out whatever the compiler inferred, it's just that lots of people do it to be explicit.
- dmix 7y ago
  All types are optional in typescript. I was just curious about Google's approach to it and if they had some standard practice of always copying the inferred type to get a great amount of coverage.
  - lonelappde 7y ago
    Copying the inferred type is the same as copying your runtime outputs into your tests. It's a statement that you believe the result is correct and it's on you to make that judgement call. It's not a policy to blindly copy everything, because that defeats the purpose of type checking and testing, turning your tests into " verify that nothing changed", not "verify that the system behaves as intended".
    - dmix 7y ago
      Thanks, thats typically how I approach it as well, it's good to hear it written out. I try not to append copied types without fully understanding their structure either. But sometimes you just need to just trust it to solve a problem. There's plenty of blog posts and docs on advanced types but I'm interested in the day-to-day best practical approaches people are taking adopting it. I should look around for some literature or talks on the subject...

_the_inflator 7y ago

It is very funny to read this from Google, since we talked about similar problems with Google regarding Angular upgrades, where features were modified that Google considered not used/no use cases known, while we were relying on these. From my point of view Core members of any super large project (like React, Angular, TypeScript) are limited by design in what they perceive as their target audience and their use cases. This is simply a matter of fact: even as a core dev you cannot know how every dev uses your product. So this is some sort of left-pad moment for TypeScript.

juliendc 7y ago

I've recently built a Node server with TypeScript and it's a joy to use with external libraries when the types are available. It's such a time saver to not have to guess which method to call with which arguments (I've had only experience with dynamic languages before). Some libraries don't have types or they are outdated but it was a minority. With the experience I've found that most of the type errors are actually between the backend and the frontend in web applications. It's still hard to fully type the entire flow from the database calls with the ORM to the objects manipulation in the frontend. How are you dealing with that? We used Nexus with GraphQL but it was still a bit cumbersome.

deleted 7y ago
[deleted]
gregplaysguitar 7y ago
We use https://graphql-code-generator.com/ https://graphql-code-generator.com/ in combination with some scripts which update the graphql schema at build time - works great. Using Apollo on the front end. Having types straddle the client/server divide is a huge win