8 ms·

I think TypeScript is uniquely positioned to be the optimal language for LLMs. Tons of training data (benefiting from all the JS examples as well) plus the stru

I think TypeScript is uniquely positioned to be the optimal language for LLMs. Tons of training data (benefiting from all the JS examples as well) plus the structure of types for LLMs to follow and tools to enforce.

yoyohello13 1y ago
God help us…
- marviel 1y ago
  what do you dislike about it?
  - treyd 1y ago
    TypeScript is arguably one of the weaker statically typed languages, with how it allows `any` to quietly violate the type checked assumptions. It makes it harder to do a lot of the basic typing mistakes in JS, but it doesn't prevent them by any means, especially if you have to interface with (typeless) JS code. So for these reasons alone I would be against using TS as a lingua franca for LLM codegen (as is GP I assume). As another commenter mentioned, LLMs have a tendency to throw their hands^Hlogits up when presented with complex TS type errors and just resort to using `any` to get it to compile (and probably hiding bugs). And that doesn't even touch the issues with the JS/TS ecosystem and runtimes more broadly.
    - agos 1y ago
      tsc can be configured to avoid implicit use of any ("noImplicitAny": true) and ESLint can be set up to avoid explicit use of any. Typeless JS code is also a thing of the past. But the devil is in the details - some libraries are typed quite crappily, some have unnecessary complex types, and the code that the LLMs was trained on is probably not the best in the world
      - tpm 1y ago
        Can be configured, but then you get to work at a real codebase halfheartedly half-converted from javascript with half the files beginning with ts-ignore. However crappy your Java codebase is going to be, it will still use types. And as just today Gemini hallucinated an API call that never existed (in a widely available and used library even), it's just better to have the ability to check that right away.
        agos 1y ago
        the codebases I've worked at in the last ten years are not as half arsed as that - and of course from my point of view are "real" enough. If a codebase is so unkempt the issue is not Typescript - and forgive for writing such a platitude, but you can write awful code in Java, too.
        tpm 1y ago
        Yes but it will be typed awful code and the typing provides a grounding of sorts. However awful code in untyped/dynamically typed langs can be unspeakably bad. I have many years of Perl experience...
OutOfHere 1y ago
There are languages that constrain types a lot more tightly than TypeScript, e.g. Kotlin, Rust, and Haskell. The more constrained the types, the more correct the program could be.
- mindwok 1y ago
  Yep, and Rust famously goes beyond this by modelling memory ownership at compile time. In fact, the more behaviour we can model at compile time the better when it comes to LLMs - there's some cool ideas here like transpiling Rust into languages for formal verification. See https://github.com/formal-land/coq-of-rust https://github.com/formal-land/coq-of-rust as an example. Formal verification was one of those things that was previously so annoying to do that it rarely made it past academic use cases or extremely important libraries, but I think LLMs take the tedium out of it. Perhaps formal verification will have a "test driven development" type of moment in the sun thanks to this.
  - koakuma-chan 1y ago
    Can LLMs properly code in Rust yet? There is way more TypeScript code out there compared to Rust, and I doubt structured output can alleviate this.
    - steveklabnik 1y ago
      They can, yes.
    - vessenes 1y ago
      my experience - yes! but. It's more of an edit - compile - fix loop than a write (seemingly) correct code on the first try. This might be a feature. There is a little occasional difficulty on syntax with rust, but there are often the same sort of logic errors / getting lost an llm would have on another codebase -- the compiler helps catch many of these.
      - NitpickLawyer 1y ago
pram 1y ago
LLMs work well with any static analysis tool. I frequently instruct Claude to use stuff like “go vet” and “deadcode” when it goes on a tear and writes a bunch of broken trash and declares mission accomplished.
- koakuma-chan 1y ago
  > LLMs work well with any static analysis tool. tsc error messages are so bad that every time my LLM sees one of those "SomeType is not assignable to SomeLongAssTypeDontEvenTryToUnderstandWhatsGoingOnHere<<<<>>>>>>>>>>>>>>>>>>>>" it just gives up and casts to any. goes for python too.
  - floydnoel 1y ago
    ha, that's always been my biggest gripe with ts
AaronAPU 1y ago
I can’t be the only one who hopes this was a joke.
- AnthonBerg 1y ago
  I believe that the rutabaga is the perfect material to make sausages out of as it has proven as excellent swine fodder with widespread adoption! (Please forgive me the extreme disrespect put forth in the above statement! It is not the intention to show disrespect; I… am quite the rutabaga enjoyer in all respects, you know? I certainly include myself within the absurdity and it is with love.)
babyent 1y ago
It’s better sure but as a power TS user it still sucks at generating better code, and consistently fucks up with generics (or doesn’t use them) or simple types sometimes.
primitivesuave 1y ago
Completely agree. Even with the basic LLMs in the $20/month Cursor plan, I can work 10x faster on TypeScript codebases than I could otherwise, while for Python that multiple feels more like 2-3x. The autocompletions are especially impressive when there is a well-organized type system. Also in response to adjacent commenters - many mission-critical TS codebases will disable the use of an explicit "any" with eslint - https://typescript-eslint.io/rules/no-explicit-any/ https://typescript-eslint.io/rules/no-explicit-any/.
johnmw 1y ago
Those who agree might be interested in "Introducing TypeChat" by Anders Hejlsberg + others (2023) [1] [1]: https://microsoft.github.io/TypeChat/blog/introducing-typechat/ https://microsoft.github.io/TypeChat/blog/introducing-typech...
- dcsan 1y ago
  Wish this project had more traction. Typechat with type checking could generate lots of synthetic data for model training too
miki123211 1y ago
And unlike many other languages, Typescript types are extremely expressive. For example, you can write a function that takes an object received from an API that uses snake_cased keys, and returns that same object, but with camelCased keys instead. This is not some "special case" in the Typescript compiler, the ability to do this emerges naturally from Typescript's features. I don't know any other language that can do this. Most people don't know enough TS to use tese things effectively, but I think one could train an LLM to be very good at them. The combination of LLMs placing such advanced constraints on themselves, and then generating code based on those constraints, seems extremely powerful.
rfoo 1y ago
> Tons of training data (benefiting from all the JS examples as well) More != better.
threeseed 1y ago
Scala would be the best given that its type system is formally modelled: https://infoscience.epfl.ch/entities/publication/6c6bb09d-a41c-46e8-aac4-d059cc8a6459 https://infoscience.epfl.ch/entities/publication/6c6bb09d-a4...

yoyohello13 1y ago

God help us…

marviel 1y ago
what do you dislike about it?
- treyd 1y ago
  TypeScript is arguably one of the weaker statically typed languages, with how it allows `any` to quietly violate the type checked assumptions. It makes it harder to do a lot of the basic typing mistakes in JS, but it doesn't prevent them by any means, especially if you have to interface with (typeless) JS code. So for these reasons alone I would be against using TS as a lingua franca for LLM codegen (as is GP I assume). As another commenter mentioned, LLMs have a tendency to throw their hands^Hlogits up when presented with complex TS type errors and just resort to using `any` to get it to compile (and probably hiding bugs). And that doesn't even touch the issues with the JS/TS ecosystem and runtimes more broadly.
  - agos 1y ago
    tsc can be configured to avoid implicit use of any ("noImplicitAny": true) and ESLint can be set up to avoid explicit use of any. Typeless JS code is also a thing of the past. But the devil is in the details - some libraries are typed quite crappily, some have unnecessary complex types, and the code that the LLMs was trained on is probably not the best in the world
    - tpm 1y ago
      Can be configured, but then you get to work at a real codebase halfheartedly half-converted from javascript with half the files beginning with ts-ignore. However crappy your Java codebase is going to be, it will still use types. And as just today Gemini hallucinated an API call that never existed (in a widely available and used library even), it's just better to have the ability to check that right away.
      - agos 1y ago
        the codebases I've worked at in the last ten years are not as half arsed as that - and of course from my point of view are "real" enough. If a codebase is so unkempt the issue is not Typescript - and forgive for writing such a platitude, but you can write awful code in Java, too.
        tpm 1y ago
        Yes but it will be typed awful code and the typing provides a grounding of sorts. However awful code in untyped/dynamically typed langs can be unspeakably bad. I have many years of Perl experience...

OutOfHere 1y ago

There are languages that constrain types a lot more tightly than TypeScript, e.g. Kotlin, Rust, and Haskell. The more constrained the types, the more correct the program could be.

mindwok 1y ago
Yep, and Rust famously goes beyond this by modelling memory ownership at compile time. In fact, the more behaviour we can model at compile time the better when it comes to LLMs - there's some cool ideas here like transpiling Rust into languages for formal verification. See https://github.com/formal-land/coq-of-rust https://github.com/formal-land/coq-of-rust as an example. Formal verification was one of those things that was previously so annoying to do that it rarely made it past academic use cases or extremely important libraries, but I think LLMs take the tedium out of it. Perhaps formal verification will have a "test driven development" type of moment in the sun thanks to this.
- koakuma-chan 1y ago
  Can LLMs properly code in Rust yet? There is way more TypeScript code out there compared to Rust, and I doubt structured output can alleviate this.
  - steveklabnik 1y ago
    They can, yes.
  - vessenes 1y ago
    my experience - yes! but. It's more of an edit - compile - fix loop than a write (seemingly) correct code on the first try. This might be a feature. There is a little occasional difficulty on syntax with rust, but there are often the same sort of logic errors / getting lost an llm would have on another codebase -- the compiler helps catch many of these.
    - NitpickLawyer 1y ago

pram 1y ago

LLMs work well with any static analysis tool. I frequently instruct Claude to use stuff like “go vet” and “deadcode” when it goes on a tear and writes a bunch of broken trash and declares mission accomplished.

koakuma-chan 1y ago
> LLMs work well with any static analysis tool. tsc error messages are so bad that every time my LLM sees one of those "SomeType is not assignable to SomeLongAssTypeDontEvenTryToUnderstandWhatsGoingOnHere<<<<>>>>>>>>>>>>>>>>>>>>" it just gives up and casts to any. goes for python too.
- floydnoel 1y ago
  ha, that's always been my biggest gripe with ts

AaronAPU 1y ago

I can’t be the only one who hopes this was a joke.

AnthonBerg 1y ago
I believe that the rutabaga is the perfect material to make sausages out of as it has proven as excellent swine fodder with widespread adoption! (Please forgive me the extreme disrespect put forth in the above statement! It is not the intention to show disrespect; I… am quite the rutabaga enjoyer in all respects, you know? I certainly include myself within the absurdity and it is with love.)

babyent 1y ago

It’s better sure but as a power TS user it still sucks at generating better code, and consistently fucks up with generics (or doesn’t use them) or simple types sometimes.

primitivesuave 1y ago

Completely agree. Even with the basic LLMs in the $20/month Cursor plan, I can work 10x faster on TypeScript codebases than I could otherwise, while for Python that multiple feels more like 2-3x. The autocompletions are especially impressive when there is a well-organized type system. Also in response to adjacent commenters - many mission-critical TS codebases will disable the use of an explicit "any" with eslint - https://typescript-eslint.io/rules/no-explicit-any/ https://typescript-eslint.io/rules/no-explicit-any/.

johnmw 1y ago

Those who agree might be interested in "Introducing TypeChat" by Anders Hejlsberg + others (2023) [1] [1]: https://microsoft.github.io/TypeChat/blog/introducing-typechat/ https://microsoft.github.io/TypeChat/blog/introducing-typech...

dcsan 1y ago
Wish this project had more traction. Typechat with type checking could generate lots of synthetic data for model training too

miki123211 1y ago

And unlike many other languages, Typescript types are extremely expressive. For example, you can write a function that takes an object received from an API that uses snake_cased keys, and returns that same object, but with camelCased keys instead. This is not some "special case" in the Typescript compiler, the ability to do this emerges naturally from Typescript's features. I don't know any other language that can do this. Most people don't know enough TS to use tese things effectively, but I think one could train an LLM to be very good at them. The combination of LLMs placing such advanced constraints on themselves, and then generating code based on those constraints, seems extremely powerful.

rfoo 1y ago

> Tons of training data (benefiting from all the JS examples as well) More != better.

threeseed 1y ago

Scala would be the best given that its type system is formally modelled: https://infoscience.epfl.ch/entities/publication/6c6bb09d-a41c-46e8-aac4-d059cc8a6459 https://infoscience.epfl.ch/entities/publication/6c6bb09d-a4...