5 ms·

Architecture for a JavaScript to C compiler

90 points by timruffles 8y ago

maxxxxx 8y ago
With all the dynamic stuff Javascript has it seems really difficult to create performant C code. There is a PHP to .NET compiler which probably has similar problems. On second thought that one is probably easier because .NEt has a dynamic runtime.
- nicoburns 8y ago
  That, and in practice modern PHP is often relatively staticly typed (classes declare their fields, etc), and many codebases even include type annotations (which are supported in PHP and usually enforced at runtime).
rkeene2 8y ago
A lot of research and hard work has gone into TclQuadCode [0], which compiles Tcl (which is even more dynamic than JavaScript) into machine code via LLVM. The authors indicated at one point it took around 5 PhDs to get it going. [0] https://core.tcl.tk/tclquadcode/dir?ci=trunk https://core.tcl.tk/tclquadcode/dir?ci=trunk
- ridiculous_fish 8y ago
  This is bizarre and fascinating. I had no idea there were Tcl codebases of a size that could benefit from this sort of perf work. How much Tcl is out there?
snek 8y ago
I don't understand why people keep making js-to-native compilers. the result is always less performance that what modern js engines can do to the original source code, because all those runtime behaviours the author mentions build up really fast. js engines actually run the code and figure out the types and such to create highly optimized machine code.
- nicoburns 8y ago
  Yeah, the JS runtimes are really impressive. I'd be interested to see what a compiler could do with TypeScript though. There could perhaps be guidelines of which features you can use in hot code paths in order to get optimised code, which could be enforced by the compiler for sections of the code which you mark...
  - chrisseaton 8y ago
    I would imagine JS JITs can determine more specific types at runtime than you can express in TypeScript source code.
    - jdc 8y ago
      Could you ELI5 why you think that?
      - chrisseaton 8y ago
        TypeScript lets you annotate that a parameter will only ever be a number, but the JIT can find that out for itself by seeing that it’s only ever a number when it runs the program and a number is all you pass to it, using stamps and class profiling. The JIT can also see types beyond that, like that it’s only ever a number and it’s always below 0xff, using stamps and value profiling. The JIT has to guard that the type is as expected, yes, but so would the static compiler in order to do the more specific types.
      - z3t4 8y ago
        Lets say you've built a house in lego. But the computer do not know it's actually a house. Something called a compiler has to take apart the lego house, then package the parts in groups of 8,16,32 or 64 (dissembles the house into stacks with 8 lego bit's in each). Then the compILer have to tell the compUter what to do with the stacks, and what each stack is. For example take this stack with red pieces and this stack with green pieces and build a wall, then take these bits, and combine with these bits to build a roof, etc. The computer is dumb, but it's very fast.
ridiculous_fish 8y ago
How are exceptions handled with this design? For example the `n < 3` may throw (it invokes the valueOf method).
- timruffles 8y ago
  I went for longjmp(), non-local goto, for precisely the reason you highlighted: I realised pretty well everything in JS can throw, so needed something easy to trigger from anywhere See https://github.com/timruffles/js-to-c/blob/1befbf4220753576e2d08c3607ab445c3ddad9ea/runtime/exceptions.c https://github.com/timruffles/js-to-c/blob/1befbf4220753576e...
ndesaulniers 8y ago
I actually think this is possible; and started prototyping it (because esprima is awesome, and not to many other languages have an equivalent that's so easy to use). Some thoughts: I think it's easier to target C++ than C, since C++ can help you write more type generic code. I think it's easy to generate tagged unions, then for optimizations try to prove monomorphism. Finally, it may be simpler to start off with support for typescript, and fail to compile if there are any ANY types. I do think it's possible though. JS/TS -> C++ -> WASM (yes, I was out of my mind when I thought of this)
- johnhenry 8y ago
  Isn't this kind of what V8 and other modern JavaScript engines do on the fly already?
  - mikece 8y ago
    Yes: one of the Google engineers working on V8 talked about it here: https://softwareengineeringdaily.com/2018/10/03/javascript-and-the-inner-workings-of-your-browser/ https://softwareengineeringdaily.com/2018/10/03/javascript-a... It was this conversation that make me wonder if at some point in the future V8 might have experimental support natively for TypeScript but it makes more sense that compiling web assembly to native binary would make more sense. Who knows? It's an awesome time to be a programmer!
truth_seeker 8y ago
I wonder how much optimization will it bring to compared to existing JS runtimes such as V8. Thanks to competing world for web browsers, JS runtimes not only efficiently parse to optimized native code but also provide really good JIT compilation benefits. Speculative optimization for V8 - https://ponyfoo.com/articles/an-introduction-to-speculative-optimization-in-v8 https://ponyfoo.com/articles/an-introduction-to-speculative-... Parallel and Concurrent GC - https://v8.dev/blog/trash-talk https://v8.dev/blog/trash-talk Good summary on 10 years of V8 - https://v8.dev/blog/10-years https://v8.dev/blog/10-years
- ridiculous_fish 8y ago
  v8-style engines do not parse to optimized native code. As described in the links, v8 parses to an AST, which then is compiled to bytecode. A bytecode VM then executes the JS, collecting runtime type information, which is input (along with the bytecode itself) into the next compilation tier; only at that point is machine code generated. The key idea is that v8 expects to execute the JS code before it can generate native code. It won't generate native code from parsing alone.
tannhaeuser 8y ago
Congrats to completing this project. What's the status and further plans for it? I didn't find a license.
magwa101 8y ago
Write it in C, there, problem solved.
maxgraey 8y ago
I wonder why people still try write transpiler from JS to C/C++ or LLVM (which make sense at least). But this not performant way and usually produce much bigger overhead than jit vm which use speculative optimizations. Some projects: 1. https://github.com/fabiosantoscode/js2cpp https://github.com/fabiosantoscode/js2cpp 2. https://github.com/raphamorim/js2c https://github.com/raphamorim/js2c 3. https://github.com/ammer/js2c https://github.com/ammer/js2c 4. https://github.com/NectarJS/nectarjs https://github.com/NectarJS/nectarjs 5. https://github.com/ovr/StaticScript https://github.com/ovr/StaticScript

90 points by timruffles 8y ago

With all the dynamic stuff Javascript has it seems really difficult to create performant C code. There is a PHP to .NET compiler which probably has similar problems. On second thought that one is probably easier because .NEt has a dynamic runtime.

nicoburns 8y ago
That, and in practice modern PHP is often relatively staticly typed (classes declare their fields, etc), and many codebases even include type annotations (which are supported in PHP and usually enforced at runtime).

rkeene2 8y ago

A lot of research and hard work has gone into TclQuadCode [0], which compiles Tcl (which is even more dynamic than JavaScript) into machine code via LLVM. The authors indicated at one point it took around 5 PhDs to get it going. [0] https://core.tcl.tk/tclquadcode/dir?ci=trunk https://core.tcl.tk/tclquadcode/dir?ci=trunk

ridiculous_fish 8y ago
This is bizarre and fascinating. I had no idea there were Tcl codebases of a size that could benefit from this sort of perf work. How much Tcl is out there?

snek 8y ago

I don't understand why people keep making js-to-native compilers. the result is always less performance that what modern js engines can do to the original source code, because all those runtime behaviours the author mentions build up really fast. js engines actually run the code and figure out the types and such to create highly optimized machine code.

nicoburns 8y ago
Yeah, the JS runtimes are really impressive. I'd be interested to see what a compiler could do with TypeScript though. There could perhaps be guidelines of which features you can use in hot code paths in order to get optimised code, which could be enforced by the compiler for sections of the code which you mark...
- chrisseaton 8y ago
  I would imagine JS JITs can determine more specific types at runtime than you can express in TypeScript source code.
  - jdc 8y ago
    Could you ELI5 why you think that?
    - chrisseaton 8y ago
      TypeScript lets you annotate that a parameter will only ever be a number, but the JIT can find that out for itself by seeing that it’s only ever a number when it runs the program and a number is all you pass to it, using stamps and class profiling. The JIT can also see types beyond that, like that it’s only ever a number and it’s always below 0xff, using stamps and value profiling. The JIT has to guard that the type is as expected, yes, but so would the static compiler in order to do the more specific types.
    - z3t4 8y ago
      Lets say you've built a house in lego. But the computer do not know it's actually a house. Something called a compiler has to take apart the lego house, then package the parts in groups of 8,16,32 or 64 (dissembles the house into stacks with 8 lego bit's in each). Then the compILer have to tell the compUter what to do with the stacks, and what each stack is. For example take this stack with red pieces and this stack with green pieces and build a wall, then take these bits, and combine with these bits to build a roof, etc. The computer is dumb, but it's very fast.

ridiculous_fish 8y ago

How are exceptions handled with this design? For example the `n < 3` may throw (it invokes the valueOf method).

timruffles 8y ago
I went for longjmp(), non-local goto, for precisely the reason you highlighted: I realised pretty well everything in JS can throw, so needed something easy to trigger from anywhere See https://github.com/timruffles/js-to-c/blob/1befbf4220753576e2d08c3607ab445c3ddad9ea/runtime/exceptions.c https://github.com/timruffles/js-to-c/blob/1befbf4220753576e...

ndesaulniers 8y ago

I actually think this is possible; and started prototyping it (because esprima is awesome, and not to many other languages have an equivalent that's so easy to use). Some thoughts: I think it's easier to target C++ than C, since C++ can help you write more type generic code. I think it's easy to generate tagged unions, then for optimizations try to prove monomorphism. Finally, it may be simpler to start off with support for typescript, and fail to compile if there are any ANY types. I do think it's possible though. JS/TS -> C++ -> WASM (yes, I was out of my mind when I thought of this)

johnhenry 8y ago
Isn't this kind of what V8 and other modern JavaScript engines do on the fly already?
- mikece 8y ago
  Yes: one of the Google engineers working on V8 talked about it here: https://softwareengineeringdaily.com/2018/10/03/javascript-and-the-inner-workings-of-your-browser/ https://softwareengineeringdaily.com/2018/10/03/javascript-a... It was this conversation that make me wonder if at some point in the future V8 might have experimental support natively for TypeScript but it makes more sense that compiling web assembly to native binary would make more sense. Who knows? It's an awesome time to be a programmer!

truth_seeker 8y ago

I wonder how much optimization will it bring to compared to existing JS runtimes such as V8. Thanks to competing world for web browsers, JS runtimes not only efficiently parse to optimized native code but also provide really good JIT compilation benefits. Speculative optimization for V8 - https://ponyfoo.com/articles/an-introduction-to-speculative-optimization-in-v8 https://ponyfoo.com/articles/an-introduction-to-speculative-... Parallel and Concurrent GC - https://v8.dev/blog/trash-talk https://v8.dev/blog/trash-talk Good summary on 10 years of V8 - https://v8.dev/blog/10-years https://v8.dev/blog/10-years

ridiculous_fish 8y ago
v8-style engines do not parse to optimized native code. As described in the links, v8 parses to an AST, which then is compiled to bytecode. A bytecode VM then executes the JS, collecting runtime type information, which is input (along with the bytecode itself) into the next compilation tier; only at that point is machine code generated. The key idea is that v8 expects to execute the JS code before it can generate native code. It won't generate native code from parsing alone.

tannhaeuser 8y ago

Congrats to completing this project. What's the status and further plans for it? I didn't find a license.

magwa101 8y ago

Write it in C, there, problem solved.

maxgraey 8y ago

I wonder why people still try write transpiler from JS to C/C++ or LLVM (which make sense at least). But this not performant way and usually produce much bigger overhead than jit vm which use speculative optimizations. Some projects: 1. https://github.com/fabiosantoscode/js2cpp https://github.com/fabiosantoscode/js2cpp 2. https://github.com/raphamorim/js2c https://github.com/raphamorim/js2c 3. https://github.com/ammer/js2c https://github.com/ammer/js2c 4. https://github.com/NectarJS/nectarjs https://github.com/NectarJS/nectarjs 5. https://github.com/ovr/StaticScript https://github.com/ovr/StaticScript