10 ms·

So am I identifying the bottlenecks that motivate this design correctly? 1. Go FFI is slow 2. Per-proto generated code specialization is slow, because of icac

So am I identifying the bottlenecks that motivate this design correctly? 1. Go FFI is slow 2. Per-proto generated code specialization is slow, because of icache pressure I know there's more to the optimization story here, but I guess these are the primary motivations for the VM over just better code generation or implementing a parser in non-Go?

johnisgood 1y ago
I keep hearing that Go's C FFI is slow, why is that? How much slower is it in comparison to other languages?
- malkia 1y ago
  I've asked ChatGPT to summarize (granted my prompt might not be ideal), but some points to note, here just first in details others in the link at the bottom: Calling C from Go (or vice versa) often requires switching from Go's lightweight goroutine model to a full OS thread model because: - Go's scheduler manages goroutines on M:N threads, but C doesn't cooperate with Go's scheduler. - If C code blocks (e.g., on I/O or mutex), Go must assume the worst and parks the thread, spawning another to keep Go alive. * Cost: This means entering/exiting cgo is significantly more expensive than a normal Go call. There’s a syscall-like overhead. ... This was only the first issue, but then it follows with "Go runtime can't see inside C to know is it allocating, blocking, spinning, etc.", then "Stack switching", "Thread Affinity and TLS", "Debug/Profiling support overhead", "Memory Ownership and GC barriers" All here - https://chatgpt.com/share/688172c3-9fa4-800a-9b8f-e1252b57d08d https://chatgpt.com/share/688172c3-9fa4-800a-9b8f-e1252b57d0...
  - johnisgood 1y ago
    Just to roll with your way: https://chatgpt.com/share/688177c9-ebc0-8011-88cc-9514d8e1674b https://chatgpt.com/share/688177c9-ebc0-8011-88cc-9514d8e167... Please do not take the numbers below at face value. I still expect an actual reply to my initial comment. Per-call overhead: C (baseline) - ~30 ns Rust (unsafe) - ~30 ns C# (P/Invoke) - ~30-50 ns LuaJIT - ~30-50 ns Go (cgo) - ~40-60 ns Java (22, FFM) - ~40-70 ns Java (JNI) - ~300-1000 ns Perl (XS) - ~500-1000 ns Python (ctypes) - ~10,000-30,000 ns Common Lisp (SBCL) - ~500-1500 ns Seems like Go is still fast enough as opposed to other programming languages with GC, so I am not sure it is fair to Go.
    - Cyph0n 1y ago
      > Rust (unsafe) As if there is an alternative :) More seriously, it’s “unsafe” from the perspective of the library calling into C, but usually “safe” for any layer above.
      - johnisgood 1y ago
        Hey, since I am in a thread where we are sharing what ChatGPT spits out, I just copy pasted it from there, too. :) For what it is worth, I asked about LuaJIT after I have shared the link, and the numbers are now different for some languages, albeit not by much. Go (cgo) became ~50-100 ns. That said, I still believe it is unfair to single out Go when it does way better than some other GC languages.
    - throwaway7783 1y ago
      Java now has FFM, that is far better and simpler than JNI, FWIW. and chatgpt says Language/API | Call Overhead (no-op C) | Notes Go (cgo) | ~40–60 ns | Stack switch + thread pinning Java FFM | ~50 ns (downcall) | Similar to JNI, can be ~30 ns with isTrivial() Java FFM (leaf) | ~30–40 ns | Optimized (isTrivial=true) JNI | ~50–60 ns | Slightly slower than FFM Rust (unsafe) | ~5–20 ns | Near-zero overhead C# (P/Invoke) | ~20–50 ns | Depends on marshaling Python (cffi) | 1000–10000 ns | Orders of magnitude slower |
      - johnisgood 1y ago
        Thanks, I added it to the list. Keep in mind that the numbers may be off (both yours and mine), so I would not take them at face value. It is interesting how in yours JNI is still pretty good. Also Rust is "~5–20 ns" in yours, so I assume "0" is the baseline.
        throwaway7783 1y ago
        This is chatgpt. Not my own benchmark. So it is probably hallucinating
- pornel 1y ago
  Go's goroutines aren't plain C threads (blocking syscalls are magically made async), and Go's stack isn't a normal C stack (it's tiny and grown dynamically). A C function won't know how to behave in Go's runtime environment, so to call a C function Go needs make itself look more like a C program, call the C function, and then restore its magic state. Other languages like C++, Rust, and Swift are similar enough to C that they can just call C functions directly. CPython is a C program, so it can too. Golang was brave enough to do fundamental things its own way, which isn't quite C-compatible.
  - johnisgood 1y ago
    What about languages like Java, or other popular languages with GC?
    - andrewflnr 1y ago
      Similar enough to C I guess, at least in their stack layout.
    - lmm 1y ago
      Java FFI is slow and cumbersome, even more so if you're using the fancy auto-async from recent versions. The JVM community has mostly bitten the bullet and rewritten the entire world in Java rather than using native libraries, you only see JNI calls for niche things like high performance linear algebra; IMO that was the right tradeoff but it's also often seen as e.g. the reason why Java GUIs on the desktop suck. Other languages generally fall into either camp of having a C-like stack and thread model and easy FFI (e.g. Ruby, TCL, OCaml) and maybe having futures/async but not in an invisible/magic way, or having a radically different threading model at the cost of FFI being slow and painful (e.g. Erlang). JavaScript is kind of special in having C-like stack but being built around calling async functions from a global event loop, so it's technically the first but feels more like the second.
      - hinkley 1y ago
        JNI is the second or maybe third FFI for Java. JRI existed before it and that was worse, including performance. The debugging and instrumentation interfaces have been rewritten more times. https://docs.oracle.com/en/java/javase/24/docs/specs/jni/intro.html https://docs.oracle.com/en/java/javase/24/docs/specs/jni/int... mentions JRI. But it seems like JNI has been replaced by third party solutions multiple times as well. https://developer.okta.com/blog/2022/04/08/state-of-ffi-java#a-brief-history-of-ffi-in-java https://developer.okta.com/blog/2022/04/08/state-of-ffi-java...
- 3836293648 1y ago
  Go's threading model involves a lot of tiny (but growable) stacks and calling C functions almost immediately stack overflows. Calling C safely is then slow because you have to allocate a larger stack, copy data around and mess with the GC.
- deleted 1y ago
  [deleted]
- 9rx 1y ago
  > How much slower is it in comparison to other languages? It's about the same as most other languages that aren't specifically optimized for C calling. Considerably faster than Python. Which is funny as everyone on HN loves to extol the virtues of Python being a "C DSL" and never think twice about its overhead, but as soon as the word Go is mentioned its like your computer is going to catch fire if you even try. Emotion-driven development is a bizarre world.
  - johnisgood 1y ago
    Yeah, that is why I am asking.
unbrice 1y ago
3. The use case is dynamic schemas and access is through the reflection API. Thus PGO has to be done at runtime...
hinkley 1y ago
I know that Java resisted improving their FFI for years because they preferred that the JIT get the extra resources. And that customers not bail out of Java every time they couldn’t figure out how to make it faster. There’s a case I recall from when HotSpot was still young, where the Java GUI team moved part of the graphics pipeline to the FFI in one release, hotspot got faster in the next, and then they rolled back the changes because it was now faster without the FFI. But eventually your compiler is good enough that the FFI Is now your bottleneck, and you need to do something.

johnisgood 1y ago

I keep hearing that Go's C FFI is slow, why is that? How much slower is it in comparison to other languages?

malkia 1y ago
I've asked ChatGPT to summarize (granted my prompt might not be ideal), but some points to note, here just first in details others in the link at the bottom: Calling C from Go (or vice versa) often requires switching from Go's lightweight goroutine model to a full OS thread model because: - Go's scheduler manages goroutines on M:N threads, but C doesn't cooperate with Go's scheduler. - If C code blocks (e.g., on I/O or mutex), Go must assume the worst and parks the thread, spawning another to keep Go alive. * Cost: This means entering/exiting cgo is significantly more expensive than a normal Go call. There’s a syscall-like overhead. ... This was only the first issue, but then it follows with "Go runtime can't see inside C to know is it allocating, blocking, spinning, etc.", then "Stack switching", "Thread Affinity and TLS", "Debug/Profiling support overhead", "Memory Ownership and GC barriers" All here - https://chatgpt.com/share/688172c3-9fa4-800a-9b8f-e1252b57d08d https://chatgpt.com/share/688172c3-9fa4-800a-9b8f-e1252b57d0...
- johnisgood 1y ago
  Just to roll with your way: https://chatgpt.com/share/688177c9-ebc0-8011-88cc-9514d8e1674b https://chatgpt.com/share/688177c9-ebc0-8011-88cc-9514d8e167... Please do not take the numbers below at face value. I still expect an actual reply to my initial comment. Per-call overhead: C (baseline) - ~30 ns Rust (unsafe) - ~30 ns C# (P/Invoke) - ~30-50 ns LuaJIT - ~30-50 ns Go (cgo) - ~40-60 ns Java (22, FFM) - ~40-70 ns Java (JNI) - ~300-1000 ns Perl (XS) - ~500-1000 ns Python (ctypes) - ~10,000-30,000 ns Common Lisp (SBCL) - ~500-1500 ns Seems like Go is still fast enough as opposed to other programming languages with GC, so I am not sure it is fair to Go.
  - Cyph0n 1y ago
    > Rust (unsafe) As if there is an alternative :) More seriously, it’s “unsafe” from the perspective of the library calling into C, but usually “safe” for any layer above.
    - johnisgood 1y ago
      Hey, since I am in a thread where we are sharing what ChatGPT spits out, I just copy pasted it from there, too. :) For what it is worth, I asked about LuaJIT after I have shared the link, and the numbers are now different for some languages, albeit not by much. Go (cgo) became ~50-100 ns. That said, I still believe it is unfair to single out Go when it does way better than some other GC languages.
  - throwaway7783 1y ago
    Java now has FFM, that is far better and simpler than JNI, FWIW. and chatgpt says Language/API | Call Overhead (no-op C) | Notes Go (cgo) | ~40–60 ns | Stack switch + thread pinning Java FFM | ~50 ns (downcall) | Similar to JNI, can be ~30 ns with isTrivial() Java FFM (leaf) | ~30–40 ns | Optimized (isTrivial=true) JNI | ~50–60 ns | Slightly slower than FFM Rust (unsafe) | ~5–20 ns | Near-zero overhead C# (P/Invoke) | ~20–50 ns | Depends on marshaling Python (cffi) | 1000–10000 ns | Orders of magnitude slower |
    - johnisgood 1y ago
      Thanks, I added it to the list. Keep in mind that the numbers may be off (both yours and mine), so I would not take them at face value. It is interesting how in yours JNI is still pretty good. Also Rust is "~5–20 ns" in yours, so I assume "0" is the baseline.
      - throwaway7783 1y ago
        This is chatgpt. Not my own benchmark. So it is probably hallucinating
pornel 1y ago
Go's goroutines aren't plain C threads (blocking syscalls are magically made async), and Go's stack isn't a normal C stack (it's tiny and grown dynamically). A C function won't know how to behave in Go's runtime environment, so to call a C function Go needs make itself look more like a C program, call the C function, and then restore its magic state. Other languages like C++, Rust, and Swift are similar enough to C that they can just call C functions directly. CPython is a C program, so it can too. Golang was brave enough to do fundamental things its own way, which isn't quite C-compatible.
- johnisgood 1y ago
  What about languages like Java, or other popular languages with GC?
  - andrewflnr 1y ago
    Similar enough to C I guess, at least in their stack layout.
  - lmm 1y ago
    Java FFI is slow and cumbersome, even more so if you're using the fancy auto-async from recent versions. The JVM community has mostly bitten the bullet and rewritten the entire world in Java rather than using native libraries, you only see JNI calls for niche things like high performance linear algebra; IMO that was the right tradeoff but it's also often seen as e.g. the reason why Java GUIs on the desktop suck. Other languages generally fall into either camp of having a C-like stack and thread model and easy FFI (e.g. Ruby, TCL, OCaml) and maybe having futures/async but not in an invisible/magic way, or having a radically different threading model at the cost of FFI being slow and painful (e.g. Erlang). JavaScript is kind of special in having C-like stack but being built around calling async functions from a global event loop, so it's technically the first but feels more like the second.
    - hinkley 1y ago
      JNI is the second or maybe third FFI for Java. JRI existed before it and that was worse, including performance. The debugging and instrumentation interfaces have been rewritten more times. https://docs.oracle.com/en/java/javase/24/docs/specs/jni/intro.html https://docs.oracle.com/en/java/javase/24/docs/specs/jni/int... mentions JRI. But it seems like JNI has been replaced by third party solutions multiple times as well. https://developer.okta.com/blog/2022/04/08/state-of-ffi-java#a-brief-history-of-ffi-in-java https://developer.okta.com/blog/2022/04/08/state-of-ffi-java...
3836293648 1y ago
Go's threading model involves a lot of tiny (but growable) stacks and calling C functions almost immediately stack overflows. Calling C safely is then slow because you have to allocate a larger stack, copy data around and mess with the GC.
deleted 1y ago
[deleted]
9rx 1y ago
> How much slower is it in comparison to other languages? It's about the same as most other languages that aren't specifically optimized for C calling. Considerably faster than Python. Which is funny as everyone on HN loves to extol the virtues of Python being a "C DSL" and never think twice about its overhead, but as soon as the word Go is mentioned its like your computer is going to catch fire if you even try. Emotion-driven development is a bizarre world.
- johnisgood 1y ago
  Yeah, that is why I am asking.

unbrice 1y ago

3. The use case is dynamic schemas and access is through the reflection API. Thus PGO has to be done at runtime...

hinkley 1y ago

I know that Java resisted improving their FFI for years because they preferred that the JIT get the extra resources. And that customers not bail out of Java every time they couldn’t figure out how to make it faster. There’s a case I recall from when HotSpot was still young, where the Java GUI team moved part of the graphics pipeline to the FFI in one release, hotspot got faster in the next, and then they rolled back the changes because it was now faster without the FFI. But eventually your compiler is good enough that the FFI Is now your bottleneck, and you need to do something.