9 ms·

This is incredible news! I’ve used protobuf in Python, Go, Kotlin and Dart and the Python implementation is totally unusable. I don’t know what black magic Goog

This is incredible news! I’ve used protobuf in Python, Go, Kotlin and Dart and the Python implementation is totally unusable. I don’t know what black magic Google uses for the Python implementation, but the classes it generates are totally opaque and impossible to inspect. I’ve been waiting for a proper python implementation for years now!

masklinn 2mo ago
> I don’t know what black magic Google uses for the Python implementation, but the classes it generates are totally opaque and impossible to inspect. TFA seems to say that they’re just thin proxies over the underlying C++ APIs, which would more than do it, and does not surprise me (the re2 Python bindings are similar, not as bad since they don’t generate Python code but they’re really c++-y — in Google’s flavour too — and uncomfortable).
- functional_dev 2mo ago
  I did not know this before... Google protobuf is not one thing, it is three! Same import, but: * old C++ extension * upb * pure Python upb parses FAST, but then every access is still C->Python and it slows it down. So for many reads the slow python one can win? This one helped me to dig deeper - https://vectree.io/c/how-python-protobuf-runtimes-work-pure-python-vs-upb-vs-the-c-backend https://vectree.io/c/how-python-protobuf-runtimes-work-pure-...
  - masklinn 2mo ago
    > upb parses FAST, but then every access is still C->Python and it slows it down. So for many reads the slow python one can win? That is pretty unlikely. TFA's version is in Rust and just barely edges out upb.
    - Chu4eeno 2mo ago
      rust code tends to be incredibly slow unless you spend a ton of time optimizing (compared to other compiled languages, and contrary to popular belief).
kccqzy 2mo ago
When I was a novice in protobuf I felt tempted to inspect the generated code, but soon I learned that the documentation is good enough that I don’t need to read the generated code.

masklinn 2mo ago

> I don’t know what black magic Google uses for the Python implementation, but the classes it generates are totally opaque and impossible to inspect. TFA seems to say that they’re just thin proxies over the underlying C++ APIs, which would more than do it, and does not surprise me (the re2 Python bindings are similar, not as bad since they don’t generate Python code but they’re really c++-y — in Google’s flavour too — and uncomfortable).

functional_dev 2mo ago
I did not know this before... Google protobuf is not one thing, it is three! Same import, but: * old C++ extension * upb * pure Python upb parses FAST, but then every access is still C->Python and it slows it down. So for many reads the slow python one can win? This one helped me to dig deeper - https://vectree.io/c/how-python-protobuf-runtimes-work-pure-python-vs-upb-vs-the-c-backend https://vectree.io/c/how-python-protobuf-runtimes-work-pure-...
- masklinn 2mo ago
  > upb parses FAST, but then every access is still C->Python and it slows it down. So for many reads the slow python one can win? That is pretty unlikely. TFA's version is in Rust and just barely edges out upb.
  - Chu4eeno 2mo ago
    rust code tends to be incredibly slow unless you spend a ton of time optimizing (compared to other compiled languages, and contrary to popular belief).

kccqzy 2mo ago

When I was a novice in protobuf I felt tempted to inspect the generated code, but soon I learned that the documentation is good enough that I don’t need to read the generated code.