10 ms·

Protobuf Editions are here: don't panic

31 points by perezd 2y ago

perezd 2y ago
Google is days away from making Protobuf Editions GA. Never heard of it? Read on.
- orf 2y ago
  What’s the meaning behind this comment?
  - perezd 2y ago
    Many folks don't know about what Protobuf Editions are, and it's about to be generally available. I was encouraging people to read the article :)
    - deleted 2y ago
      [deleted]
  - beeboobaa3 2y ago
    looks like an LLM spambot
    - perezd 2y ago
      lol, well. Sorry man, I'm a real boy. Check my account age. I can solve some math problems or look at what photos are bridges if you want.
      - dilyevsky 2y ago
        Meh, that's too easy now - if you really want to prove it post something politically incorrect.
tb303 2y ago
> It’s unclear what an edition's official support lifetime will be today, although Google has suggested it might be “like 10 years.” ambiguous timeline? sounds right for google.
- VHRanger 2y ago
  The timeline depends on when the project will stop being useful to generate promotions. Thanks Sundar for great org incentive structure!
candiddevmike 2y ago
I still don't understand why protobuf has to be in so many Go libraries. It's impossible to avoid unless you roll your own metrics, logging, maybe even mux...
- TillE 2y ago
  It's simple, lightweight, safe serialization. If you need to convert data structures to bytes, you're going to use protobuf or something comparable.
- jsheard 2y ago
  Even if protobuf isn't your preferred serializer, having any tight binary format be the standard across the ecosystem is miles better than everything slinging JSON around. Lord knows how much energy we are wasting turning binary data into human readable text that no human will ever look at before it's parsed back into binary.
  - hot_gril 2y ago
    Performance aside, it's also really nice how a protobuf spec tells you what data structure to expect ahead of time for an API.
    - jon_richards 2y ago
      The number of APIs with a single JSON example as the only documentation is... unfortunate. Especially if you want to know if a field is nullable.
      - mf192 2y ago
        Taking this a bit further, it's surprising how many companies sell API as a service but provide minimal support for that API, publishing like 1 client/sdk (and often not in the language you need, lol). Just mindblowing that companies don't adopt a schema-first approach to appeal to a broader customer base. I've lost count of the number of hnd rolled clients I've had to write over the years. It gets even better if the API itself is backed by some RPC framework. With that, users can simply generate a client and get going. It feels like we can do better as an industry, this cracked me up because it's so true. "a single JSON example as the only documentation is... unfortunate"
hot_gril 2y ago
I've always liked the simplicity of protobuf, and this change seems to add unnecessary complexity. And: "What problems are Editions solving? In short, nothing really (for end users). The introduction of Editions is the result of major Google-internal refactoring[...]"
- hehdhdjehehegwv 2y ago
  Design by promo.
  - riwsky 2y ago
    Promobuf
- hot_gril 2y ago
  Thinking about this more, this looks overly dismissive. Seems Google wants to be very explicit about future syntax and allow users to mix in different features. I don't think it's worthwhile, but it's doing something. Is the author just annoyed that Buf now has to put in the work to support editions?
rco8786 2y ago
As if I couldn’t hate protobufs any more. They go and layer on yet more complexity. These sorts of things need to be as simple as humanly possibly and get out of your way so you can actually do some programming. Instead protobufs are the opposite. Full of backwards compat issues, footguns, and needless complexity that make them an absolute horror to work with.
- drunner 2y ago
  Do you have a preferred message protocol?
  - VHRanger 2y ago
    JSON with a good schema if you're doing it across languages?
    - swatcoder 2y ago
      XML, JSON, and YAML belong to a totally diffent class of use than protobuf, flatbuffers, capnproto, etc It's like comparing apples and screwdrivers.
      - hot_gril 2y ago
        They're comparable for a lot of use cases. In fact many protobuf-based APIs will also have a JSON mode.
        swatcoder 2y ago
        Exactly because the consumers of those API's exhibit a strong preference for one or the other. Library/module/API developers often seek to deliver maximimum adaptibility in their interfaces so that consumers of them can satisfy their own more specific/pressing requirements.
        dilyevsky 2y ago
jujube3 2y ago
Oh look, Google is shipping their org chart again. > What problems are Editions solving? > > In short, nothing really (for end users). The introduction of Editions > is the result of major Google-internal refactoring of how protoc and its > plugin architecture implements and observes feature checks when generating > code. This isn’t intended to force breaking changes to existing projects, > nor is it designed to impact any of the existing encodings. > > It should be a boring change that gives plugin maintainers finer-grained > control over how future versions of their Protobuf runtimes behave, > improvements are made, and new features are introduced. Having said that, > it’s impossible to ignore the explosion of verbosity that Editions has > introduced to the project as a side-effect of this level of available control. According to Google themselves, this change "solves no user problems" and introduces "an explosion of verbosity."
- comex 2y ago
  This blog post wasn’t written by Google themselves, but by Buf, a confusingly-named startup that makes their own protobuf implementation.
- zeroxfe 2y ago
  I don't understand why this is a problem. It's not uncommon to see multi-decade old systems refactored for extensibility and maintainability every once in a while. What worked when it was initially designed won't work forever, and it's hard to make users lives better if the maintainers can't make their own lives better. (BTW, this isn't "shipping their org chart.")
  - aeonik 2y ago
    I thought Conway's law was fairly prolific. Though, the closer it is to a true a "law", then the closer GP's comment approaches a tautology. https://wikipedia.org/wiki/Conway%27s_law https://wikipedia.org/wiki/Conway%27s_law
  - tjpnz 2y ago
    Did it ever work? I remember having to write some pretty gnarly hacks to work around the broken Python code protoc would barf out. Working outside of Go and perhaps Java/Kotlin I've little faith that they're not going to break things further.

31 points by perezd 2y ago

perezd 2y ago

Google is days away from making Protobuf Editions GA. Never heard of it? Read on.

orf 2y ago
What’s the meaning behind this comment?
- perezd 2y ago
  Many folks don't know about what Protobuf Editions are, and it's about to be generally available. I was encouraging people to read the article :)
  - deleted 2y ago
    [deleted]
- beeboobaa3 2y ago
  looks like an LLM spambot
  - perezd 2y ago
    lol, well. Sorry man, I'm a real boy. Check my account age. I can solve some math problems or look at what photos are bridges if you want.
    - dilyevsky 2y ago
      Meh, that's too easy now - if you really want to prove it post something politically incorrect.

tb303 2y ago

> It’s unclear what an edition's official support lifetime will be today, although Google has suggested it might be “like 10 years.” ambiguous timeline? sounds right for google.

VHRanger 2y ago
The timeline depends on when the project will stop being useful to generate promotions. Thanks Sundar for great org incentive structure!

candiddevmike 2y ago

I still don't understand why protobuf has to be in so many Go libraries. It's impossible to avoid unless you roll your own metrics, logging, maybe even mux...

TillE 2y ago
It's simple, lightweight, safe serialization. If you need to convert data structures to bytes, you're going to use protobuf or something comparable.
jsheard 2y ago
Even if protobuf isn't your preferred serializer, having any tight binary format be the standard across the ecosystem is miles better than everything slinging JSON around. Lord knows how much energy we are wasting turning binary data into human readable text that no human will ever look at before it's parsed back into binary.
- hot_gril 2y ago
  Performance aside, it's also really nice how a protobuf spec tells you what data structure to expect ahead of time for an API.
  - jon_richards 2y ago
    The number of APIs with a single JSON example as the only documentation is... unfortunate. Especially if you want to know if a field is nullable.
    - mf192 2y ago
      Taking this a bit further, it's surprising how many companies sell API as a service but provide minimal support for that API, publishing like 1 client/sdk (and often not in the language you need, lol). Just mindblowing that companies don't adopt a schema-first approach to appeal to a broader customer base. I've lost count of the number of hnd rolled clients I've had to write over the years. It gets even better if the API itself is backed by some RPC framework. With that, users can simply generate a client and get going. It feels like we can do better as an industry, this cracked me up because it's so true. "a single JSON example as the only documentation is... unfortunate"

hot_gril 2y ago

I've always liked the simplicity of protobuf, and this change seems to add unnecessary complexity. And: "What problems are Editions solving? In short, nothing really (for end users). The introduction of Editions is the result of major Google-internal refactoring[...]"

hehdhdjehehegwv 2y ago
Design by promo.
- riwsky 2y ago
  Promobuf
hot_gril 2y ago
Thinking about this more, this looks overly dismissive. Seems Google wants to be very explicit about future syntax and allow users to mix in different features. I don't think it's worthwhile, but it's doing something. Is the author just annoyed that Buf now has to put in the work to support editions?

rco8786 2y ago

As if I couldn’t hate protobufs any more. They go and layer on yet more complexity. These sorts of things need to be as simple as humanly possibly and get out of your way so you can actually do some programming. Instead protobufs are the opposite. Full of backwards compat issues, footguns, and needless complexity that make them an absolute horror to work with.

drunner 2y ago
Do you have a preferred message protocol?
- VHRanger 2y ago
  JSON with a good schema if you're doing it across languages?
  - swatcoder 2y ago
    XML, JSON, and YAML belong to a totally diffent class of use than protobuf, flatbuffers, capnproto, etc It's like comparing apples and screwdrivers.
    - hot_gril 2y ago
      They're comparable for a lot of use cases. In fact many protobuf-based APIs will also have a JSON mode.
      - swatcoder 2y ago
        Exactly because the consumers of those API's exhibit a strong preference for one or the other. Library/module/API developers often seek to deliver maximimum adaptibility in their interfaces so that consumers of them can satisfy their own more specific/pressing requirements.
        dilyevsky 2y ago

jujube3 2y ago

Oh look, Google is shipping their org chart again. > What problems are Editions solving? > > In short, nothing really (for end users). The introduction of Editions > is the result of major Google-internal refactoring of how protoc and its > plugin architecture implements and observes feature checks when generating > code. This isn’t intended to force breaking changes to existing projects, > nor is it designed to impact any of the existing encodings. > > It should be a boring change that gives plugin maintainers finer-grained > control over how future versions of their Protobuf runtimes behave, > improvements are made, and new features are introduced. Having said that, > it’s impossible to ignore the explosion of verbosity that Editions has > introduced to the project as a side-effect of this level of available control. According to Google themselves, this change "solves no user problems" and introduces "an explosion of verbosity."

comex 2y ago
This blog post wasn’t written by Google themselves, but by Buf, a confusingly-named startup that makes their own protobuf implementation.
zeroxfe 2y ago
I don't understand why this is a problem. It's not uncommon to see multi-decade old systems refactored for extensibility and maintainability every once in a while. What worked when it was initially designed won't work forever, and it's hard to make users lives better if the maintainers can't make their own lives better. (BTW, this isn't "shipping their org chart.")
- aeonik 2y ago
  I thought Conway's law was fairly prolific. Though, the closer it is to a true a "law", then the closer GP's comment approaches a tautology. https://wikipedia.org/wiki/Conway%27s_law https://wikipedia.org/wiki/Conway%27s_law
- tjpnz 2y ago
  Did it ever work? I remember having to write some pretty gnarly hacks to work around the broken Python code protoc would barf out. Working outside of Go and perhaps Java/Kotlin I've little faith that they're not going to break things further.