8 ms·

To be honest I kind of find myself drifting away from gRPC/protobuf in my recent projects. I love the idea of an IDL for describing APIs and a great compiler/co

To be honest I kind of find myself drifting away from gRPC/protobuf in my recent projects. I love the idea of an IDL for describing APIs and a great compiler/codegen (protoc) but there's just soo many idiosyncrasies baked into gRPC at this point that it often doesn't feel worth it IMO. Been increasingly using LSP style JSON-RPC 2.0, sure it's got it's quirks and is far from the most wire/marshaling efficient approach but JSON codecs are ubiquitous and JSON-RPC is trivial to implement. In-fact I recently even wrote a stack allocated, server implementation for microcontrollers in Rust https://github.com/OpenPSG/embedded-jsonrpc https://github.com/OpenPSG/embedded-jsonrpc. Varlink (https://varlink.org/ https://varlink.org/) is another interesting approach, there's reasons why they didn't implement the full JSON-RPC spec but their IDL is pretty interesting.

malkia 2y ago
Apart from being text format, I'm not sure how well JSON-RPC handles doubles vs long integers and other types, where protobuf can be directed to handle them appropriately. That is a problem in JSON itself, so you may neeed to encode some numbers using... "string"
- dpeckett 2y ago
  I'd say the success of REST kind of proves that's something that for the most part can be worked around. Often comes down to the JSON codec itself, many codecs will allow unmarshalling/marshalling fields straight into long int types. Also JS now has BigInt types and the JSON decoder can be told to use them. So I'd argue it's kind of a moot point at this stage.
  - tikhonj 2y ago
    Sure, but you can work around gRPC's issues too—"workable" might be the only bar that matters in practice, but it's a remarkably low bar. The risk with JSON is that too many systems understand it, and intermediate steps can mess up things like numeric precision as well as being inconsistent about handling things out of spec (field order, duplicate fields... etc). This definitely bites people in practice—I saw an experience report on that recently, but can't find the link just now :/
  - paulddraper 2y ago
    > Also JS now has BigInt types and the JSON decoder can be told to use them. The parser needs to know when to parse as BigInt vs String.
  - malkia 2y ago
    It's not, because some middleman (library, framework, etc.) would assume that JSON is really about sending integers as doubles, hence you are getting only 53 or was it 54 bits precision, and then you end up sending an integer as "string" - but then what is this really? I get it, it's probably not a concern for a lot of applications, but when comes to science, games, data it's of big concern... and this excluding the fact that you have to convert back and forth that number a... number of times, and send it on the wire inefficiently - and also miss a way to send it more efficiently using gorilla encoding or something else like that. JSON is great for a lot of things, but not for high throughput RPC.
  - eadmund 2y ago
    > I'd say the success of REST I think that you mean the success of JSON APIs. REST is orthogonal to JSON/Protobuf/HTML/XML/S-expressions/whatever.
  - dagss 2y ago
    JS having BigInt type has nothing to do with JSON. Backend languages have had BigInt types forever. It isn't relevant to JSON as a format. Just one example: Azure Cosmos DB stores JSON documents. If you try to store integers larger than 53 bits there those integers will be silently rounded to 53 bits. I know someone who got burned by this very badly loosing their data; I was able to explain them exactly why... JSON basically does not define what a number is; and that is a disaster for it as an API format.
girvo 2y ago
Same; at my previous job for the serialisation format for our embedded devices over 2G/4G/LoRaWAN/satellite I ended up landing on MessagePack, but that was partially because the "schema"/typed deserialisation was all in the same language for both the firmware and the server (Nim, in this case) and directly shared source-to-source. That won't work for a lot of cases of course, but it was quite nice for ours!
elcritch 2y ago
My favorite serde format is Msgpack since it can be dropped in for an almost one-to-one replacement of JSON. There's also CBOR which is based on MsgPack but has diverged a bit and added and a data definition language too (CDDL). Take JSON-RPC and replace JSON with MsgPack for better handling of integer and float types. MsgPack/CBOR are easy to parse in place directly into stack objects too. It's super fast even on embedded. I've been shipping it for years in embedded projects using a Nim implementation for ESP32s (1) and later made a non-allocating version (2). It's also generally easy to convert MsgPack/CBOR to JSON for debugging, etc. There's also an IoT focused RPC based on CBOR that's an IETF standard and a time series format (3). The RPC is used a fair bit in some projects. 1: https://github.com/elcritch/nesper/blob/devel/src/nesper/servers/rpc/rpcsocket_mpack.nim https://github.com/elcritch/nesper/blob/devel/src/nesper/ser... 2: https://github.com/EmbeddedNim/fastrpc https://github.com/EmbeddedNim/fastrpc 3: https://hal.science/hal-03800577v1/file/Towards_a_Standard_Time_Series_Representation_for_IoT_using_CBOR_Templates____Sebastian_Molina-18.pdf https://hal.science/hal-03800577v1/file/Towards_a_Standard_T...
crabmusket 2y ago
> I love the idea of an IDL for describing APIs and a great compiler/codegen (protoc) Me too. My context is that I end up using RPC-ish patterns when doing slightly out-of-the-ordinary web stuff, like websockets, iframe communications, and web workers. In each of those situations you start with a bidirectional communication channel, but you have to build your own request-response layer if you need that. JSON-RPC is a good place to start, because the spec is basically just "agree to use `id` to match up requests and responses" and very little else of note. I've been looking around for a "minimum viable IDL" to add to that, and I think my conclusion so far is "just write out a TypeScript file". This works when all my software is web/TypeScript anyway.
- dpeckett 2y ago
  Now that's an interesting thought, I wonder if you could use a modified subset of TypeScript to create a IDL/DDL for JSON-RPC. Then compile that schema into implementations for various target languages.
  - IggleSniggle 2y ago
    Typia kinda does this, but currently only has a Typescript -> Typescript compiler.
  - crabmusket 2y ago
    Yeah that's what I'd look into. Maybe TS -> Json Schema -> target language.
ajross 2y ago
That's sort of where I've landed too. Protobufs would seem to fit the problem area well, but in practice the space between "big-system non-performance-sensitive data transfer metaformat"[1] and "super-performance-sensitive custom binary parser"[2] is... actually really small. There are just very few spots that actually "need" protobuf at a level of urgency that would justify walking away from self-describing text formats (which is a big, big disadvantage for binary formats!). [1] Something very well served by JSON [2] Network routing, stateful packet inspection, on-the-fly transcoding. Stuff that you'd never think to use a "standard format" for.
- bboygravity 2y ago
  Add "everything that communicates with a microcontroller" to 2. That means potentially: the majority of devices in the world.
  - thadt 2y ago
    Perhaps surprisingly, I think microcontrollers may be a place where Protobufs are not a bad fit. Using something like Nanopb [1] gives you the size/speed/flexibility advantages of protocol buffers without being too heavyweight. It’ll be a bit slower than your custom binary protocol, but it comes with quite a few advantages, depending on the context. [1] https://github.com/nanopb/nanopb https://github.com/nanopb/nanopb
    - ajross 2y ago
      We carry a nanopb integration in Zephyr. And even there... meh. It's true that there are some really bad binary protocols in the embedded world. And protobufs are for sure a step up from a median command parser or whatever. And they have real size advantages vs. JSON for tiny/sub-megabyte devices, which is real. But even there, I find that really these are very big machines in a historical sense. And text parsing is really not that hard, or that big. The first HTTP server was on a 25MHz 68040! Just use JSON. Anything else in the modern world needs to be presumed to be premature optimization absent a solid analysis with numbers.
perezd 2y ago
The better stack rn is buf + Connect RPC: https://connectrpc.com/ https://connectrpc.com/ All the compatibility, you get JSON+HTTP & gRPC, one platform.
- rochacon 2y ago
  ConnectRPC is very cool, thanks for sharing. I would like to add 2 other alternatives that I like: - dRPC (by Storj): https://drpc.io https://drpc.io (also compatible with gRPC) - Twirp (by Twitch): https://github.com/twitchtv/twirp https://github.com/twitchtv/twirp (no gRPC compatibility)
- bbkane 2y ago
  Buf seems really nice, but I'm not completely sure what's free and what's not with the Buf platform, so I'm hesitant to make it a dependency for my little open source side project ideas. I should read the docs a bit more.
  - bheadmaster 2y ago
    Buf CLI itself is licensed under a permissive Apache 2.0 License [0]. Since Buf is a compiler, its output cannot be copyrighted (similar to proprietary or GPL licensed compilers). DISCLAIMER: I am not a lawyer. Buf distinguishes a few types of plugins: the most important being local and remote. Local plugins are executables installed on your own machine, and Buf places no restrictions on use of those. Remote plugins are hosted on BSR (Buf Schema Registry) servers [1], which are rate limited. All remote plugins are also available as local plugins if you install them. It's worth to mention that the only time I've personally hit the rate limits of remote plugins is when I misconfigured makefile dependencies to run buf on every change of my code, instead of every change of proto definitions. So, for most development purposes, even remote plugins should be fine. Additionally, BSR also offers hosting of user proto schemas and plugins, and this is where pricing comes in [2]. [0] https://github.com/bufbuild/buf/blob/main/LICENSE https://github.com/bufbuild/buf/blob/main/LICENSE [1] https://buf.build/blog/remote-plugin-execution https://buf.build/blog/remote-plugin-execution [2] https://buf.build/pricing https://buf.build/pricing
hansvm 2y ago
> efficiency State of the art for both gzipped json and protobufs is a few GB/s. Details matter (big strings, arrays, and binary data will push protos to 2x-10x faster in typical cases), but it's not the kind of landslide victory you'd get from a proper binary protocol. There isn't much need to feel like you're missing out.
- lowbloodsugar 2y ago
  Except gzip is tragically slow, so crippling protobuf by running it through gzip could indeed slow it down to json speeds.
  - hansvm 2y ago
    "gzipped json" vs "protobuf"
    - lowbloodsugar 2y ago
      Then something is very wrong.
      - hansvm 2y ago
        Protobufs have a massive data dependency baked into the wire format, turning parsing into an intensive core-bound problem. Interestingly, they're not usually smaller than gzipped JSON either (the compression it has built-in is pretty rudimentary), so if you don't compress it and don't have a stellar network you might actually pay more for the total transfer+decode than gzipped JSON, despite usually being somewhat faster to parse.
        lowbloodsugar 2y ago
        Got any references to share?
        hansvm 2y ago
bccdee 2y ago
What I really like about protobuf is the DDL. Really clear schema evolution rules. Ironclad types. Protobuf moves its complexity into things like default zero values, which are irritating but readily apparent. With json, it's superficially fine, but later on you discover that you need to be worrying about implementation-specific stuff like big ints getting mangled, or special parsing logic you need to set default values for string enums so that adding new values doesn't break backwards compatibility. Json-schema exists but really isn't built for these sorts of constraints, and if you try to use json-schema like protobuf, it can get pretty hairy. Honestly, if protobuf just serialized to a strictly-specified subset of json, I'd be happy with that. I'm not in it for the fast ser/de, and something human-readable could be good. But when multiple services maintained by different teams are passing messages around, a robust schema language is a MASSIVE help. I haven't used Avro, but I assume it's similarly useful.
mirekrusin 2y ago
Also json parsers are crazy fast nowadays, most people don't realize how fast they are.
- Cthulhu_ 2y ago
  While true, it's still a text and usually http/tcp based format; data -> json representation -> compression? -> http -> tcp -> decompression -> parsing -> data. Translating to / from a text just feels inefficient.
  - mirekrusin 2y ago
    With projects I work on it's over websockets, js/ts has builtin support, easy to log, debug, extend/work with etc. Binary protocols have exactly same steps. Redis is also using text based protocol and people don't seem to be bothered too much about it.

malkia 2y ago

Apart from being text format, I'm not sure how well JSON-RPC handles doubles vs long integers and other types, where protobuf can be directed to handle them appropriately. That is a problem in JSON itself, so you may neeed to encode some numbers using... "string"

dpeckett 2y ago
I'd say the success of REST kind of proves that's something that for the most part can be worked around. Often comes down to the JSON codec itself, many codecs will allow unmarshalling/marshalling fields straight into long int types. Also JS now has BigInt types and the JSON decoder can be told to use them. So I'd argue it's kind of a moot point at this stage.
- tikhonj 2y ago
  Sure, but you can work around gRPC's issues too—"workable" might be the only bar that matters in practice, but it's a remarkably low bar. The risk with JSON is that too many systems understand it, and intermediate steps can mess up things like numeric precision as well as being inconsistent about handling things out of spec (field order, duplicate fields... etc). This definitely bites people in practice—I saw an experience report on that recently, but can't find the link just now :/
- paulddraper 2y ago
  > Also JS now has BigInt types and the JSON decoder can be told to use them. The parser needs to know when to parse as BigInt vs String.
- malkia 2y ago
  It's not, because some middleman (library, framework, etc.) would assume that JSON is really about sending integers as doubles, hence you are getting only 53 or was it 54 bits precision, and then you end up sending an integer as "string" - but then what is this really? I get it, it's probably not a concern for a lot of applications, but when comes to science, games, data it's of big concern... and this excluding the fact that you have to convert back and forth that number a... number of times, and send it on the wire inefficiently - and also miss a way to send it more efficiently using gorilla encoding or something else like that. JSON is great for a lot of things, but not for high throughput RPC.
- eadmund 2y ago
  > I'd say the success of REST I think that you mean the success of JSON APIs. REST is orthogonal to JSON/Protobuf/HTML/XML/S-expressions/whatever.
- dagss 2y ago
  JS having BigInt type has nothing to do with JSON. Backend languages have had BigInt types forever. It isn't relevant to JSON as a format. Just one example: Azure Cosmos DB stores JSON documents. If you try to store integers larger than 53 bits there those integers will be silently rounded to 53 bits. I know someone who got burned by this very badly loosing their data; I was able to explain them exactly why... JSON basically does not define what a number is; and that is a disaster for it as an API format.

girvo 2y ago

Same; at my previous job for the serialisation format for our embedded devices over 2G/4G/LoRaWAN/satellite I ended up landing on MessagePack, but that was partially because the "schema"/typed deserialisation was all in the same language for both the firmware and the server (Nim, in this case) and directly shared source-to-source. That won't work for a lot of cases of course, but it was quite nice for ours!

elcritch 2y ago

My favorite serde format is Msgpack since it can be dropped in for an almost one-to-one replacement of JSON. There's also CBOR which is based on MsgPack but has diverged a bit and added and a data definition language too (CDDL). Take JSON-RPC and replace JSON with MsgPack for better handling of integer and float types. MsgPack/CBOR are easy to parse in place directly into stack objects too. It's super fast even on embedded. I've been shipping it for years in embedded projects using a Nim implementation for ESP32s (1) and later made a non-allocating version (2). It's also generally easy to convert MsgPack/CBOR to JSON for debugging, etc. There's also an IoT focused RPC based on CBOR that's an IETF standard and a time series format (3). The RPC is used a fair bit in some projects. 1: https://github.com/elcritch/nesper/blob/devel/src/nesper/servers/rpc/rpcsocket_mpack.nim https://github.com/elcritch/nesper/blob/devel/src/nesper/ser... 2: https://github.com/EmbeddedNim/fastrpc https://github.com/EmbeddedNim/fastrpc 3: https://hal.science/hal-03800577v1/file/Towards_a_Standard_Time_Series_Representation_for_IoT_using_CBOR_Templates____Sebastian_Molina-18.pdf https://hal.science/hal-03800577v1/file/Towards_a_Standard_T...

crabmusket 2y ago

> I love the idea of an IDL for describing APIs and a great compiler/codegen (protoc) Me too. My context is that I end up using RPC-ish patterns when doing slightly out-of-the-ordinary web stuff, like websockets, iframe communications, and web workers. In each of those situations you start with a bidirectional communication channel, but you have to build your own request-response layer if you need that. JSON-RPC is a good place to start, because the spec is basically just "agree to use `id` to match up requests and responses" and very little else of note. I've been looking around for a "minimum viable IDL" to add to that, and I think my conclusion so far is "just write out a TypeScript file". This works when all my software is web/TypeScript anyway.

dpeckett 2y ago
Now that's an interesting thought, I wonder if you could use a modified subset of TypeScript to create a IDL/DDL for JSON-RPC. Then compile that schema into implementations for various target languages.
- IggleSniggle 2y ago
  Typia kinda does this, but currently only has a Typescript -> Typescript compiler.
- crabmusket 2y ago
  Yeah that's what I'd look into. Maybe TS -> Json Schema -> target language.

ajross 2y ago

That's sort of where I've landed too. Protobufs would seem to fit the problem area well, but in practice the space between "big-system non-performance-sensitive data transfer metaformat"[1] and "super-performance-sensitive custom binary parser"[2] is... actually really small. There are just very few spots that actually "need" protobuf at a level of urgency that would justify walking away from self-describing text formats (which is a big, big disadvantage for binary formats!). [1] Something very well served by JSON [2] Network routing, stateful packet inspection, on-the-fly transcoding. Stuff that you'd never think to use a "standard format" for.

bboygravity 2y ago
Add "everything that communicates with a microcontroller" to 2. That means potentially: the majority of devices in the world.
- thadt 2y ago
  Perhaps surprisingly, I think microcontrollers may be a place where Protobufs are not a bad fit. Using something like Nanopb [1] gives you the size/speed/flexibility advantages of protocol buffers without being too heavyweight. It’ll be a bit slower than your custom binary protocol, but it comes with quite a few advantages, depending on the context. [1] https://github.com/nanopb/nanopb https://github.com/nanopb/nanopb
  - ajross 2y ago
    We carry a nanopb integration in Zephyr. And even there... meh. It's true that there are some really bad binary protocols in the embedded world. And protobufs are for sure a step up from a median command parser or whatever. And they have real size advantages vs. JSON for tiny/sub-megabyte devices, which is real. But even there, I find that really these are very big machines in a historical sense. And text parsing is really not that hard, or that big. The first HTTP server was on a 25MHz 68040! Just use JSON. Anything else in the modern world needs to be presumed to be premature optimization absent a solid analysis with numbers.

perezd 2y ago

The better stack rn is buf + Connect RPC: https://connectrpc.com/ https://connectrpc.com/ All the compatibility, you get JSON+HTTP & gRPC, one platform.

rochacon 2y ago
ConnectRPC is very cool, thanks for sharing. I would like to add 2 other alternatives that I like: - dRPC (by Storj): https://drpc.io https://drpc.io (also compatible with gRPC) - Twirp (by Twitch): https://github.com/twitchtv/twirp https://github.com/twitchtv/twirp (no gRPC compatibility)
bbkane 2y ago
Buf seems really nice, but I'm not completely sure what's free and what's not with the Buf platform, so I'm hesitant to make it a dependency for my little open source side project ideas. I should read the docs a bit more.
- bheadmaster 2y ago
  Buf CLI itself is licensed under a permissive Apache 2.0 License [0]. Since Buf is a compiler, its output cannot be copyrighted (similar to proprietary or GPL licensed compilers). DISCLAIMER: I am not a lawyer. Buf distinguishes a few types of plugins: the most important being local and remote. Local plugins are executables installed on your own machine, and Buf places no restrictions on use of those. Remote plugins are hosted on BSR (Buf Schema Registry) servers [1], which are rate limited. All remote plugins are also available as local plugins if you install them. It's worth to mention that the only time I've personally hit the rate limits of remote plugins is when I misconfigured makefile dependencies to run buf on every change of my code, instead of every change of proto definitions. So, for most development purposes, even remote plugins should be fine. Additionally, BSR also offers hosting of user proto schemas and plugins, and this is where pricing comes in [2]. [0] https://github.com/bufbuild/buf/blob/main/LICENSE https://github.com/bufbuild/buf/blob/main/LICENSE [1] https://buf.build/blog/remote-plugin-execution https://buf.build/blog/remote-plugin-execution [2] https://buf.build/pricing https://buf.build/pricing

hansvm 2y ago

> efficiency State of the art for both gzipped json and protobufs is a few GB/s. Details matter (big strings, arrays, and binary data will push protos to 2x-10x faster in typical cases), but it's not the kind of landslide victory you'd get from a proper binary protocol. There isn't much need to feel like you're missing out.

lowbloodsugar 2y ago
Except gzip is tragically slow, so crippling protobuf by running it through gzip could indeed slow it down to json speeds.
- hansvm 2y ago
  "gzipped json" vs "protobuf"
  - lowbloodsugar 2y ago
    Then something is very wrong.
    - hansvm 2y ago
      Protobufs have a massive data dependency baked into the wire format, turning parsing into an intensive core-bound problem. Interestingly, they're not usually smaller than gzipped JSON either (the compression it has built-in is pretty rudimentary), so if you don't compress it and don't have a stellar network you might actually pay more for the total transfer+decode than gzipped JSON, despite usually being somewhat faster to parse.
      - lowbloodsugar 2y ago
        Got any references to share?
        hansvm 2y ago

bccdee 2y ago

What I really like about protobuf is the DDL. Really clear schema evolution rules. Ironclad types. Protobuf moves its complexity into things like default zero values, which are irritating but readily apparent. With json, it's superficially fine, but later on you discover that you need to be worrying about implementation-specific stuff like big ints getting mangled, or special parsing logic you need to set default values for string enums so that adding new values doesn't break backwards compatibility. Json-schema exists but really isn't built for these sorts of constraints, and if you try to use json-schema like protobuf, it can get pretty hairy. Honestly, if protobuf just serialized to a strictly-specified subset of json, I'd be happy with that. I'm not in it for the fast ser/de, and something human-readable could be good. But when multiple services maintained by different teams are passing messages around, a robust schema language is a MASSIVE help. I haven't used Avro, but I assume it's similarly useful.

mirekrusin 2y ago

Also json parsers are crazy fast nowadays, most people don't realize how fast they are.

Cthulhu_ 2y ago
While true, it's still a text and usually http/tcp based format; data -> json representation -> compression? -> http -> tcp -> decompression -> parsing -> data. Translating to / from a text just feels inefficient.
- mirekrusin 2y ago
  With projects I work on it's over websockets, js/ts has builtin support, easy to log, debug, extend/work with etc. Binary protocols have exactly same steps. Redis is also using text based protocol and people don't seem to be bothered too much about it.