5 ms·

Nice, have you come across NATS? https://nats.io https://nats.io. The server natively supports WebSockets. There are many clients including Deno, Node, WebSocke

by bruth 4y ago

Nice, have you come across NATS? https://nats.io https://nats.io. The server natively supports WebSockets. There are many clients including Deno, Node, WebSockets, Rust, Go, C, Python, etc. In addition to stateless messaging, it supports durable streams, and optimized API layers on top like key-value, and object storage. The server also natively supports MQTT 3.1.1.

SpaghettiX 4y ago
Nats is not something I see as a competitor for external clients (browsers, mobile apps), primarily because it doesn't handle reconnections / message delivery / quality-of-service / at-least-once or exactly-once delivery (except for MQTT). > When the connection is lost, your application would have to re-create it and all subscriptions if any. https://github.com/nats-io/stan.go#connection-status https://github.com/nats-io/stan.go#connection-status Therefore, I don't see what it adds here. It seems designed for service communication, not client-server. They also don't list browsers as a use case https://docs.nats.io/nats-concepts/overview#use-cases https://docs.nats.io/nats-concepts/overview#use-cases. (though it is of course possible, it's just not ideal IMHO.) They still have a js/browser client library though if you want to use them: https://github.com/nats-io/nats.ws https://github.com/nats-io/nats.ws. And yes, their servers "have websocket support".
- SpaghettiX 4y ago
  They also don't list themselves as competitors to products that do target this market: ably, firebase messaging, pubnub, pusher. Instead, they compare themselves to Kafka, RabbitMQ, Pulsar, and gRPC, none of which work well on browsers. (Yes grpc-web "works" on the browser, but I suggest everyone avoids it.) https://docs.nats.io/nats-concepts/overview/compare-nats https://docs.nats.io/nats-concepts/overview/compare-nats
- bruth 4y ago
  In fact it does all of these things now properly! STAN (NATS Streaming) was deprecated two years ago in favor of a new embedded subsystem called JetStream: https://docs.nats.io/nats-concepts/jetstream https://docs.nats.io/nats-concepts/jetstream released in March 2021.
  - SpaghettiX 4y ago
    Even with NATS jetstream, NATS has a focus on service communication. "It supports websockets" and "qos" does not mean it will work robustly with web apps if nobody uses NATS for that use case. See https://github.com/nats-io/nats.ws/issues/172 https://github.com/nats-io/nats.ws/issues/172 for an example issue. If NATS is not used for websockets in browsers, it will have a mine field of issues to fix. And what about all the other clients (mobile, mobile web)? Sure there may be a NATS client library for it, but it won't handle user connectivity issues, because again it's aimed at service communication where the network is great. People are using NATS in kubernetes, not web browsers.
    - bruth 4y ago
      > Even with NATS jetstream, NATS has a focus on service communication. It indeed excels at service communication as well. However, a core use case for NATS is the edge, be it your definition (browsers and mobile), but also in cars, factories, tractors, low-orbit satellites, etc, whether it is running on Kubernetes, k3s, or bare metal. The issue you called out is a Firefox-specific issue, but it will be addressed and not indicative of an inherit limitation of NATS. Check out this playlist of a live event I organized last fall with a variety of live demos: https://youtube.com/playlist?list=PLgqCaaYodvKY6xRbvB6ffON0_ahyhDsoM https://youtube.com/playlist?list=PLgqCaaYodvKY6xRbvB6ffON0_...
      - SpaghettiX 4y ago
        Currently, the only people I see talking about NATS for edge is Synadia - they're also not very specific. In theory/documentation, "edge" is a core NATS use case, but in practice why does NATS compare themselves to Kafka and microservices? Most of that playlist is not edge-focused. Can you explain what concepts and problems you have to solve to support the "edge" - none of that is in your website. > The issue you called out is a Firefox-specific issue, but it will be addressed and not indicative of an inherit limitation of NATS. My point is NATS is not being used in browsers, mobile apps or edge use cases. It doesn't even explore the concepts. It looks like it doesn't care about Firefox. For IoT, what does NATS bring on top of MQTT? NATS ends up being an MQTT broker so it will have to compete with all of them. Why don't you start comparing yourself to products and technology that serve the edge (other edge-focused companies (ably, pusher, pubnub), and other MQTT brokers)? Side note: Would appreciate it if you disclosed your affiliation with Synadia and NATS before advertising it.
paulgb 4y ago
NATS is great, we use it in another project, Plane (https://plane.dev https://plane.dev). The reasons I didn’t use it instead of making DriftDB: - In NATS, unless you set up authentication, any user can subscribe to “>” and get a firehose of every message, even if they don’t know the room IDs. - NATS Jetstream supports rollups, but they roll up the entire stream, rather than up to a certain sequence number. This would break our ability to do leaderless compaction.
- bruth 4y ago
  Was not aware of Plane, nice! Regarding the two points: - A unique room ID/subject is a form of authentication. Essentially anyone having that unique identifier can join, akin to a token. This is straightforward to setup in NATS avoiding the ">" for all problem (which I may now need to write a blog post about ;-) - Rollups are supported on a per-subject basis. Each room could be modeled as a subject and individually rolled up.
  - paulgb 4y ago
    Re. #1, can you elaborate? The tracking issue for this is still open[1]. As far as I can tell this is a hard blocker for any use case where a random user on the web can connect to NATS, since it means that user can wiretap any room without knowing the room ID. Re #2, the problem is that a rollup of a subject in NATS rolls up the whole subject, so there’s a race condition if you try to use it the way DriftDB uses it. If one client is computing a compaction while another client sends a message, that message will be erased by the compaction. This works if a single producer is writing to a stream, because that producer can stop emitting messages during the compaction. But in our case, each client can produce messages at any time. DriftDB solves this by sending a sequence number alongside the rollup of the last message included in the rollup, and the server preserves messages after that sequence number. [1] https://github.com/nats-io/nats-server/issues/2667 https://github.com/nats-io/nats-server/issues/2667
    - bruth 4y ago
      #1: This isn't a hard requirement to achieve the desired permissions. For the DriftDB use case, my understanding is that all members in the room have full pub/sub k/v permissions, so that could be achieved by declaring a new permission pinned to the room when the room is created or joined (this can be done dynamically without a config file reload). #2 Publishes do support optimistic concurrency control using the `Nats-Last-Expected-Sequence` (stream level) or `Nats-Last-Expected-Subject-Sequence` for the subject-level. This ensures to concurrent publishes will be serialized and all but one is rejected with "conflict wrong sequence" error. For example headers in Rust[0] and WS[1] [0]: https://docs.rs/async-nats/latest/async_nats/header/index.html https://docs.rs/async-nats/latest/async_nats/header/index.ht...

by bruth 4y ago

SpaghettiX 4y ago

Nats is not something I see as a competitor for external clients (browsers, mobile apps), primarily because it doesn't handle reconnections / message delivery / quality-of-service / at-least-once or exactly-once delivery (except for MQTT). > When the connection is lost, your application would have to re-create it and all subscriptions if any. https://github.com/nats-io/stan.go#connection-status https://github.com/nats-io/stan.go#connection-status Therefore, I don't see what it adds here. It seems designed for service communication, not client-server. They also don't list browsers as a use case https://docs.nats.io/nats-concepts/overview#use-cases https://docs.nats.io/nats-concepts/overview#use-cases. (though it is of course possible, it's just not ideal IMHO.) They still have a js/browser client library though if you want to use them: https://github.com/nats-io/nats.ws https://github.com/nats-io/nats.ws. And yes, their servers "have websocket support".

SpaghettiX 4y ago
They also don't list themselves as competitors to products that do target this market: ably, firebase messaging, pubnub, pusher. Instead, they compare themselves to Kafka, RabbitMQ, Pulsar, and gRPC, none of which work well on browsers. (Yes grpc-web "works" on the browser, but I suggest everyone avoids it.) https://docs.nats.io/nats-concepts/overview/compare-nats https://docs.nats.io/nats-concepts/overview/compare-nats
bruth 4y ago
In fact it does all of these things now properly! STAN (NATS Streaming) was deprecated two years ago in favor of a new embedded subsystem called JetStream: https://docs.nats.io/nats-concepts/jetstream https://docs.nats.io/nats-concepts/jetstream released in March 2021.
- SpaghettiX 4y ago
  Even with NATS jetstream, NATS has a focus on service communication. "It supports websockets" and "qos" does not mean it will work robustly with web apps if nobody uses NATS for that use case. See https://github.com/nats-io/nats.ws/issues/172 https://github.com/nats-io/nats.ws/issues/172 for an example issue. If NATS is not used for websockets in browsers, it will have a mine field of issues to fix. And what about all the other clients (mobile, mobile web)? Sure there may be a NATS client library for it, but it won't handle user connectivity issues, because again it's aimed at service communication where the network is great. People are using NATS in kubernetes, not web browsers.
  - bruth 4y ago
    > Even with NATS jetstream, NATS has a focus on service communication. It indeed excels at service communication as well. However, a core use case for NATS is the edge, be it your definition (browsers and mobile), but also in cars, factories, tractors, low-orbit satellites, etc, whether it is running on Kubernetes, k3s, or bare metal. The issue you called out is a Firefox-specific issue, but it will be addressed and not indicative of an inherit limitation of NATS. Check out this playlist of a live event I organized last fall with a variety of live demos: https://youtube.com/playlist?list=PLgqCaaYodvKY6xRbvB6ffON0_ahyhDsoM https://youtube.com/playlist?list=PLgqCaaYodvKY6xRbvB6ffON0_...
    - SpaghettiX 4y ago
      Currently, the only people I see talking about NATS for edge is Synadia - they're also not very specific. In theory/documentation, "edge" is a core NATS use case, but in practice why does NATS compare themselves to Kafka and microservices? Most of that playlist is not edge-focused. Can you explain what concepts and problems you have to solve to support the "edge" - none of that is in your website. > The issue you called out is a Firefox-specific issue, but it will be addressed and not indicative of an inherit limitation of NATS. My point is NATS is not being used in browsers, mobile apps or edge use cases. It doesn't even explore the concepts. It looks like it doesn't care about Firefox. For IoT, what does NATS bring on top of MQTT? NATS ends up being an MQTT broker so it will have to compete with all of them. Why don't you start comparing yourself to products and technology that serve the edge (other edge-focused companies (ably, pusher, pubnub), and other MQTT brokers)? Side note: Would appreciate it if you disclosed your affiliation with Synadia and NATS before advertising it.

paulgb 4y ago

NATS is great, we use it in another project, Plane (https://plane.dev https://plane.dev). The reasons I didn’t use it instead of making DriftDB: - In NATS, unless you set up authentication, any user can subscribe to “>” and get a firehose of every message, even if they don’t know the room IDs. - NATS Jetstream supports rollups, but they roll up the entire stream, rather than up to a certain sequence number. This would break our ability to do leaderless compaction.

bruth 4y ago
Was not aware of Plane, nice! Regarding the two points: - A unique room ID/subject is a form of authentication. Essentially anyone having that unique identifier can join, akin to a token. This is straightforward to setup in NATS avoiding the ">" for all problem (which I may now need to write a blog post about ;-) - Rollups are supported on a per-subject basis. Each room could be modeled as a subject and individually rolled up.
- paulgb 4y ago
  Re. #1, can you elaborate? The tracking issue for this is still open[1]. As far as I can tell this is a hard blocker for any use case where a random user on the web can connect to NATS, since it means that user can wiretap any room without knowing the room ID. Re #2, the problem is that a rollup of a subject in NATS rolls up the whole subject, so there’s a race condition if you try to use it the way DriftDB uses it. If one client is computing a compaction while another client sends a message, that message will be erased by the compaction. This works if a single producer is writing to a stream, because that producer can stop emitting messages during the compaction. But in our case, each client can produce messages at any time. DriftDB solves this by sending a sequence number alongside the rollup of the last message included in the rollup, and the server preserves messages after that sequence number. [1] https://github.com/nats-io/nats-server/issues/2667 https://github.com/nats-io/nats-server/issues/2667
  - bruth 4y ago
    #1: This isn't a hard requirement to achieve the desired permissions. For the DriftDB use case, my understanding is that all members in the room have full pub/sub k/v permissions, so that could be achieved by declaring a new permission pinned to the room when the room is created or joined (this can be done dynamically without a config file reload). #2 Publishes do support optimistic concurrency control using the `Nats-Last-Expected-Sequence` (stream level) or `Nats-Last-Expected-Subject-Sequence` for the subject-level. This ensures to concurrent publishes will be serialized and all but one is rejected with "conflict wrong sequence" error. For example headers in Rust[0] and WS[1] [0]: https://docs.rs/async-nats/latest/async_nats/header/index.html https://docs.rs/async-nats/latest/async_nats/header/index.ht...