6 ms·

there are a few (growing) number of production installations of NSQ... our (bitly's) cluster spans a a few datacenters and hits peaks of 80k messages/second.

there are a few (growing) number of production installations of NSQ... our (bitly's) cluster spans a a few datacenters and hits peaks of 80k messages/second. I can answer any questions you have (one of the authors)

daemon13 13y ago
Any advice on dealing with de-duping? Also, how was performance using JSON data format compared with ProtoBuffers & MsgPack?
- matticakes 13y ago
  NSQ treats the message data as an opaque blob so the format wouldn't directly affect it (except on some lower level related to overall message size I suppose). It would impact your producers (encoding) and consumers (decoding), obviously. re: de-duping - there are lots of things to consider, I highly recommend reading through http://cs.brown.edu/courses/csci2270/archives/2012/papers/weaker/cidr07p15.pdf http://cs.brown.edu/courses/csci2270/archives/2012/papers/we..., it's a fantastic paper. At a high level the answer is idempotency. What sort of use case are you thinking of? (context would help answer your de-dupe question)