NSQ treats the message data as an opaque blob so the format wouldn't directly affect it (except on some lower level related to overall message size I suppose).

NSQ treats the message data as an opaque blob so the format wouldn't directly affect it (except on some lower level related to overall message size I suppose). It would impact your producers (encoding) and consumers (decoding), obviously. re: de-duping - there are lots of things to consider, I highly recommend reading through http://cs.brown.edu/courses/csci2270/archives/2012/papers/weaker/cidr07p15.pdf http://cs.brown.edu/courses/csci2270/archives/2012/papers/we..., it's a fantastic paper. At a high level the answer is idempotency. What sort of use case are you thinking of? (context would help answer your de-dupe question)