At Uken Studios we use NSQ as a chain of queues for log processing. rails -> fluentd -> NSQ (firehose) -> log splitter (python script) -> NSQ (elasticsearch,

At Uken Studios we use NSQ as a chain of queues for log processing. rails -> fluentd -> NSQ (firehose) -> log splitter (python script) -> NSQ (elasticsearch, s3, hadoop) We have this setup live since November and processed more than 25b messages without hiccups.