10 ms·

spark, storm, flink, beam, hazelcast... and then there are all the vendor locked choices confluent, kinesis, azure probably has something in that space to The

spark, storm, flink, beam, hazelcast... and then there are all the vendor locked choices confluent, kinesis, azure probably has something in that space to The whole cloud computing space got me confused. I don't know what horse to bet on and don't have the time to get familiar with every new framework. Is this the new javascript world? If so I'd like to skip the next couple of years until we found our react equivalent. edit: Not to be read as an invitation to discuss how react is not the de-facto standard of ui web frameworks

cangencer 6y ago
This is very true. Stream processing is both old and new and I think it takes time for technology like this to really mature. There's currently a standardisation effort around Streaming SQL which may bear some fruit, but probably still many years away. Right even if you want to use some standard language like SQL to describe streaming queries there's differences in each tool both in syntax and semantics.
- chrisjc 6y ago
  Are you referring to this? https://arxiv.org/abs/1905.12133 https://arxiv.org/abs/1905.12133 HN Link: https://news.ycombinator.com/item?id=20059006 https://news.ycombinator.com/item?id=20059006
  - cangencer 6y ago
    The paper is part of the standardization effort but is not the final authority on the process. It is a very good reference on how to approach streaming SQL, even though the Jet model will have a few differences to the presented paper. p.s.: sorry for the late reply (somehow I wrote and didn't publish).
imglorp 6y ago
Distributed Systems (the OReilly trout book) has a nice overview of the streaming landscape (the first four you mentioned). The first several chapters being a general tech background of stream processing: events, watermarks, redundancy etc. http://streamingsystems.net/fig/10-36 http://streamingsystems.net/fig/10-36
- nwsm 6y ago
  I already have Designing Data-Intensive Applications (2017), do you think I would get much more out of that book?
  - imglorp 6y ago
    Yeah I have that one too and yes it's a good general overview of the broad space and yes there is some overlap. If you're thinking of selecting a streaming solution in particular--and it's definitely not for everyone--Streaming Systems is more in depth into the workings and tradeoffs and might be helpful understanding your problem. I'd check the TOC on the link above to decide.

cangencer 6y ago

This is very true. Stream processing is both old and new and I think it takes time for technology like this to really mature. There's currently a standardisation effort around Streaming SQL which may bear some fruit, but probably still many years away. Right even if you want to use some standard language like SQL to describe streaming queries there's differences in each tool both in syntax and semantics.

chrisjc 6y ago
Are you referring to this? https://arxiv.org/abs/1905.12133 https://arxiv.org/abs/1905.12133 HN Link: https://news.ycombinator.com/item?id=20059006 https://news.ycombinator.com/item?id=20059006
- cangencer 6y ago
  The paper is part of the standardization effort but is not the final authority on the process. It is a very good reference on how to approach streaming SQL, even though the Jet model will have a few differences to the presented paper. p.s.: sorry for the late reply (somehow I wrote and didn't publish).

imglorp 6y ago

Distributed Systems (the OReilly trout book) has a nice overview of the streaming landscape (the first four you mentioned). The first several chapters being a general tech background of stream processing: events, watermarks, redundancy etc. http://streamingsystems.net/fig/10-36 http://streamingsystems.net/fig/10-36

nwsm 6y ago
I already have Designing Data-Intensive Applications (2017), do you think I would get much more out of that book?
- imglorp 6y ago
  Yeah I have that one too and yes it's a good general overview of the broad space and yes there is some overlap. If you're thinking of selecting a streaming solution in particular--and it's definitely not for everyone--Streaming Systems is more in depth into the workings and tradeoffs and might be helpful understanding your problem. I'd check the TOC on the link above to decide.