5 ms·
Exciting work! Some questions: 1. Any plans on open-sourcing this? 2. Why not have a tiered architecture that can provide lower latency? p99 of 1s can be too
by bothra90 3y ago
Exciting work! Some questions:
1. Any plans on open-sourcing this?
2. Why not have a tiered architecture that can provide lower latency? p99 of 1s can be too high for some use-cases.
3. Related to 2, how does WarpStream compare to tiered storage in Pulsar?
[Edit 1] Added (3)
- smrtinsert 3y agoYeah really kind of surprised the free tier isn't the self hosted kind. That will keep me looking.
- richieartoul 3y ago(WarpStream Cofounder) FWIW we’re considering a version where you can host the metadata yourself for enterprise users. For the free tier though we didn’t think it made sense since for a workload that could fit into our free tier, it didn’t seem like anyone would want to be responsible for the metadata layer themselves. Would love your feedback on that.
- banditelol 3y agoI'm also quite surprised there's no option for self hosting metadata on the free tier. To be fair, in my experience having managed metadata server for prefect (while they have similar Bring Your Own Server model) is quite hard to get right. But at the end we decided to keep maintaining it because our company prefer having all the data in our servers (including metadata). But I'm still excited to try this, since at least now I could play around (and learn) with a partially kafka compatible system, without the burden of maintaining all of Kafka parts (and costs). Thanks!