9 ms·

DeepstreamHub: a faster Firebase with an open core

118 points by chadfowler 9y ago

Sujan 9y ago
What exactly can be replaced with DeepstreamHub if your are using Firebase right now?
- RomanPushkin 9y ago
  +1 interested in replacing Firebase, would like to know more details
- wolframhempel 9y ago
  deepstreamHub's records can be used instead of Firebase's realtime db/data-sync part, deepstream's events are a replacement for their lately added PubSub capabilities. deepstreamHub does request response as well while Firebase offers integration with numerous Google services. The core differences are in the way deepstreamHub structures its data (small, related documents with independent lifecycles rather than one big data tree), its speed and open core
  - Sujan 9y ago
    Ok, so this is really a replacement for Firebase realtime database, which is 1 of >25 services that Firebase offer nowadays. Right?
    - wolframhempel 9y ago
      that's true - Firebase is pretty much a Gateway to google's cloud and add-platform. deepstreamHub is an alternative to Firebase core (or what Firebase used to be pre Google acquisition)
      - Sujan 9y ago
        Thanks. (Still looks good!)
- akanet 9y ago
  I've recently talked with Wolfram about this, but currently I use Firebase mostly with the Firepad library, Deepstream is missing: * a better presence story * a better story around adding data to a list and subscribing to changes to specific parts of it
bdamm 9y ago
It's great to see so many choices for IoT infrastructure.
- yasserf 9y ago
  co-founder/techie @ deepstreamHub I completely agree! One of the reasons we started is because we believe we have an interesting way of getting devices of all types ( IoT / browsers / phones.. etc ) talking together to generate new ways of displaying and moving data. At the end of the day the selection of choices such as the eclipse iot project https://projects.eclipse.org/projects/iot https://projects.eclipse.org/projects/iot, https://relayr.io/ https://relayr.io/, aws IoT gateways, (shameless plug) deepstream.io and many others allow developers to stop having to think about how to get devices to communicate, but instead think about what they want it to do. I'm slightly overly passionate about this because my bachelor project a decade ago was around smart homes and I spent the majority of the time trying to get protocols to collaborate happily, where now that's just considered part of boiler plate setup! Really excited to see the potential outcome due to this general movement.
rawnlq 9y ago
How does this compare with https://horizon.io/ https://horizon.io/?
- newsat13 9y ago
  Interesting, I though horizon is dead (because Rethink went down). Last news I heard was them going with LF but haven't seen much updates after that
  - lima 9y ago
    Give it some time.
    - dbnoch 9y ago
      Can you provide more information regarding this reply? My opinion is if a service goes dark, I probably shouldn't consider using it.
      - habitue 9y ago
        Recently we had a meeting about the next phase of the project now that Horizon and RethinkDB are owned by the linux foundation: https://www.youtube.com/watch?v=EG0aucGzJE8&t=279s https://www.youtube.com/watch?v=EG0aucGzJE8&t=279s
marknadal 9y ago
Competitor here, please don't become an "open core" company - "Open Core" = "Crippleware". Here is my thoughts/arguments around it, dealing with the RethinkDB and Parse shut down as well: https://hackernoon.com/the-implications-of-rethinkdb-and-parse-shutdowns-c076460058f7 https://hackernoon.com/the-implications-of-rethinkdb-and-par... Plus, was invited to speak about it on the Changelog Podcast: https://changelog.com/podcast/236 https://changelog.com/podcast/236 Please stay truly open :) that way we can still compete. ;) Or else we'll be the only company that is fully Open Source (replication, auto failover, etc.)!
- daveguy 9y ago
  I think open core is ok. If the community is not happy with the core that is open they can fork and add their own implementation for excluded features.
  - marknadal 9y ago
    There was a big uproar over InfluxDB doing this "open core" (aka "Crippleware") approach: https://news.ycombinator.com/item?id=11262318 https://news.ycombinator.com/item?id=11262318 . It isn't the right approach, and we ( http://gun.js.org http://gun.js.org ) aren't going to take it. More than happy to be the only one left standing, devoted to true open values - but I hope others join us.
- eloff 9y ago
  Leaving aside how odd it is to see a "competitor" give business model advice, open core is a perfectly fine open-source business model. Pure support based models, with very few exceptions, leave too much % of value on the table to make a strong business. Purely closed models take away too many freedoms from the customer (but can be very successful nonetheless.) Open core is a very reasonable compromise. The problem with InfluxDB is that they pulled a bait and switch - they launched looking like they were going to be fully open, and then went back on that later. But at the end of the day any compromising on open-source ideals is going to piss someone off - heck even just choosing one OS license over another is going to piss people off. It doesn't matter. In business you need to have a thick skin to stick to your conviction about what is best for both customers and the company - whatever that may be.
drinchev 9y ago
I like deepstream.io. We use it in production. The use case is actually Server <-> Server communication and so far it works good.
srikz 9y ago
So, is this equivalent of Realm Mobile Platform[1]? I found realm to be convenient to use in mobile apps. Haven't tried the mobile platform yet. Anyone here can give a quick comparison of RMP and DeepstreamHub? [1]: https://realm.io/products/realm-mobile-platform/ https://realm.io/products/realm-mobile-platform/
- mwcampbell 9y ago
  They're quite different. In particular, the Realm Mobile Platform is built on a full-blown database running on the user's device. One of Realm's biggest selling points is offline-first applications. As far as I can tell, this is not the case with Deepstream.
  - craighosang 9y ago
    Just to build on what mwcampbell said (and in full disclosure, I'm on the team at Realm). Architecturally, Realm and DeepstreamHub are quite different. Realm at its core is a distributed object database that updates changes to objects in realtime. DeepstreamHub, as far as I can tell, relies on a pub/sub and request/response architecture. They are two fundamentally different approaches to the challenge of realtime. One of the biggest advantages with Realm is that you work with your objects in your native programming language, there's no JSON or ORMs to deal with, just your live objects that update automatically in the local datastore as changes are made on another sync'd client or on the server.
    - wolframhempel 9y ago
      Actually, deepstream has a very similar datasync approach with its records, pub sub and rpcs are mainly a supporting feature.
    - srikz 9y ago
      Thanks for the explanation. I should try out both with this understanding to see which approach is more suitable for different situations.
redka 9y ago
Not exactly Firebase in that it doesn't provide an arbitrarily nested JSON structure as a datastore but more of a document-based approach. Overall a pretty good alternative to Firebase but there is no security rules, no default data store or auth so you need a lot more work. Lists are not very good being just a record with an array of record keys and not the actual data. I use the self-hosted version (deepstream.io) at work where we can't use Firebase because of company policy and regulations.
- deleted 9y ago
  [deleted]
- sorenbs 9y ago
  Deepstream is a great product, but not exactly firebase. For anybody looking for a fully fledged platform to replace firebase, that's exactly what we are targeting with https://www.graph.cool https://www.graph.cool
- datasage 9y ago
  We switched from firebase to deepstream, and are hosting it ourselves. With firebase since we were not running a node or java app, we ended up relying heavily on the rest interface. Which was sort of the not well taken care of. We would see crazy spikes and we could not get support to fix it. Multi tenancy would occasionally cause other performance issues. We eneded up running a bit of a monkey patched system for awhile using kineses to pipe updates to a java/node app which then would write updates via the web socket interface. Firebase also lacked support for pub/sub events, which can be sort of faked, but not entirely reliably. Deepstream model is a bit different as you said, and it does have some weaknesses. I have submitted a number of patches already to fix some of the edge cases i've run into and will likely submit more. Unlike firebase, we can actually directly improve the deepstream application core as well as build options into the platform that better support our use cases.
  - redka 9y ago
    Yes. Firebase needs your apps to written in JS or Java. The lack of support for pub/sub was never a problem for me though since the channelRef.push(msg).remove() works 100% reliably.
katowulf 9y ago
I don't see any benchmarks showing this as "faster." Seems like a pretty serious claim. From the Firebase JavaScript SDK, I can do one thousand individual gets with a 3.4k payload at 1-2ms each. http://jsfiddle.net/katowulf/8wkon62c/ http://jsfiddle.net/katowulf/8wkon62c/ Mileage varies, of course, but I'd love to see a benchmark I can run side by side that can beat those numbers. I can't even get close with a simple GET, which returns in 86ms.
- stemuk 9y ago
  Here you go: https://deepstream.io/info/performance/single-node-vs-cluster/ https://deepstream.io/info/performance/single-node-vs-cluste...
- wolframhempel 9y ago
  also: https://deepstream.io/info/performance/four-billion-messages-per-hour/ https://deepstream.io/info/performance/four-billion-messages... direct benchmarks are something that Firebase' Terms and Conditions make hard to do
  - googsh0tz 9y ago
    Benches just aren't meant for this type of analysis. Two different engines doing two different things. Benches are for recording "personal best" stats, not "laps at the public pool". I.e. Testing against self and not others.
stevenhuang 9y ago
Very interesting, I like that it's open. PubNub was on my mind, but I would need to look into this. If anyone has any experiences with PubNub vs DeepstreamHub, I'd love to know.

118 points by chadfowler 9y ago

Sujan 9y ago

What exactly can be replaced with DeepstreamHub if your are using Firebase right now?

RomanPushkin 9y ago
+1 interested in replacing Firebase, would like to know more details
wolframhempel 9y ago
deepstreamHub's records can be used instead of Firebase's realtime db/data-sync part, deepstream's events are a replacement for their lately added PubSub capabilities. deepstreamHub does request response as well while Firebase offers integration with numerous Google services. The core differences are in the way deepstreamHub structures its data (small, related documents with independent lifecycles rather than one big data tree), its speed and open core
- Sujan 9y ago
  Ok, so this is really a replacement for Firebase realtime database, which is 1 of >25 services that Firebase offer nowadays. Right?
  - wolframhempel 9y ago
    that's true - Firebase is pretty much a Gateway to google's cloud and add-platform. deepstreamHub is an alternative to Firebase core (or what Firebase used to be pre Google acquisition)
    - Sujan 9y ago
      Thanks. (Still looks good!)
akanet 9y ago
I've recently talked with Wolfram about this, but currently I use Firebase mostly with the Firepad library, Deepstream is missing: * a better presence story * a better story around adding data to a list and subscribing to changes to specific parts of it

bdamm 9y ago

It's great to see so many choices for IoT infrastructure.

yasserf 9y ago
co-founder/techie @ deepstreamHub I completely agree! One of the reasons we started is because we believe we have an interesting way of getting devices of all types ( IoT / browsers / phones.. etc ) talking together to generate new ways of displaying and moving data. At the end of the day the selection of choices such as the eclipse iot project https://projects.eclipse.org/projects/iot https://projects.eclipse.org/projects/iot, https://relayr.io/ https://relayr.io/, aws IoT gateways, (shameless plug) deepstream.io and many others allow developers to stop having to think about how to get devices to communicate, but instead think about what they want it to do. I'm slightly overly passionate about this because my bachelor project a decade ago was around smart homes and I spent the majority of the time trying to get protocols to collaborate happily, where now that's just considered part of boiler plate setup! Really excited to see the potential outcome due to this general movement.

rawnlq 9y ago

How does this compare with https://horizon.io/ https://horizon.io/?

newsat13 9y ago
Interesting, I though horizon is dead (because Rethink went down). Last news I heard was them going with LF but haven't seen much updates after that
- lima 9y ago
  Give it some time.
  - dbnoch 9y ago
    Can you provide more information regarding this reply? My opinion is if a service goes dark, I probably shouldn't consider using it.
    - habitue 9y ago
      Recently we had a meeting about the next phase of the project now that Horizon and RethinkDB are owned by the linux foundation: https://www.youtube.com/watch?v=EG0aucGzJE8&t=279s https://www.youtube.com/watch?v=EG0aucGzJE8&t=279s

marknadal 9y ago

Competitor here, please don't become an "open core" company - "Open Core" = "Crippleware". Here is my thoughts/arguments around it, dealing with the RethinkDB and Parse shut down as well: https://hackernoon.com/the-implications-of-rethinkdb-and-parse-shutdowns-c076460058f7 https://hackernoon.com/the-implications-of-rethinkdb-and-par... Plus, was invited to speak about it on the Changelog Podcast: https://changelog.com/podcast/236 https://changelog.com/podcast/236 Please stay truly open :) that way we can still compete. ;) Or else we'll be the only company that is fully Open Source (replication, auto failover, etc.)!

daveguy 9y ago
I think open core is ok. If the community is not happy with the core that is open they can fork and add their own implementation for excluded features.
- marknadal 9y ago
  There was a big uproar over InfluxDB doing this "open core" (aka "Crippleware") approach: https://news.ycombinator.com/item?id=11262318 https://news.ycombinator.com/item?id=11262318 . It isn't the right approach, and we ( http://gun.js.org http://gun.js.org ) aren't going to take it. More than happy to be the only one left standing, devoted to true open values - but I hope others join us.
eloff 9y ago
Leaving aside how odd it is to see a "competitor" give business model advice, open core is a perfectly fine open-source business model. Pure support based models, with very few exceptions, leave too much % of value on the table to make a strong business. Purely closed models take away too many freedoms from the customer (but can be very successful nonetheless.) Open core is a very reasonable compromise. The problem with InfluxDB is that they pulled a bait and switch - they launched looking like they were going to be fully open, and then went back on that later. But at the end of the day any compromising on open-source ideals is going to piss someone off - heck even just choosing one OS license over another is going to piss people off. It doesn't matter. In business you need to have a thick skin to stick to your conviction about what is best for both customers and the company - whatever that may be.

drinchev 9y ago

I like deepstream.io. We use it in production. The use case is actually Server <-> Server communication and so far it works good.

srikz 9y ago

So, is this equivalent of Realm Mobile Platform[1]? I found realm to be convenient to use in mobile apps. Haven't tried the mobile platform yet. Anyone here can give a quick comparison of RMP and DeepstreamHub? [1]: https://realm.io/products/realm-mobile-platform/ https://realm.io/products/realm-mobile-platform/

mwcampbell 9y ago
They're quite different. In particular, the Realm Mobile Platform is built on a full-blown database running on the user's device. One of Realm's biggest selling points is offline-first applications. As far as I can tell, this is not the case with Deepstream.
- craighosang 9y ago
  Just to build on what mwcampbell said (and in full disclosure, I'm on the team at Realm). Architecturally, Realm and DeepstreamHub are quite different. Realm at its core is a distributed object database that updates changes to objects in realtime. DeepstreamHub, as far as I can tell, relies on a pub/sub and request/response architecture. They are two fundamentally different approaches to the challenge of realtime. One of the biggest advantages with Realm is that you work with your objects in your native programming language, there's no JSON or ORMs to deal with, just your live objects that update automatically in the local datastore as changes are made on another sync'd client or on the server.
  - wolframhempel 9y ago
    Actually, deepstream has a very similar datasync approach with its records, pub sub and rpcs are mainly a supporting feature.
  - srikz 9y ago
    Thanks for the explanation. I should try out both with this understanding to see which approach is more suitable for different situations.

redka 9y ago

Not exactly Firebase in that it doesn't provide an arbitrarily nested JSON structure as a datastore but more of a document-based approach. Overall a pretty good alternative to Firebase but there is no security rules, no default data store or auth so you need a lot more work. Lists are not very good being just a record with an array of record keys and not the actual data. I use the self-hosted version (deepstream.io) at work where we can't use Firebase because of company policy and regulations.

deleted 9y ago
[deleted]
sorenbs 9y ago
Deepstream is a great product, but not exactly firebase. For anybody looking for a fully fledged platform to replace firebase, that's exactly what we are targeting with https://www.graph.cool https://www.graph.cool
datasage 9y ago
We switched from firebase to deepstream, and are hosting it ourselves. With firebase since we were not running a node or java app, we ended up relying heavily on the rest interface. Which was sort of the not well taken care of. We would see crazy spikes and we could not get support to fix it. Multi tenancy would occasionally cause other performance issues. We eneded up running a bit of a monkey patched system for awhile using kineses to pipe updates to a java/node app which then would write updates via the web socket interface. Firebase also lacked support for pub/sub events, which can be sort of faked, but not entirely reliably. Deepstream model is a bit different as you said, and it does have some weaknesses. I have submitted a number of patches already to fix some of the edge cases i've run into and will likely submit more. Unlike firebase, we can actually directly improve the deepstream application core as well as build options into the platform that better support our use cases.
- redka 9y ago
  Yes. Firebase needs your apps to written in JS or Java. The lack of support for pub/sub was never a problem for me though since the channelRef.push(msg).remove() works 100% reliably.

katowulf 9y ago

I don't see any benchmarks showing this as "faster." Seems like a pretty serious claim. From the Firebase JavaScript SDK, I can do one thousand individual gets with a 3.4k payload at 1-2ms each. http://jsfiddle.net/katowulf/8wkon62c/ http://jsfiddle.net/katowulf/8wkon62c/ Mileage varies, of course, but I'd love to see a benchmark I can run side by side that can beat those numbers. I can't even get close with a simple GET, which returns in 86ms.

stemuk 9y ago
Here you go: https://deepstream.io/info/performance/single-node-vs-cluster/ https://deepstream.io/info/performance/single-node-vs-cluste...
wolframhempel 9y ago
also: https://deepstream.io/info/performance/four-billion-messages-per-hour/ https://deepstream.io/info/performance/four-billion-messages... direct benchmarks are something that Firebase' Terms and Conditions make hard to do
- googsh0tz 9y ago
  Benches just aren't meant for this type of analysis. Two different engines doing two different things. Benches are for recording "personal best" stats, not "laps at the public pool". I.e. Testing against self and not others.

stevenhuang 9y ago

Very interesting, I like that it's open. PubNub was on my mind, but I would need to look into this. If anyone has any experiences with PubNub vs DeepstreamHub, I'd love to know.