20 ms·

in case anyone finds this useful, here's the slowly growing collection of links to similar tools: https://tinybase.org/ https://tinybase.org/ https://www.evol

in case anyone finds this useful, here's the slowly growing collection of links to similar tools: https://tinybase.org/ https://tinybase.org/ https://www.evolu.dev/ https://www.evolu.dev/ https://replicache.dev/ https://replicache.dev/ https://fireproof.storage/ https://fireproof.storage/ https://vlcn.io/ https://vlcn.io/ https://www.instantdb.com/ https://www.instantdb.com/ https://loro.dev/ https://loro.dev/ https://electric-sql.com/ https://electric-sql.com/ https://docs.y-sweet.dev/ https://docs.y-sweet.dev/ https://syncedstore.org/docs/ https://syncedstore.org/docs/ https://collabs.readthedocs.io/en/latest/ https://collabs.readthedocs.io/en/latest/ https://remotestorage.io/ https://remotestorage.io/ https://rxdb.info/offline-first.html https://rxdb.info/offline-first.html https://github.com/siriusastrebe/jsynchronous https://github.com/siriusastrebe/jsynchronous https://www.powersync.com/ https://www.powersync.com/ https://pouchdb.com/ https://pouchdb.com/ https://jazz.tools/ https://jazz.tools/ https://www.triplit.dev/ https://www.triplit.dev/ https://automerge.org/ https://automerge.org/ https://www.dxos.org/ https://www.dxos.org/ I hope one day to try them all :-) Or read a summary from someone who does.

ozim 1y ago
I wonder why there are so many, just people reinventing stuff that no one really needs? For me personally I have 4 of those as visited, pouchdb, automerge, loro and sqlsync of course. I was trying to fit such a tool into existing architectures that I deal with at work but nothing really makes sense. My guess is those solutions are in totally wrong abstraction layer, creators think that would be best thing since sliced bread - but in reality having rest API and some persistence on client is like 99% good enough. With service workers in browser and mobile apps no problem of just having data stores. Sending out specific partial updates, just reloading full state from the server is just easy to explain to the users and easy to implement. Last write wins with some auditing log is also good for something like 99.9% of applications and is super easy to explain to people - what's not easy to explain and not easy to implement is merging conflicts on database data. It is not easy to make audit logs server side so they are not tampered with if you just sync full database instead of doing REST requests. This approach with "sync databases" feels for me exactly like someone proposing use of LateX because it is great to people who need to write 10 page essays.
- whizzter 1y ago
  It's about the multiplayer application case, think Google Write/Sheets/etc. Applications with data that can change by multiple users and you can both see it live and the application (that keeps state in memory/localdb) is also resilient to disconnects. The reason people descend into this madness is because visible replication code is tricky and the general feeling is that it'll infect parts that shouldn't be infected (or at least not without a general framework). So at a somewhat trivial level you have: A: A bare log replication system (where the application needs awareness for most object types multiplying object complexity). B: A object replication system where the framework handles all object types coherently and the application "only" needs to be aware of how to fetch objects, think a KV store that stores fairly basic objects with some addressing support. C: Since recent crowd "wisdom" dictates that most KV stores will likely re-implement SQL functionality badly, people go straight to doing the SQL case (maybe they've had a curiosity about SQL databases already that they're scratching) I've recently built A (basic replication and LWW) and building the application I'm definitively feeling an itch to just start over or adjust to support B (a simple replicated KV store) to separate the concerns more, I can see how I would also feel the SQL itch of C (but having done SQL like systems before it's not as bad for me). For this first application A will suffice (since the offline needs are relatively trivial) but having a more complicated application in mind I'm strongly considering B for that project (along with designs or third party libs to make it happen). I think a big gap in the space is that most seem focused on "documents" (esp the CRDT based tools), ie a document being the atomic piece that is synchronized but imo it leaves a big gap in that now all regular application management tools like SQL query tools are useless since essentially you only have a bunch of "blobs" or worse. If you want the regular enterprise SQL backend these tools don't seem to have a focus on synchronizing to those regular backend storage systems.
  - ozim 1y ago
    I don’t see it you missed the context or I miss something. Multiplayer documents are real time synchronized and since they are documents that’s totally not use case for DB synchronization. All the tools are for offline to online data synchronization. Different use case than document.
codeulike 1y ago
"nearly every problem these days is a synchronisation problem" https://news.ycombinator.com/item?id=43434239 https://news.ycombinator.com/item?id=43434239
isaachinman 1y ago
Wrote a very-related blog post here: https://marcoapp.io/blog/offline-first-landscape https://marcoapp.io/blog/offline-first-landscape
- vladstudio 1y ago
  thanks for the link! just in time for me.

ozim 1y ago

I wonder why there are so many, just people reinventing stuff that no one really needs? For me personally I have 4 of those as visited, pouchdb, automerge, loro and sqlsync of course. I was trying to fit such a tool into existing architectures that I deal with at work but nothing really makes sense. My guess is those solutions are in totally wrong abstraction layer, creators think that would be best thing since sliced bread - but in reality having rest API and some persistence on client is like 99% good enough. With service workers in browser and mobile apps no problem of just having data stores. Sending out specific partial updates, just reloading full state from the server is just easy to explain to the users and easy to implement. Last write wins with some auditing log is also good for something like 99.9% of applications and is super easy to explain to people - what's not easy to explain and not easy to implement is merging conflicts on database data. It is not easy to make audit logs server side so they are not tampered with if you just sync full database instead of doing REST requests. This approach with "sync databases" feels for me exactly like someone proposing use of LateX because it is great to people who need to write 10 page essays.

whizzter 1y ago
It's about the multiplayer application case, think Google Write/Sheets/etc. Applications with data that can change by multiple users and you can both see it live and the application (that keeps state in memory/localdb) is also resilient to disconnects. The reason people descend into this madness is because visible replication code is tricky and the general feeling is that it'll infect parts that shouldn't be infected (or at least not without a general framework). So at a somewhat trivial level you have: A: A bare log replication system (where the application needs awareness for most object types multiplying object complexity). B: A object replication system where the framework handles all object types coherently and the application "only" needs to be aware of how to fetch objects, think a KV store that stores fairly basic objects with some addressing support. C: Since recent crowd "wisdom" dictates that most KV stores will likely re-implement SQL functionality badly, people go straight to doing the SQL case (maybe they've had a curiosity about SQL databases already that they're scratching) I've recently built A (basic replication and LWW) and building the application I'm definitively feeling an itch to just start over or adjust to support B (a simple replicated KV store) to separate the concerns more, I can see how I would also feel the SQL itch of C (but having done SQL like systems before it's not as bad for me). For this first application A will suffice (since the offline needs are relatively trivial) but having a more complicated application in mind I'm strongly considering B for that project (along with designs or third party libs to make it happen). I think a big gap in the space is that most seem focused on "documents" (esp the CRDT based tools), ie a document being the atomic piece that is synchronized but imo it leaves a big gap in that now all regular application management tools like SQL query tools are useless since essentially you only have a bunch of "blobs" or worse. If you want the regular enterprise SQL backend these tools don't seem to have a focus on synchronizing to those regular backend storage systems.
- ozim 1y ago
  I don’t see it you missed the context or I miss something. Multiplayer documents are real time synchronized and since they are documents that’s totally not use case for DB synchronization. All the tools are for offline to online data synchronization. Different use case than document.

codeulike 1y ago

"nearly every problem these days is a synchronisation problem" https://news.ycombinator.com/item?id=43434239 https://news.ycombinator.com/item?id=43434239

isaachinman 1y ago

Wrote a very-related blog post here: https://marcoapp.io/blog/offline-first-landscape https://marcoapp.io/blog/offline-first-landscape

vladstudio 1y ago
thanks for the link! just in time for me.