5 ms·
Web Applications from the Future: A Database in the Browser
- rektide 5y agoI don't mind the technical discussion at all, it's a fun write-up, but if you "look at the the ecosystem of web applications and measure by difficulty" and find the most difficult possible app you can, and using that to shape your vision of how to build web apps, lionize what a web app is, I think one is very very liable to end up with a completely non-web monster that ignores many of the strongest strengths of application & site design that are possible. Google Docs is themselves moving to canvas based rendering, which might as well be turning the screen into a giant VNC session into their codebases[1]. Web, dead, pushing pixels in people's faces, in. All the extensions that extend & enhance Google Docs are about to die, being replaced with a very small, much narrower API provided explicitly by google. I see statements like, > This gets me excited about whats to come, because what's at the edge of difficulty today tends to become the new normal tomorrow. And think, please, let us not be path dependent on a web-recreation of classic desktop apps. The web is more interesting, it has online content, liveliness & connectivity to it that far surpasses the other platform's norms. Let us think of how we might advance the web for good web things. There's worth & value to examining hard problems, but I am worried that this attitude has us set out to build faster horses. [1] https://news.ycombinator.com/item?id=27129858 https://news.ycombinator.com/item?id=27129858
- Taek 5y agoI feel like rendering stuff like this is almost a violation of net neutrality. Large companies can spend lots of money on server-side compute and subsidize an experience that can't be replicated sustainably by smaller companies.
- whoisnnamdi 5y agoThis is an incredibly well-written article and helps track and rationalize some of the trends I've been seeing as well around backends as a service that enable developers to easily spin up modern web applications Thanks so much for writing it!
- stopachka 5y agoThanks for the kind words!
- eyelovewe 5y agoWe had Meteor with “mini mongo” and you locally subscribed to streams and it was all a quite bloated PÓS IIRC. At some point someone will make a virtual browser that runs in the browser, or maybe a kubernetes that runs in the browser and the circle of ironic self referencing will be complete. Just replace the browser and use ports 80 and 443 and be done with it, instead of pretending that cpu cycles and RAM are free. The browser is not the app platform of the future, it’s the current bandaid solution is all. I also don’t agree that this article is well written. It exclusively addresses the world according to front end dev, mulch as graphql does. I’m sorry that coding elaborate stuff requires you to keep track of your elaborate stuff. Set your global var equal to the parsed xhr.result and redraw your GUI and get over it, IMO
- hootbootscoot 5y agoBackend devs, meanwhile, wonder where their Frontend as a service is... "backend as a service" = "can you just, like, do that magic CRUD stuff or whatever you do"
- hootbootscoot 5y agoanyway, Postgrest already existed before GraphQL Here's a frontend devs BAAS, replete with soup and desert. (if you don't mind using db users, you even get user-auth per route...) https://postgrest.org/en/stable/ https://postgrest.org/en/stable/
- hootbootscoot 5y agoBut hey, now I'm suspicious... are there Meteor dev's here? IIRC, right on the heels of Meteor we got Apollo from the same folks... The thing is, I'm puzzled, given the insane unnecessary complexity all this ES2030 and "build toolchains" (who knew javascript needed to be compiled, are we making binary bitstreams? is this an embedded microcontroller? is this an FPGA?) and other esoteric "let's replicate all of comp-sci in the browser" frontend tech, why on earth do such smart folks need the backend as a service? surely such folks who can deal with webpack build scripts can write a little php or ruby or java or something...? What is with this desire to completely neutralize and gut the backend stack? why on earth invert what a webserver is? Request=> Response (which might possibly be a rendered webpage) do folks actually think any of this is necessary and/or good? Look, I'm all for web API's, web audio, webrtc, etc... I just don't see where a database in the browser is anything but 'cruisin for a bruisin'... it's not like databases get their own dedicated servers or anything.. or do you mean something like a mobile app using pouchdb? "offline first and syncing to a single real database in the cloud as soon as one rejoins the network" because a browser-based database can only ever be a compromise, such that a compelling reason could force it, but that's something akin to "DB per user" aka the Couch/Pouch ecosystem.
- ampdepolymerase 5y agoNot a single mention of CRDTs or operational transforms in the article. If all you need is a client side database to replicate Figma and Google Docs then they would have long shut down.
- swagasaurus-rex 5y ago"as long as we’re okay with having a single leader, and are fine with last-write-wins kind of semantics, we can drastically simplify this and just facts are enough" It's not a direct reference to CfRDTs or operational transforms. They're not wrong that last-write-wins is easiest to implement, but some applications it doesn't quite cut it.
- dnautics 5y agoMost CRDTs (last-write-wins being one of them) will result in a terrible user experience, and nobody would want to use it for collaboration. That's why most of these collab systems use OT, which is not trivial to implement.
- bitL 5y agoBoth CRDTs and OTs are broken. If you wrote them on your own, it's always "the next fixed bug will finally make it work!".
- avereveard 5y agolast write win isn't that easy either, it's easy only if you ever only send the full data at each write, which is not a great system for large data.
- neogodless 5y agoFor readers like me who do not know every single term in data: https://en.wikipedia.org/wiki/Conflict-free_replicated_data_type https://en.wikipedia.org/wiki/Conflict-free_replicated_data_...
- code-is-code 5y agoYou should have a look at RxDB which does exactly what you have described. Also notice that neither firebase nor supabase is really offline-first. https://github.com/pubkey/rxdb https://github.com/pubkey/rxdb
- xrd 5y agoRxDB + GraphQL (via Hasura) is amazing, and offers all the things the author notes.
- dgreensp 5y agoHaving been a dev on EtherPad, Google Wave, Coda, and other real-time collaborative apps with OT, undo, and so on... I think it's correct that, ideally, there would be a framework that handles real-time collaboration, undo/redo, and offline support for you, and then you build your app with these problems already solved. I will probably create such a framework eventually. I don't see it as a database engineering problem, it's more like a framework or application architecture, which every app like Google Docs or Figma has its own version of. Writing such a framework is not too much harder than writing such an app, it just requires a little more abstraction and some documentation. If you've never written an undo manager, sync engine, etc., and you aren't writing a complex app, it's hard to arrive at the right design by thinking about pure data sync. Also, storing and querying data are solved problems; it's more a question of coming up with a generic data model for an application and defining its semantics.
- kwerk 5y agoWhere would one go to follow along if you start building this framework? You’ve got one pending GH star from me :)
- BiteCode_dev 5y ago+1. I would even pay to watch you code this.
- sambroner 5y ago
- Cthulhu_ 5y agothis comment turned into a bit of a personal rant, apologies. I really liked this post, because it touches on so many things that I have to build as well at the moment. I'm building a configuration management interface; the front-end is basically authentication and heaps of forms, the back-end transforms it into XML and uses some shell scripts to rsync them to servers and SNMP to trigger a deployment. But the users have worries about overwriting each other's work, they ask for undo / revert support, and there's things like audit logging, user and permissions management, etc involved. At the moment I'm slowly building all of that with a React/TS front end and a Go backend using a REST API, just trying to be as tidy and complete as possible. But it's a lot of work, and I'm afraid that once I get to things like versioning, locking, undo / revert, auditing and permissions, my fairly straightforward codebase will just explode in complexity, with each endpoint having loads of documentation and code to represent the different concerns. Client- and server-side validation is another concern, importing existing data, migrating user data from a staging to production environment, etc. It's a lot. It's a project that should be done by a full team of developers, maybe even multiple but ATM I'm doing it on my own. I'll never finish it. I'm currently trying to plough through and make sure everything is set up, but I'm hoping we'll get a big financial injection and I'd be able to hire a bunch of consultants (that I'm familiar with as being decent if expensive developers). I don't know how the previous guy managed to get as far as he did on the older version, other than being a mediocre but productive and persistent developer for all that time, seeing the whole thing grow over time instead of trying to reach feature parity with a product nine years in the making.
- stopachka 5y agoEmpathize with you, and from the writing I am sensing the stress you are in. One thought that may help: As an engineer, your job is to communicate risks. You can’t control whether this becomes impossible, but you can control how you work on it and how you communicate. One way to do this is to keep an up to date design document with progress and risks. Looks like there are some issues that are worrying you. If mot already would write it out and get thoughts from stakeholders. Rooting for you!
- ralusek 5y agoTo me it always come down to permissions. I've tried firebase, I've tried graphql. I just can't use them for anything other than admin endpoints or things with very simple permissions. I'm glad the author mentioned permissions, but I would need to see some seriously compelling evidence that I could trust a declarative, resource-based permissions system to accomplish what I need.
- zubairq 5y agoWe already have this in Visual Javascript. When you build an application and export it as HTML it has a full SQL database that runs just in the browser: https://github.com/yazz/visualjavascript https://github.com/yazz/visualjavascript
- j-pb 5y agoThere is a very handy RDF DL subset that has all of the desired properties described in the article. Funnily enough we are targeting the browser AND FPGAs, and I think the latter is the much more interesting use case for a distributed reactive CRDT-like database. Datalog is actually trivial to incrementally materialize in open world semantics. Which is why datomic (while a great foundation in theory) turns out to be non ideal. TxnIDs and retractions are essentially nonmononic negation in disguise, and CALM (consistency as logical monotonicity) a.k.a. distributedness, doesn't go well with that. Seeing that we're not the only ones dreaming of this gives me hope though, that we might get out of the tar pit someday.
- tlarkworthy 5y agoJust use Firebase (firestore) ... It's got: offline first, latency compensation, pubsub, partial sync, authorization rules and serverside timestamps, and global transactions, and 5 9s availability backed by spanner for umpteen languages. https://tomlarkworthy.endpointservices.net/blogs/firestores-technical-advantages.html https://tomlarkworthy.endpointservices.net/blogs/firestores-...
- barbarbar 5y agoBut then one day your Google account is closed by AI. Or the product is abandoned by Google.
- typingmonkey 5y agoFirestore is not really offline-first. Your app wont start without internet connection because the auth handler needs that.
- tlarkworthy 5y agono, it works as long as you logged in once. You have to ensure auth persistence is on (https://firebase.google.com/docs/auth/web/auth-state-persistence https://firebase.google.com/docs/auth/web/auth-state-persist...). You cannot login when offline, but if you were already logged in, and expiry is set to indefinate, you can stay logged in while offline.
- twobitshifter 5y ago[Automatic Undo/Redo Using SQLite](https://www.sqlite.org/undoredo.html https://www.sqlite.org/undoredo.html) No sure on the scalability of SQLite but here’s a simple way to get database undo/redo.
- vaughan 5y agoGreat post. Captured the problem nicely. I went on a similar journey recently. There is a moment when you step back and realize that all the data-wrangling code we write in apps is essentially what an SQL query planner does. And our frontend is just one big materialized view that we need to update. I often think that if we had a database that runs in the browser, and we could subscribe to queries, and missing local data would be fetched on-demand, frontend development would be a lot easier. Its of course much more complicated than that. I think the reality though is that backend scaling requirements always end up dwarfing frontend productivity concerns. And there is also a huge amount of glue between backend data sources, preventing the creation of a clean database on the backend to sync with, meaning we are creating API gateways and GraphQL federation layers, and all we can hope for is a good client-side caching layer. If you look deeper though you will find many projects doing all these kinds of things already. Mobile developers are very familiar with offline techniques and SQLite on the client, and it doesn't feel like anything special to them. For the web/desktop, maybe no one has packaged it in the right way yet, or maybe we are still digesting GraphQL, SSR, and serverless, and then there will be another shift with offline-first, reactive SQL in the frontend. It's funny though that if you think long enough about these concerns, you always seem to end up wanting some Datalog thing.
- abss 5y agoYou should also look to OpenDSU (opendsu.com) that covers in a unique ways these aspects. Basicaly it goes to build on the vision in wich applications are running inside digital wallets and you control your data (client side encryption) while beeing able to collaborate with others and get security in a decentraliassd way using various types of anchoring in ledgers (not necesarly distributed or your typical blockchains but of course this could be the case too)
- farnulfo 5y agoAPI rate limit exceeded for user ID 984574.
- enos_feedler 5y agoA lot of the principles here echo the same design principles that MeteorJS was built on. Why didn't that work and why now? Is there something about _today_ that makes the web application space different?
- atatatat 5y agoPeople actually corralled into using webapps by big boys, phones not being pieces of shite.
- psvj 5y agothis site appears to have exceeded its API rate limit. I wonder if that is a problem what will be solved by 'web applications of the future'. but i guess ill never know now
- deleted 5y ago[deleted]
- thewakalix 5y agoAbout storing history rather than current-state: I think Urbit works similarly.
- nickthemagicman 5y agoGetting API rate limit exceeded. What is this hosted on?