9 ms·
How to use a Google Spreadsheet as a database
- mooreds 11y agoI beat my head on APIs from Google spreadsheets, so good on ya! How does blockspring deal with Google Spreadsheet availability issues? I remember the spreadsheet not always being available.
- donpinkus 11y agoThanks! Yea I was trying to use Google Spreadsheet, getting annoyed with the JSONP and the goofy way they returned their JSON, so figured I'd just wrap it in something easier to use... What "availability issues" did you run into? I haven't noticed any with my sheets yet
- justincormack 11y agoI run a cache in front of mine. It is usually available, but can be slow.
- mooreds 11y agoJust the spreadsheet data being intermittently down, and of course, impossible to troubleshoot. We ran processes against it fairly often, and even had it proxied to some mobile devices.
- bduerst 11y agoSame here! We went from using Spreadsheets to using our own flat files on Drive, but the API service would throw random rejection errors for both. Long story short, we learned that you should not try to use Drive or Google docs as a program database. It's designed first and foremost for users.
- pg_is_a_butt 11y ago"random rejection errors"... AKA "webscale". things are going to fail. you need to add fault checking and retries with exponential backoff.
- jontro 11y agoI was looking for the google spreadsheets api earlier today and I noticed it has not been migrated away from GData yet. I wonder why this is not getting any love https://code.google.com/p/gdata-java-client/ https://code.google.com/p/gdata-java-client/ <--- Spreadsheets api is one of the three apis still alive out of all gdata apis
- pg_is_a_butt 11y agobecause the spreadsheets were originally developed by an outside company that was acquired and injected into the google apps ecosystem. supposedly google is working on a new "Sheets API" but it isn't available for the public yet... however i think the new "Sheets" use that API internally now.
- eddyparkinson 11y agoavailability: you need a retry. See http://stackoverflow.com/questions/22099780/unable-to-complete-the-http-request-when-using-spreadsheet-api/22134470#22134470 http://stackoverflow.com/questions/22099780/unable-to-comple...
- deleted 11y ago[deleted]
- eatonphil 11y agoI've been wanting to use Drive as a db recently on an app I'm working on as cheaper/free data storage. That is, instead of hosting a db and storing user data there, I would store it on each user's google drive. I see it as a way to save costs by not worrying about the security of my databases, cost of space/uptime/traffic, etc. What are issues am I not thinking of?
- donpinkus 11y agoIt's an interesting idea - here are some issues to consider: 1. Joining data will be very slow. If you need to access 500 database to get the "comments" on a "post", you're going to have issues. 2. How will you change the database structure as you iterate? 3. Storage cost of data is so low, that by the time you would start paying for data, you would have greatly exceeded the capabilities of Google Sheets. 4. Google sheets are slower than databases - there are no indexes, keys, the data is not stored in a way meant for most db operations (selections, etc) I'm sure theres more but these seem to be the biggest ones for me. That said, you should definitely try it - it's an interesting project at the very least.
- dudus 11y agoStorage costs are not low. They are free since Google Spreadsheets do not count towards your Storage Limits. You'll be limited only by the 2M maximum cells a spreadsheet can hold.
- ww520 11y agoWhat're the rate limits on Google Spreadsheet on update and read?
- donpinkus 11y agoThat's a really good question. I haven't hit a limit yet and tested with 200 queries / minute. Will update this comment when I find that out.
- ngoel36 11y agoI'm shocked that using something like https://github.com/gimite/google-spreadsheet-ruby https://github.com/gimite/google-spreadsheet-ruby to, at the very least, let Google Spreadsheets as a proxy for a bare-bones CMS hasn't been more widespread
- donpinkus 11y agoYea seriously... I thought of it since a buddy has some Tiki Bar that he wanted a site for, and I really didn't want to drop into WordPress or anything serious. Knew he could handle a spreadsheet. Checkout http://www.tarbell.io/ http://www.tarbell.io/ - It's a CMS designed around google sheets. I think it's a bit of setup, but might be a solid solution.
- pkpp1233 11y agoYeah it's really surprising. These are my favorite examples online :): - http://blog.apps.npr.org/2014/04/23/how-we-built-borderland-out-of-a-spreadsheet.html http://blog.apps.npr.org/2014/04/23/how-we-built-borderland-... - http://www.gamasutra.com/blogs/WillHankinson/20150323/239489/Turning_Data_Into_Enemies_How_We_Used_Google_Spreadsheets_as_a_CMS_for_Unity_in_Defend_the_Dam.php http://www.gamasutra.com/blogs/WillHankinson/20150323/239489...
- huskyr 11y agoUsing Google Spreadsheets as a bare-bones CMS is a very popular option in news media, like newspapers.
- mavhc 11y agoWhere's the free, web based, easy to use database, the web version of MS Access?
- crazygringo 11y agoI also wonder. It seems like the glaring hole in the Google Apps suite.
- jewel 11y agoI've had this as my startup idea for at least four years now. I'd build a web-based database product that would fit in google apps with the intention of getting bought by google and integrated into google apps. Even if I didn't make any money on the deal, at least I'd be able to have something better for tracking the progress of the cub scouts awards. It'd also let my wife and me set up the complex budget that we're trying to do right now in spreadsheets. I bring this up with people who work for small businesses and they all recognize the pain point. Those that are old enough remember the good old days when you could put an Access file on a network share and then run your entire business out of it.
- eatonphil 11y agoIf you're interested, I've wanted to work on this too. In particular, creating an online, PaaS database service so you can store and manipulate data via SQL queries without needing to setup and maintain (and share across machines) your own sql server. It's something I find myself wishing I had.
- zyxley 11y agoThe key parts of Access aren't that it can have SQL queries, it's that it (a) has a UI where you can easily edit tables (without worrying much about schemas and so on), (b) has a UI where you can plug tables into WYSIWYG-edited reports, and (c) has a UI where you can plug tables into WYSIWYG-edited forms.
- 11y ago
- dudus 11y agoIn the real world I think there are 2 main use cases here. # 1st) Using Google Spreadsheets as a CMS In this case you'd store data in a Google Spreadsheet and retrieve the content before showing to the user. Probably it makes sense to put some durable caching in place so you can sync the cache offline and worry less about Google Spreadsheets API downtime, quotas or latency. In this scenario the app would only read data from the Spreadsheet and not write. It will probably not support writes consistently for anything more than a toy. # 2nd) Use Google Drive to store user Data The main difference here is that in this case it would make more sense to store the spreadsheet in the user account, not yours. You'd fetch the userData once he logs in your application. If this is the use case there are better things than writing spreadsheets to users Google Drive. There's actually a feature in Google Drive to store application data: https://developers.google.com/drive/web/appdata https://developers.google.com/drive/web/appdata
- bcRIPster 11y agoA spreadsheet is NOT a database! Argh! As a person who gets asked to fix these kinds of projects once they hit a wall (performance/concurrency/etc) and then have to migrate them to a proper DB platform, just stop it! Put it on in a DB up front and save some poor developer their sanity. Please.
- hbhakhra 11y agoHe mentions that its explicitly for quick prototyping. I've started a few projects but stopped when setting up the infrastructure became too tedious. For the quick and dirty prototype, this seems perfect.
- camhenlin 11y agoI normally use nedb for that, works great. I'm sure there's probably something similar for whatever other platform you might be using as well
- bcRIPster 11y agoI understand, but every time I get handed one of these it's because someone who didn't know what they were doing was rapid prototyping something and woops they have 200 hundred users and it's crashing, it's corrupting data, etc... At one job I'm like "people, just use Access, it's installed on your computer" and they look at me like I'm talking dark wizardry shit with their fingers itching on their pitchforks because they don't know if I'm going to eat their babies. Arrrrgh!
- cdcarter 11y agoWouldn't that put you out of work, then? ;) But in all seriousness, using a spreadsheet for prototyping makes perfect sense. Why waste a ton of time setting up a database when you're still figuring out what you are doing, and a spreadsheet works just fine? Yes, there's some hassle when you have to migrate, but that's compared to the hassle of setup. The amount of time to get the first iteration launched is a LOT more valuable than time down the road.
- 11y ago
- jaybna 11y agoClosest any company has come to a decent web version of Access is Intuit Quickbase. But it is priced for enterprise and not hobbiest. Lacks full SQL but can do some pretty amazing things. Also has reasonable REST-like API. I built a PoC data backend for an iPhone app really easily.
- cdcarter 11y agoAnother option is of course Salesforce/Force.com. Point and click, incredible customization potential, easy to build UI and reports, point-and-click business processes, and very extensible by code if your admins can't accomplish something. Still very expensive, but free for non-profits!
- aofstad 11y agoWe also recently released an early version of the Airtable API. It provides an API that's specific to each database you've configured in the app: https://airtable.com https://airtable.com https://airtable.com/api https://airtable.com/api
- ecesena 11y agoI'm a big fan of spreadsheets instead of db -- it has it's limitations of course, but works great for apps with a few thousand read-only records (items/products or even just text captions). I built HasGluten [1, 2] with react + google spreadsheet, hosted on github for free, you get a cheap, scalable, geo-distributed software stack, with simple interfaces to maintain both code (GitHub Pages) and data (Google Sheets — also great for the non-tech). [1] http://hasgluten.com http://hasgluten.com [2] https://github.com/hasgluten/hasgluten https://github.com/hasgluten/hasgluten
- thalesmello 11y agoIn Brazil all food manufactures are required to put in the package whether the product contains gluten or not. Isn't that the case in the United States?
- ecesena 11y agoIt is, but try to go to Safeway and buy a GF powerbar... :) Jokes aside, we maintain the list mostly for ingredients. Many people, especially "novice", often ask the same questions -- or you may have doubts for strange/unfamiliar foods, for instance if you're traveling in a foreign country.
- dpweb 11y agoYou can grab published Google Sheets in JSON using, https://spreadsheets.google.com/feeds/list/{{doc_id}}/od6/public/values?alt=json https://spreadsheets.google.com/feeds/list/{{doc_id}}/od6/pu... example, https://spreadsheets.google.com/feeds/list/1btWWclsRW6-wrIdCD8Po5a0VTSh2BJhiPohWiJX8WaQ/od6/public/values?alt=json https://spreadsheets.google.com/feeds/list/1btWWclsRW6-wrIdC...
- ecesena 11y agoI'm actually using multiple sheets and jsonp [1], but that's essentially what I'm doing. [1] https://github.com/hasgluten/hasgluten/blob/master/src/app/pages/Layout.js#L161 https://github.com/hasgluten/hasgluten/blob/master/src/app/p...
- wuyingzhong1 11y ago1. open google spreadsheet 2. think about your application 3. chose normal form, design schema, add integrity constraints, build indexes and query execution and optimization engine and done!
- grw_ 11y agohere is django library for this: https://github.com/georgewhewell/django-sheets https://github.com/georgewhewell/django-sheets
- donpinkus 11y agonice. who wants to fork this for Rails? :P
- clucktheduck 11y agothis is cool beans!
- jedschmidt 11y agoI really like the idea of using Google Spreadsheets as a quick and familiar GUI for entry or querying on data sets, as long as you understand the tradeoffs (write latency isn't great and they max out at 400,000 cells). But this is especially nice when you build a layer on top of Google reduces lock-in, instead of adding another proprietary API. This is what I did with sheet-down[1], which turns a Google Spreadsheet into a LevelDB-compatible data store that can be swapped out with a file system or other compatible backend[2] once you outgrow Google. [1] https://github.com/jed/sheet-down https://github.com/jed/sheet-down [2] https://github.com/rvagg/node-levelup/wiki/Modules#storage https://github.com/rvagg/node-levelup/wiki/Modules#storage
- bradleyland 11y agoHow do you handle type constraints? For example, how do you prevent users from inputing text in to number fields? I know Google Sheets has validators, but I don't know of any way to restrict someone from changing those validations while also providing the ability to input data. Really curious, because I'd love to use something like this in our app.
- jedschmidt 11y agoMy situation is for an intranet environment, so I suppose I could just ask nicely that they don't change validations. But chances are most of my target users aren't familiar enough with Google Spreadsheets to even get that far.
- donpinkus 11y agoThey upped the limit from 400,000 cells - I think it's around 2M now, will try to find the source
- mslate 11y agoNice content marketing
- donpinkus 11y agoThanks! Nice observation :)
- Ciantic 11y agoI have used Google Spreadsheet as a database, the hassle is not worth it. You need to do some serious caching: insert caching, update caching etc. The API calls weren't very sturdy few years ago. Even then you hesitate to give rights to laymen to edit the spreadsheet since everything breaks if they screw up. And what is the point if it can't be shared?
- jlouvel 11y agoAPISpark (PaaS for APIs) is capable of creating a REST API (JSON/YAML/XML formats) on top of a Google Spreadsheet, using it as a database: http://restlet.com/technical-resources/apispark/tutorials/turn-spreadsheet-to-api http://restlet.com/technical-resources/apispark/tutorials/tu... In the latest version, it comes with a server-side API cache to prevent GSheet latency and availability issues. Note: I'm the founder of APISpark
- eddyparkinson 11y agoNot quite the same as gridspree, but has a lot in common. blockspring has SQL like commands in the client. gridspree has data formatting in the client. https://assembly.com/gridspree https://assembly.com/gridspree