6 ms·
SPQR 1.3.0: a production-ready system for horizontal scaling of PostgreSQL
- soleri17 2y agoDo you have some plans for k8s operator?
- denchick 2y agohttps://github.com/pg-sharding/spqr/issues/571 https://github.com/pg-sharding/spqr/issues/571
- u320 2y agoDoes it handle byzantine failures?
- gclawes 2y agoNo, only Roman failures
- bheadmaster 2y agoLike the one where they failed to conquer that village of Gauls?
- SketchySeaBeast 2y agoAs usual, Asterixes are to be avoided.
- alexvoda 2y agoBesides, there are plenty of Obelixs to be found in Egypt. There is no need to get your phalic stone monuments from the Gauls. (I expected Roman puns and was not disappointed)
- lenerdenator 2y agoSounds like a pretty germane feature to have.
- skrebbel 2y agoParent means Eastern Roman failures
- m0rshu 2y ago[dead]
- hot_gril 2y agoSo during a Parthian event, they lose the Crassus.
- deleted 2y ago[deleted]
- geenat 2y agoWhy use SPQR over Citus? Vitess?
- lebski88 2y agoThe fact that it's a router and not a PG extension is helpful if you are using cloud hosted postgres and don't want to get into the business of managing backups etc yourselves. Only azure supports Citus as far as I know.
- scraplab 2y agoCrunchyBridge will run you a Citus cluster on any of the major cloud providers.
- feverzsj 2y agoSPQR, as the name suggests, is just a query router.
- calvinmorrison 2y agoYes, that's what the name suggests
- verandaguy 2y agoThe name also suggests plans to go to war with the Carthaginians.
- jitl 2y ago
- bookofjoe 2y agoSPQR https://en.wikipedia.org/wiki/SPQR https://en.wikipedia.org/wiki/SPQR
- deleted 2y ago[deleted]
- Mortiffer 2y agoA comparison to https://www.citusdata.com/ https://www.citusdata.com/ would be nice
- mdaniel 2y agofor one thing: https://github.com/pg-sharding/spqr/blob/1.3.0/LICENSE https://github.com/pg-sharding/spqr/blob/1.3.0/LICENSE (BSD2) https://github.com/citusdata/citus/blob/v12.1.2/LICENSE https://github.com/citusdata/citus/blob/v12.1.2/LICENSE (AGPLv3)
- waldrews 2y agoI still have a hard time understanding where the AGPL copyleft feature kicks in on a product like Citus. Are you safe just deploying it as part of your backend? Like, do you have to actually touch Citus's own code to trigger it? What if you start using some sprocs or sample connectivity code? Anybody know a good set of guidelines?
- wavemode 2y agoThis is the entirety of the text which was added to the AGPL causing it to differ from the plain GPL: > Notwithstanding any other provision of this License, if you modify the Program, your modified version must prominently offer all users interacting with it remotely through a computer network (if your version supports such interaction) an opportunity to receive the Corresponding Source of your version by providing access to the Corresponding Source from a network server at no charge, through some standard or customary means of facilitating copying of software. This Corresponding Source shall include the Corresponding Source for any work covered by version 3 of the GNU General Public License that is incorporated pursuant to the following paragraph. In other words, if you deploy a modified version of Citus, your modified version is AGPL licensed, and thus you must provide the source code of this modified version to all users who interact with it remotely (e.g. through your web application). What it does not state is that you must provide the source code of your entire web application just because you deployed a modified version of Citus, nor that your web application becomes an AGPL-licensed derived work of Citus because you used it over a network. The AGPL also does not require anything at all from you other than the plain GPL's basic requirements if the version of Citus you deploy is unmodified. (These are all incredibly common misconceptions on the internet, by people who've never read the license nor the GNU website.)
- vb-8448 2y agosono pazzi questi romani
- vincnetas 2y ago"On the first page where we find the map of France we see a Roman banner with S.P.Q.R. which means "Senatus Populus Que Romanus" (The senate and people of Rome). The Italian translator made a great pun about it and translated it into "Sono Pazzi questi Romani". The latter can be translated to "These Romans are crazy"." https://www.asterix-obelix.nl/index.php?page=hjh/dos-italy.inc https://www.asterix-obelix.nl/index.php?page=hjh/dos-italy.i...
- wlll 2y agoUnrelated, but you reminded me of this clip from The Life of Brian: https://www.youtube.com/watch?v=wjOfQfxmTLQ https://www.youtube.com/watch?v=wjOfQfxmTLQ It's funnier if you know that the ear-twisting Roman at the start was basically a caricature of many Latin teachers that were around when Latin was taught more widely in schools (such as when the Monty Python team were young).
- davemo 2y agoVeni, Vidi, Vici
- deleted 2y ago[deleted]
- jimmywetnips 2y agoNice
- ThinkBeat 2y agoHow far can one get these days with vertical scaling of Postgres? I dont know how well the engine could make use of the extra resources. If it scales well vertically, I expect it to cover 80% - 95% of use cases for people using Postgres. Probably with less complexity, and less overhead than scaling horziontally
- endymi0n 2y agoFar. As in, really, really far. We started out with Postgres because it was just the simple and sensible option for a production prototype, and when somebody came around telling me we need something more scalable recently, I calculated that there's not even enough addressable market in the world for our business for more than 4x our size. That's exactly the two remaining vertical doublings of our DB instance (to ridiculously looking RAM & CPU numbers) we could still do in case we need it. Other than that, thanks to a lot of the recent work on connection handling and concurrency since PG 11, Postgres is getting better and better actually using these additional resources well: https://www.enterprisedb.com/blog/performance-comparison-major-PostgreSQL-versions https://www.enterprisedb.com/blog/performance-comparison-maj...
- ThinkBeat 2y agoSeveral years ago I was a contractor on a project for that would sip up data from all forms of sensor and moving objects around a large city. It was decided that we needed Kafka and a few similar tools to handle it. It was not hard to calculate the current max traffic or estimate the traffic growth over the next 10 years. I did a demo of the system running on my laptop (all of it) + Postgres handling 100x the current data without too much difficulty. Still they went with the "scale" solution because it was the right design. (and of course the consultants and me got quite lot more work todo so made a good deal more money)
- mdasen 2y agoAt this point you can get 24TB of RAM in an EC2 instance (along with 448 vCPUs, 100Gbps of network bandwidth and 38Gbps of EBS bandwidth). That won't scale forever, but Stack Overflow has been running on a single primary/standby setup with 1.5TB of RAM so that would be 16x Stack Overflow's RAM. I think a lot of work goes into horizontal scaling which is necessary at a certain scale, but very few people actually get anywhere near that scale. It can be important to understand which things are needed at your scale and where you can simply buy some beefier hardware. I've been at places where people run a dozen sharded DB servers with each server having 16GB of RAM. Maybe that's resume-driven-development where someone wants to say they've done that.
- lenage 2y agoHappy to see so many tools created for PostgreSQL, Thanks for your great work
- AtlasBarfed 2y agoSQL is problematic to horizontally scale, joins in particular. I see nothing about partition tolerance, so I assume it isn't at all. Give me Aphyr tests or there is no reason to pay attention.