imbradn

6 ms·

1.
by imbradn 9mo ago
It was carried forward from days before checksums were a thing for the main data files. Enabling them on existing clusters was not added until PG 14 (in core - there was a utility prior that that could do it, but not much before). So before
2.
by imbradn 1y ago
We used it to shard a database at Instacart after your time instead of the “replicate everything and drop the data on the shard approach”. That combined with dropping all the indexes (minus the primary key) worked well for the data copy&#x2
3.
by imbradn 1y ago
Logical replication row filtering is a much better alternative to the ship and drop approach.
4.
by imbradn 2y ago
Often lost in these discussions is how much more difficult upgrading is at scale. The article talks about the challenges with upgrading a 4TB database. In my world - that’s a small database. Trying to setup a logical replica of a much large
5.
by imbradn 2y ago
No, vacuum issues are not solved. This will reduce the amount of scanning needed in many cases when vacuuming indexes. It will mean more efficient vacuum and quicker vacuums which will help in a lot of cases.
6.
by imbradn 2y ago
It's based on Patroni - which they wrote and maintain and is the most popular HA Failover software for Postgres.
7.
by imbradn 2y ago
The acquisition of Oriole is very interesting. Getting that to a GA product would be a game changer for a lot of workloads.
8.
by imbradn 2y ago
Except for the un-implemented features which they might need. It also uses serializable isolation and in their implementation reads are blocked by writes unlike in Postgres. Those are both significant changes that can have far reaching appl
9.
by imbradn 2y ago
IOPS isn’t a linear thing here. The vacuums needed to prevent transaction wraparound (vaccum Freeze)can’t be throttled and are much more expensive than regular vacuums. By splitting the tables they are likely reducing the need for those vac
10.
by imbradn 3y ago
That article covers the basis of how we do upgrades at Instacart, but is quite old. This is a more modern look at how we accomplish the process. We have used this process to upgrade a lot of very large and very active databases successfully
11.
by imbradn 3y ago
But in those days, in addition to the license prices for closed source DBs, you also had to pay all the hardware costs to run them on, and hire an army of DBAs to support and maintain them for you. Running HA Postgres is not easy and requir
12.
Instacart Does Zero-Downtime PostgreSQL Cutovers(tech.instacart.com)
19 points by imbradn 3y ago|3 comments
13.
by imbradn 8y ago
There is a History tab with examples, such as: https://explain.depesz.com/s/vzlS This is an indispensable tool for Postgres query plan analysis. Even for those who are experienced at reading them, this can help to sor