5 ms·

I've lost count of how many "Migrating from X to Postgres" articles I've seen. I don't think I've once seen a migrating away from Postgres article.

by etler 1y ago

betaby 1y ago
Here you go https://www.uber.com/en-CA/blog/postgres-to-mysql-migration/ https://www.uber.com/en-CA/blog/postgres-to-mysql-migration/
- psionides 1y ago
  Yeah so there's basically just that one ;)
  - hu3 1y ago
    https://engineeringblog.yelp.com/2024/10/migrating-from-postgres-to-mysql.html https://engineeringblog.yelp.com/2024/10/migrating-from-post...
    - wredcoll 1y ago
      You're not wrong, but this is the tldr: > The DB used is PostgreSQL which is not used anywhere else at Yelp, which meant that only a small rotation of long-tenured employees knew Postgres well enough to do outage response. This caused issues in maintenance, visibility, and outage response times. The teams working on the Restaurants products are not infra teams, and the Yelp-wide infra teams (understandably) focus on Yelp-standard infrastructure. As a result, when we did see issues with Postgres it was often a scramble to find people with relevant expertise. > So, we switched out this DB in-place with a Yelp-standard MySQL DB.
- speed_spread 1y ago
  It's a very Uber thing to do to enter a one way from the wrong end.
  - DonHopkins 1y ago
    At least they have the moxie to flip off people going the right way. https://www.youtube.com/watch?v=Tz6A6t55qXk https://www.youtube.com/watch?v=Tz6A6t55qXk
delish 1y ago
Related: Oxide's podcast, "Whither CockroachDB," which reflects on experience with postgres at Joyent, then the choice to use cockroach in response to prior experiences with postgres. https://www.youtube.com/watch?v=DNHMYp8M40k https://www.youtube.com/watch?v=DNHMYp8M40k I'm trying to avoid editorializing in my above summary, for fear of mischaracterizing their opinions or the current state of postgres. Their use of postgres was 10 years ago, they were using postgres for a high-availability use case -- so they (and I) don't think "postgres bad, cockroach good." But like Bryan Cantrill says, "No one cares about your workload like you do." So benchmark! Don't make technical decisions via "vibes!"
yen223 1y ago
I have participated in a Postgres -> Clickhouse migration, but I haven't bothered writing an article about it.
- I_am_tiberius 1y ago
  The entire database? Isn't that very limiting due to slow write speeds in Clickhouse? I saw ch more as a db for mainly read activities.
  - jacobsenscott 1y ago
    CH excels at extremely high volume writes. You probably can't throw enough data at it.
    - I_am_tiberius 1y ago
      Sorry, meant writes in terms of update/delete.
      - hodgesrm 1y ago
        There are many use cases that do not alter data. Observability is a canonical example.
      - nasretdinov 1y ago
        Many use cases can be addressed by using ReplacingMergeTree and such, so even when you need updates you can get them relatively cheaply
sa46 1y ago
I helped with the initial assessment for a migration from Postgres with Citus to SingleStore. https://www.singlestore.com/made-on/heap/ https://www.singlestore.com/made-on/heap/
notTooFarGone 1y ago
We migrated from postgres to ADX based on cost analysis done of the managed version on Azure. Now we have lovely kql queries and pretty much start new with postgres again...
rakejake 1y ago
Probably a corollary of the fact that most usecases can be served by an RDBMS running on a decently specced machine, or on different machines by sharding intelligently. The number of usecases for actual distributed DBs and transactions is probably not that high.
evanelias 1y ago
Not an article, and I have no direct knowledge of this either way, but I would strongly suspect that Instagram migrated off Postgres a while back. Probably to fb-mysql + myrocks, or some other RocksDB based solution. The compression level is vastly superior to any available Postgres-based solution, and at Instagram’s scale it amounts to extremely compelling hardware cost savings. Also if they were still primarily on pg, it would be one of the largest pg deployments in existence, and there would be obvious signs of the eng impact of that (conference talks, FOSS contributions, etc). Bigger-picture: Postgres is an amazing database, and it’s often the right choice, but nothing in tech is always the best choice 100% of the time. There’s always trade-offs somewhere.
dev_l1x_be 1y ago
Roughly the same count as migrating from Postgres to X.
yakkomajuri 1y ago
I think your point still stands, and I'm a big Postgres advocate/user myself btw. But yeah we did migrate our _analytics_ data to ClickHouse (while still keeping Postgres for more transactional stuff) back when I was at PostHog. Writeup: https://posthog.com/blog/how-we-turned-clickhouse-into-our-eventmansion https://posthog.com/blog/how-we-turned-clickhouse-into-our-e...
- mplanchard 1y ago
  We also did this, using change data capture and kafka to stream data to clickhouse as it gets written to postgres. Clickhouse is incredible tech. We’ve been very pleased with it for OLAP queries, and it’s taken a lot of load off the postgres instance, so it can more easily handle the very high write load it gets subjected to.

I've lost count of how many "Migrating from X to Postgres" articles I've seen. I don't think I've once seen a migrating away from Postgres article.

by etler 1y ago

I've lost count of how many "Migrating from X to Postgres" articles I've seen. I don't think I've once seen a migrating away from Postgres article.

betaby 1y ago

Here you go https://www.uber.com/en-CA/blog/postgres-to-mysql-migration/ https://www.uber.com/en-CA/blog/postgres-to-mysql-migration/

psionides 1y ago
Yeah so there's basically just that one ;)
- hu3 1y ago
  https://engineeringblog.yelp.com/2024/10/migrating-from-postgres-to-mysql.html https://engineeringblog.yelp.com/2024/10/migrating-from-post...
  - wredcoll 1y ago
    You're not wrong, but this is the tldr: > The DB used is PostgreSQL which is not used anywhere else at Yelp, which meant that only a small rotation of long-tenured employees knew Postgres well enough to do outage response. This caused issues in maintenance, visibility, and outage response times. The teams working on the Restaurants products are not infra teams, and the Yelp-wide infra teams (understandably) focus on Yelp-standard infrastructure. As a result, when we did see issues with Postgres it was often a scramble to find people with relevant expertise. > So, we switched out this DB in-place with a Yelp-standard MySQL DB.
speed_spread 1y ago
It's a very Uber thing to do to enter a one way from the wrong end.
- DonHopkins 1y ago
  At least they have the moxie to flip off people going the right way. https://www.youtube.com/watch?v=Tz6A6t55qXk https://www.youtube.com/watch?v=Tz6A6t55qXk

delish 1y ago

Related: Oxide's podcast, "Whither CockroachDB," which reflects on experience with postgres at Joyent, then the choice to use cockroach in response to prior experiences with postgres. https://www.youtube.com/watch?v=DNHMYp8M40k https://www.youtube.com/watch?v=DNHMYp8M40k I'm trying to avoid editorializing in my above summary, for fear of mischaracterizing their opinions or the current state of postgres. Their use of postgres was 10 years ago, they were using postgres for a high-availability use case -- so they (and I) don't think "postgres bad, cockroach good." But like Bryan Cantrill says, "No one cares about your workload like you do." So benchmark! Don't make technical decisions via "vibes!"

yen223 1y ago

I have participated in a Postgres -> Clickhouse migration, but I haven't bothered writing an article about it.

I_am_tiberius 1y ago
The entire database? Isn't that very limiting due to slow write speeds in Clickhouse? I saw ch more as a db for mainly read activities.
- jacobsenscott 1y ago
  CH excels at extremely high volume writes. You probably can't throw enough data at it.
  - I_am_tiberius 1y ago
    Sorry, meant writes in terms of update/delete.
    - hodgesrm 1y ago
      There are many use cases that do not alter data. Observability is a canonical example.
    - nasretdinov 1y ago
      Many use cases can be addressed by using ReplacingMergeTree and such, so even when you need updates you can get them relatively cheaply

sa46 1y ago

I helped with the initial assessment for a migration from Postgres with Citus to SingleStore. https://www.singlestore.com/made-on/heap/ https://www.singlestore.com/made-on/heap/

notTooFarGone 1y ago

We migrated from postgres to ADX based on cost analysis done of the managed version on Azure. Now we have lovely kql queries and pretty much start new with postgres again...

rakejake 1y ago

Probably a corollary of the fact that most usecases can be served by an RDBMS running on a decently specced machine, or on different machines by sharding intelligently. The number of usecases for actual distributed DBs and transactions is probably not that high.

evanelias 1y ago

Not an article, and I have no direct knowledge of this either way, but I would strongly suspect that Instagram migrated off Postgres a while back. Probably to fb-mysql + myrocks, or some other RocksDB based solution. The compression level is vastly superior to any available Postgres-based solution, and at Instagram’s scale it amounts to extremely compelling hardware cost savings. Also if they were still primarily on pg, it would be one of the largest pg deployments in existence, and there would be obvious signs of the eng impact of that (conference talks, FOSS contributions, etc). Bigger-picture: Postgres is an amazing database, and it’s often the right choice, but nothing in tech is always the best choice 100% of the time. There’s always trade-offs somewhere.

dev_l1x_be 1y ago

Roughly the same count as migrating from Postgres to X.

yakkomajuri 1y ago

I think your point still stands, and I'm a big Postgres advocate/user myself btw. But yeah we did migrate our _analytics_ data to ClickHouse (while still keeping Postgres for more transactional stuff) back when I was at PostHog. Writeup: https://posthog.com/blog/how-we-turned-clickhouse-into-our-eventmansion https://posthog.com/blog/how-we-turned-clickhouse-into-our-e...

mplanchard 1y ago
We also did this, using change data capture and kafka to stream data to clickhouse as it gets written to postgres. Clickhouse is incredible tech. We’ve been very pleased with it for OLAP queries, and it’s taken a lot of load off the postgres instance, so it can more easily handle the very high write load it gets subjected to.