6 ms·

I was using TimescaleDB some very long time ago, things have changed quite a lot since (it's now even named differently). In my current setup I was thinking on

by k_bx 3mo ago

I was using TimescaleDB some very long time ago, things have changed quite a lot since (it's now even named differently). In my current setup I was thinking on doing both: upgrading postgresql to timescaledb (to archive old data etc.), and to deploy ClickHouse in parallel. I'm still considering whether to go big on PeerDB to get ClickHouse mirror or just deploy it separately without additional fragility layer. Would you not recommend using timescaledb at all? I definitely want to avoid alpha-quality software pain, since PostgreSQL is one of the most rock-solid parts of the stack at the moment.

__s 3mo ago
Worked on peerdb. If you're able to batch changes on your end & push to both postgres & clickhouse, do that. Only move to peerdb when you know you need cdc
- fsuts 3mo ago
  Just looked up PeerDB expecting a Db as per its name. But it’s a ETl tool. Stupid naming
  - saisrirampur 3mo ago
    I know I know. Some people have loved it as it captures what it does (peering dbs) and some haven't because of the exact reason you called out. So we get it! :)
himata4113 3mo ago
I would just run both and decomission the old one when a) all data is migrated, b) old data is no longer relevant and can be archived
wkrp 3mo ago
In my (minor) experience Timescale works fine. The developer experience is good and it is very convenient to be able to JOIN against your hypertables. My only real complaints are operational (no logical replication, normal postgres update complaints), but man Clickhouse is really slick. I wrote some small reviews of the two in my submission history if you want a bit more detail.
saisrirampur 3mo ago
We at PeerDB/CickPipes tried to make that mirroring as frictionless as possible. It is validated at scale across 1000s of customer moving over half a PB of a data per month from Postgres to ClickHouse. You should give it a shot and you might be surprised. Side note: May be there is way even more native than CDC/logical decoding that you never have to worry about keeping PG and CH in sync. Stay tuned for some updates from us on that end! - Sai from ClickHouse

by k_bx 3mo ago

__s 3mo ago

Worked on peerdb. If you're able to batch changes on your end & push to both postgres & clickhouse, do that. Only move to peerdb when you know you need cdc

fsuts 3mo ago
Just looked up PeerDB expecting a Db as per its name. But it’s a ETl tool. Stupid naming
- saisrirampur 3mo ago
  I know I know. Some people have loved it as it captures what it does (peering dbs) and some haven't because of the exact reason you called out. So we get it! :)

himata4113 3mo ago

I would just run both and decomission the old one when a) all data is migrated, b) old data is no longer relevant and can be archived

wkrp 3mo ago

In my (minor) experience Timescale works fine. The developer experience is good and it is very convenient to be able to JOIN against your hypertables. My only real complaints are operational (no logical replication, normal postgres update complaints), but man Clickhouse is really slick. I wrote some small reviews of the two in my submission history if you want a bit more detail.

saisrirampur 3mo ago

We at PeerDB/CickPipes tried to make that mirroring as frictionless as possible. It is validated at scale across 1000s of customer moving over half a PB of a data per month from Postgres to ClickHouse. You should give it a shot and you might be surprised. Side note: May be there is way even more native than CDC/logical decoding that you never have to worry about keeping PG and CH in sync. Stay tuned for some updates from us on that end! - Sai from ClickHouse