8 ms·
Why not use postgres listen/notify instead of rabbitmq pub sub.
by tzahifadida 3y ago
Why not use postgres listen/notify instead of rabbitmq pub sub.
- anentropic 3y agoIt uses Postgres rather than RabbitMQ: https://github.com/hatchet-dev/hatchet?tab=readme-ov-file#how-does-this-compare-to-alternatives-celery-bullmq https://github.com/hatchet-dev/hatchet?tab=readme-ov-file#ho...
- anentropic 3y agoI see... apparently it uses both
- abelanger 3y agoWhen I started on this codebase, we needed to implement some custom exchange logic that maps very neatly to fanout exchanges and non-durable queues in RabbitMQ and weren't built out on our PostgreSQL layer yet. This was a bootstrapping problem. Like I mentioned in the comment, we'd like to switch to pub/sub pattern that lets us distribute our engine over multiple geographies. Listen/notify could be the answer once we migrate to PG 16, though there are some concerns around connection poolers like pg_bouncer having limited support for listen/notify. There's a Github discussion on this if you're curious: https://github.com/hatchet-dev/hatchet/discussions/224 https://github.com/hatchet-dev/hatchet/discussions/224.
- tzahifadida 3y agoI use haproxy with go listen notify of one of the libs. It works as long as the connection is up. I.e.i have a timeout of 30 min configured in haproxy. Then you have to assume you lost sync and recheck. That is not that bad every 30min... at least for me. You can configure to never close...