KraftyOne

9 ms·

1.
Postgres SELECT DISTINCT does not scale(dbos.dev)
3 points by KraftyOne 19d ago|0 comments
2.
Postgres SELECT DISTINCT does not scale(dbos.dev)
3 points by KraftyOne 26d ago|0 comments
3.
Postgres SELECT DISTINCT does not scale(dbos.dev)
4 points by KraftyOne 1mo ago|0 comments
4.
Anyscale Joins NScale(anyscale.com)
3 points by KraftyOne 2mo ago|0 comments
5.
by KraftyOne 2mo ago
This is something we mention in the third section of the article: dead tuples and autovacuum caused real performance hits, which we (partially) mitigated through index optimization (minimizing the number and size of indexes, and making inde
6.
Making Postgres queues scale(dbos.dev)
128 points by KraftyOne 2mo ago|34 comments
7.
by KraftyOne 2mo ago
The core optimization is to buffer notifications in-memory and send them in a batch instead of sending them as part of every transaction. So that's a general-purpose optimization for Postgres apps using LISTEN/NOTIFY.
8.
by KraftyOne 2mo ago
If you mean the optimizations coming in Postgres 19, the original post addresses this: > As an aside, there’s been some online discussion of a Postgres patch ( https://github.com/postgres/postgres/commit/282
9.
by KraftyOne 2mo ago
To be clear, it's not a custom patch to pg itself, but an application-side buffering and batching optimization.
10.
Postgres LISTEN/NOTIFY actually scales(dbos.dev)
373 points by KraftyOne 2mo ago|84 comments
11.
Just Use Postgres for Job Queues(dbos.dev)
7 points by KraftyOne 2mo ago|0 comments
12.
by KraftyOne 3mo ago
Outbox's power is that it turns an atomicity problem into an idempotency problem. You atomically write to the outbox, then you have an idempotent "workflow" that processes events from the outbox. This turns "at most once
13.
by KraftyOne 3mo ago
You still need idempotency for side effects outside your database, that's true (and fundamental to durability). But now you get exactly-once semantics for operations on your database, which can be quite valuable if your workflow perfor
14.
by KraftyOne 3mo ago
That's what the post is about! Once you're doing that, you really do have transactions between the state and the queue.
15.
by KraftyOne 3mo ago
Every item will be written to the queue exactly once (as the update is transactional). Queue processing may need at-least-once semantics, yes, depending on what exactly you're doing.
16.
by KraftyOne 3mo ago
You build a distributed system on top of this! For example, you may have many distributed workers durably executing workflows from the Postgres-backed task queue. The Postgres transactions allow you to atomically perform operations spanning
17.
by KraftyOne 3mo ago
Yes, the core design is building a workflow system on a database--essentially, replacing the central orchestrator most workflow systems use with a Postgres database. This previous blog post goes into more detail: https://www.dbos
18.
by KraftyOne 3mo ago
The key is that the UDF's enqueue is transactional with the database update. Let's say the database update is inserting a new order. This provides the guarantee that if a new order is inserted, a job to process the order is also e
19.
by KraftyOne 3mo ago
Exactly! It's a distributed system, with many processes performing work in parallel, with a central database as a coordination point, used as little as possible. A mutex wouldn't get quite the same performance :)
20.
Postgres transactions are a distributed systems superpower(dbos.dev)
232 points by KraftyOne 3mo ago|96 comments
21.
by KraftyOne 3mo ago
Unfortunately, DSQL is still lacking many core Postgres features (most notably foreign keys) that DBOS and DBOSify depend on. DBOS works with most flavors of Postgres though, including regular Aurora/RDS, Cloud SQL, AlloyDB, Supabase,
22.
by KraftyOne 3mo ago
Take any Temporal example and swap the imports and connection strings, it's a drop-in replacement :)
23.
by KraftyOne 3mo ago
We'd need DBOS in C# first! It's something we're considering for the future--if you're working in C#, happy to chat about it, reach out at peter.kraft@dbos.dev.
24.
by KraftyOne 3mo ago
Exactly, Postgres as an orchestrator instead of a Temporal server as an orchestrator. This page goes into more detail (for DBOS, but DBOSify is the same principle but using the Temporal API directly): https://docs.dbos.dev/e
25.
by KraftyOne 3mo ago
I'm very sorry about this. I'll figure out why the form wasn't responded to. In the meantime, email me directly at peter.kraft@dbos.dev.
26.
by KraftyOne 3mo ago
Yes it can (with documented exceptions)! The suite is here: https://github.com/dbos-inc/dbosify-py/tree/main/tests/confo... Here's the documentation: https://github.com/dbos-in
27.
Just Use Postgres for Task Queues(dbos.dev)
2 points by KraftyOne 3mo ago|0 comments
28.
Is this blog written by AI?(brooker.co.za)
5 points by KraftyOne 3mo ago|0 comments
29.
by KraftyOne 3mo ago
Hi HN, I'm Peter, creator of DBOSify. Here to answer any questions you have!
30.
Show HN: DBOSify – Drop-in Temporal replacement built on Postgres(github.com)
91 points by KraftyOne 3mo ago|19 comments

More ›