hmaxdml

8 ms·

1.
GitHub Is Down Again(githubstatus.com)
9 points by hmaxdml 27d ago|1 comments
2.
by hmaxdml 1mo ago
I thought that fell under BYOD (device) :)
3.
by hmaxdml 2mo ago
What's reasonable? DBOS has users running queues at millions of tasks per hour. For fair queuing you can have partitioned queues where only active partitions consume resources
4.
by hmaxdml 2mo ago
Not saying this in a mean way: It seems to me this can be boiled down to "using things without understanding how they work doesn't scale". Yes, vanilla listen/notify doesn't scale. But the OP actually figured out ho
5.
by hmaxdml 3mo ago
The database is exactly the hardcore piece of engineering that's been designed to scale and be fault tolerant for decades
6.
by hmaxdml 3mo ago
Because you likely already have a database and likely don't need to bring on an entire new distributed system to orchestrate your workflows.
7.
by hmaxdml 3mo ago
A PG-backed queue is in code right after being in PG, and the beauty of a neat durable queue framework is in exposing it conveniently and efficiently.
8.
by hmaxdml 4mo ago
Postgres does scale pretty well: https://www.dbos.dev/blog/benchmarking-workflow-execution-sc... Tens of thousands of workflows per second
9.
by hmaxdml 4mo ago
That's why their entire business model -- like Astronomer's -- is geared toward cloud hosting. The architecture is so complex it takes a full time team to run it.
10.
by hmaxdml 4mo ago
Have you looked into DBOS? Same thesis: durable and reliable workflows are hard to manage -- it just doesn't have to be as hard as Temporal makes it be :)
11.
by hmaxdml 4mo ago
DBOS python supports SQLite. Go is supporting it next release
12.
by hmaxdml 4mo ago
I've talked to dozens of engineers who built their home grown "durable" stack. Most of them eventually moved on to buying vs building, when their system actually scaled. It's just not a side-hustle to build a foundationa
13.
by hmaxdml 4mo ago
Yeah, we've observed that too: people start implementing their own retry logic, idempotency, etc. But then they grow a hard to maintain, complex stack that's not their core business logic. There's a reason why there is a dedi
14.
by hmaxdml 4mo ago
Listen/notify is poised to become much better in PG 18 and 19
15.
by hmaxdml 4mo ago
As you said, the example is simple and it might not be obvious to people without prod experience what the problems can be. Postgres can give you all the primitives you need to solve this at the application layer. Durable workflows on Postgr
16.
by hmaxdml 4mo ago
We've found that durable workflows is a much needed primitive for agents control flow. They give a structure for deterministic replays, observability, and, of course, fault tolerance, that operators need to make the agent loop reliable
17.
Stonebraker on Postgres, Disagreeing with Google, and Future Problems [video](youtube.com)
1 points by hmaxdml 5mo ago|0 comments
18.
by hmaxdml 7mo ago
Automatic crash detection for your process is built-in our Conductor offering. The library has a default recovery mode when used standalone. What do you find strange with workflow versioning? Would love to consider improving the semantics.
19.
by hmaxdml 7mo ago
I notice you didn't provide any specific comparison alongside that comment, which makes me feel frustrated because I think the Temporal workflow SDK is very different. Architecturally, Temporal and DBOS are at two opposites of the dura
20.
by hmaxdml 7mo ago
Thanks for the comment (author here). I wanted this post to focus on the Golang specific implementation, not dwell on the durable execution ecosystem at large. With respect to context, I don't know that anyone invented "having the
21.
by hmaxdml 7mo ago
Conductor is about enterprise features like automatic workflow recovery, alerting, or RBAC. The GUI is a nice to have -- but all your workflow data are in Postgres. You can access it very easily.
22.
Go-Native Durable Execution(dbos.dev)
56 points by hmaxdml 7mo ago|19 comments
23.
by hmaxdml 8mo ago
Durable execution has already been mentioned as the existing solution for this problem, but I would like to call out a specific pattern that DE makes obsolete: the outbox pattern. Imagine just being able to do do send a() send b() And know
24.
by hmaxdml 8mo ago
These are all important concerns, but I'd go for an off the shelf library that does it for me (disclaimer I work at https://github.com/dbos-inc )
25.
Go-Native Durable Execution(dbos.dev)
2 points by hmaxdml 8mo ago|0 comments
26.
by hmaxdml 10mo ago
The hype is because DE is such an dev exp improvement over building your own queue. Good DE frameworks come with workflows, pub/sub, notifications, distributed queues with tons of flow control options, etc.
27.
by hmaxdml 10mo ago
one way is to follow https://www.linkedin.com/company/dbos-inc for updates
28.
by hmaxdml 1y ago
Good call. We'll see how to integrate it in our docs better. The cost of DBOS durable execution is 1 write per step (checkpoint the outcome) and 2 additional writes per workflows (upsert the workflow status, checkpoint the outcome). Th
29.
by hmaxdml 1y ago
DBOS stores all the workflow metadata in postgres, which is readily queryable for observability. We've recently seen a user setup an entire Grafana dashboard to observe their numerous workflows. A postgres server can host many database
30.
by hmaxdml 1y ago
We decided to use Postgres because of the relational semantics, the ease of integration with user applications, and it's remarkable popularity

More ›