6 ms·

I aggree - I'm not understanding the value of the project either if you look at the example here https://github.com/microsoft/pg_durable/blob/main/examples/invo

I aggree - I'm not understanding the value of the project either if you look at the example here https://github.com/microsoft/pg_durable/blob/main/examples/invoice-approval/without-df.md https://github.com/microsoft/pg_durable/blob/main/examples/i... It's an interesting technical achievement I guess, but it's very bizarre to try and read this SELECT df.start( @> ( ($$SELECT ... FROM demo.invoices WHERE status = 'pending'$$ |=> 'inv') ~> df.if_rows('inv', $$UPDATE ... SET status = 'processing'$$ ~> (df.http(...) |=> 'resp') ~> df.if($$SELECT $r.ok$$, -- classify, branch, wait for signal ... ), df.sleep(5) ) ), 'invoice-approval-pipeline' );

gdecandia 3mo ago
Contributor here - at Microsoft we've built AI workflows on pg_durable and seen it substantially reduce code and increase reliability. Agree that the DSL ergonomics can be improved. Our pipelines use a higher level language and therefore simplified, but pg_durable is meant to solve a wider array of problems. We're happy to take suggestions for improvements.
- faxmeyourcode 3mo ago
  Somebody else in the thread brought up the benefit of snapshotting a database at a point in time stores not only the state of execution but also the code, etc. That is a unique benefit I'd be interested in exploring over storing your orchestration outside of the database. Not trying to dismiss the project - it looks like a lot of hard work has gone in and somebody has a use for it. I just come from an airflow style external orchestrator frame of mind that manages durability state in postgres but keeps the control flow out. Sorry if I came off as a bit snarky
- Onavo 3mo ago
  Do you plan to open source the high level wrapper too?
rswail 3mo ago
Without reading any of the doco, it appears to be a job definition called invoice-approval-pipeline that runs every 5 seconds. The steps are: 1. Get all the pending invoices 2. Set their state to "processing" 3. Call out to an external service/process to do the actual processing, wait for a response. 4. If the response is OK, do something 5. Wait 5 seconds and then start again. Not sure I love the syntax and the way SQL is embedded between the $$ But it is in the database, can be updated and modified in the same way as all the other stored procedures/functions, allows job control, I assume other control structures for parallel steps etc. Gonna go read the doco now.
- franckpachot 3mo ago
  "dollar quoting" is the PostgreSQL way to quote strings with quotes, avoiding double quoting or escape characters. I like to use the tagged version of it, like $sql$ SELECT ... $sql$ to describe what is inside.
miohtama 3mo ago
Before this I thought it was impossible to surpass Perl.

gdecandia 3mo ago

Contributor here - at Microsoft we've built AI workflows on pg_durable and seen it substantially reduce code and increase reliability. Agree that the DSL ergonomics can be improved. Our pipelines use a higher level language and therefore simplified, but pg_durable is meant to solve a wider array of problems. We're happy to take suggestions for improvements.

faxmeyourcode 3mo ago
Somebody else in the thread brought up the benefit of snapshotting a database at a point in time stores not only the state of execution but also the code, etc. That is a unique benefit I'd be interested in exploring over storing your orchestration outside of the database. Not trying to dismiss the project - it looks like a lot of hard work has gone in and somebody has a use for it. I just come from an airflow style external orchestrator frame of mind that manages durability state in postgres but keeps the control flow out. Sorry if I came off as a bit snarky
Onavo 3mo ago
Do you plan to open source the high level wrapper too?

rswail 3mo ago

Without reading any of the doco, it appears to be a job definition called invoice-approval-pipeline that runs every 5 seconds. The steps are: 1. Get all the pending invoices 2. Set their state to "processing" 3. Call out to an external service/process to do the actual processing, wait for a response. 4. If the response is OK, do something 5. Wait 5 seconds and then start again. Not sure I love the syntax and the way SQL is embedded between the $$ But it is in the database, can be updated and modified in the same way as all the other stored procedures/functions, allows job control, I assume other control structures for parallel steps etc. Gonna go read the doco now.

franckpachot 3mo ago
"dollar quoting" is the PostgreSQL way to quote strings with quotes, avoiding double quoting or escape characters. I like to use the tagged version of it, like $sql$ SELECT ... $sql$ to describe what is inside.

miohtama 3mo ago

Before this I thought it was impossible to surpass Perl.