lorendsr

10 ms·

1.
by lorendsr 3y ago
Temporal is designed to handle lower latency use cases than data pipeline systems like Airflow. It also has added a feature recently called Update designed for request-response style interactions that allows for communication with Workflows
2.
by lorendsr 3y ago
There are certainly use cases for which it's more than is required. Like the most simple would be adding a cron string to a GitHub Action or Vercel function, but in most cases, and certainly Slack's case, you want more reliability
3.
by lorendsr 3y ago
I wrote up this comparison: https://community.temporal.io/t/what-are-the-pros-and-cons-o...
4.
by lorendsr 3y ago
Using Temporal is in the category of building it yourselves, not a billing service. But it makes it take much less time to build, because it makes the changes, scaling, and grandfathering quick to do. Especially with the new Scheduled workf
5.
by lorendsr 3y ago
Here's a Temporal v Prefect comparison I wrote: https://community.temporal.io/t/what-are-the-pros-and-cons-o... tldr is Temporal is more general-purpose: for reliable programming in general, vs data pipelines. It
6.
by lorendsr 3y ago
Thanks! You do need to run it on the same code version. There are different ways deploy code changes. If you use one of our in-built versioning systems, the version is recorded in the workflow history (and you can either keep track of versi
7.
by lorendsr 3y ago
Author here, curious if there are any major TTD debuggers I missed? Also let me know if anything in the post didn't make sense, and I'll edit to clarify!
8.
Time-Travel Debugging Production Code(temporal.io)
9 points by lorendsr 3y ago|4 comments
9.
by lorendsr 3y ago
2PC tends to have limited throughput due to the participants needing to hold a lock between the voting and commit phase, and all the participants need to support the protocol. Sagas work across different services and data stores and can hav
10.
by lorendsr 3y ago
At some point, if you can't automatically fix something, you have to stop and report to a human for manual intervention/repair. While a saga doesn't guarantee that you avoid manual repair, it significantly reduces the need fo
11.
by lorendsr 3y ago
If I'm getting your point right, I agree! If you have the workflow / durable execution primitive to depend on (a durable function is guaranteed to complete executing), then there are a lot of pieces of distributed systems stacks
12.
by lorendsr 3y ago
Sagas are for when you can't do an update in an ACID transaction, for example when updating state across different types of data stores. If you're asking whether the catch clause in a Temporal Workflow saga is guaranteed to execut
13.
by lorendsr 3y ago
There are definitely ease of use benefits to more tailored solutions. If workflow definitions are really simple and don't change much, JSON might be easy. Most things I prefer the DX of writing the logic in code. And it wouldn't b
14.
by lorendsr 3y ago
Temporal has different database options: Cassandra, Postgres, MySQL, SQLite. > source data in CSV files; - Capillaries script (JSON file) that defines the workflow and the transforms; - Python code that performs complex calculations (onl
15.
by lorendsr 3y ago
Temporal also shares the principle of being tolerant to database and processing node failures
16.
by lorendsr 3y ago
Your code is run in a worker process. If the worker fails, another worker will pick up where the first left off. The state of each workflow function is stored by the Temporal Cluster. The Cluster is a SPOF, but each component of it is fault
17.
by lorendsr 3y ago
ETL tools target that specific use case, versus Temporal workflows are much more general purpose—for any backend code you want to run reliably. I wrote some more about this here: https://community.temporal.io/t/what-are
18.
by lorendsr 3y ago
The major differences between an event bus and a workflow engine is the workflow engine: - Creates and updates events/messages for you - Maintains consistency between the events and timers and the state of the processing flow. More inf
19.
by lorendsr 3y ago
Traditional workflow systems are often used for specific types of business processes, particularly those that are async like human-in-the-loop. Temporal, while it uses the "workflow" terminology, is a new type of thing. At a basic
20.
by lorendsr 3y ago
Temporal (and similar systems like Cadence, AWS SWF, Azure Durable Functions) allows you more expressiveness and better DX than defining DAGs in a UI or markup file. You can write (almost) arbitrary code, and the library translates the code
21.
by lorendsr 3y ago
Correct, the workflow's guarantee to always complete executing independent of process/hardware failures is dependent on the database not losing data. You host your workflow code with Temporal's Worker library, which talks to
22.
by lorendsr 3y ago
After you deploy the new workflow code, you can reset [1] a workflow's execution state to before the DelayAsync statement was called, and then the workflow will sleep for 7 days from now. That doesn’t take into account the time it’s al
23.
by lorendsr 3y ago
For those not familiar with workflows as code, a workflow is a method that is executed in a way that can't fail—each step the program takes is persisted, so that if execution is interrupted (the process crashes or machine loses power),
24.
by lorendsr 4y ago
It's a way of writing code that can't fail to execute, even if services are down or the process dies. Here's more info and a small example: https://temporal.io/blog/building-reliable-distributed-syste...
25.
by lorendsr 4y ago
You could call them both workflow management systems. One of the differences is Temporal/Cadence uses code to define workflows instead of YAML. It's a large enough difference that Temporal is defining their category as "durab
26.
by lorendsr 4y ago
The simplest way to self-host is with Render: https://render.com/docs/deploy-temporal Scaling takes a lot of expertise, and I'd say almost everyone saves time & money and gains in reliability by using Temporal
27.
by lorendsr 4y ago
All the durable execution systems have to run your code in certain way that persists steps like RPCs (and need to provide a mechanism for you to tell the system which functions have RPCs) so they can recover in case of process failures. The
28.
by lorendsr 4y ago
The point is that you can write code instead of JSON/YAML like traditional microservice orchestration like AWS step functions. And it’s not a limited dsl—you have the full lang at your disposal, with the one requirement that determinis
29.
by lorendsr 4y ago
We have Go and Java SDKs that have better performance characteristics if that’s what you’re optimizing for. I think for many businesses, optimizing for development speed is a higher priority (eg if the devs already know JS, use that). The N
30.
by lorendsr 4y ago
Yeah, I’ve also written this write-to-db-after-each-meaningful-line-of-code style code, and this is a great improvement. See the first 20m of this talk for an example: https://youtu.be/EFIF8gk9zy8 Starting a workflow is cur

More ›