6 ms·

I think queues are the wrong abstraction to model business processes. That's why a trivial issue like a non recoverable failure during processing a message beco

I think queues are the wrong abstraction to model business processes. That's why a trivial issue like a non recoverable failure during processing a message becomes such a headache. The same goes for ordering. An orchestrator like temporal.io allows modeling your business use case using higher level abstractions that hide all this low level complexity. Disclaimer: I'm the tech lead of the temporal.io open source project and the CEO of the affiliated company.

cgio 5y ago
Hiding this complexity is useful if it also means handling it. What are the key patterns you apply in temporal to hide it? I’ve had a look at temporal and find it really interesting.
- bergundy 5y ago
  Instead of directly using queues in a Temporal Workflow, the Workflow (which is written with plain code), schedules an Activity that the system is responsible for, behind the scenes, the Activity is just an item put on a queue. Activities have retry policies which are also handled by the system. If an Activity attempt fails and should not retry according to the policy, an exception is thrown in the Workflow to be handled using code. Using the TypeScript SDK, you can catch that exception here: https://github.com/temporalio/samples-typescript/blob/9d910866f403ef4bd83e5091d623f1d692ce7d11/activities-examples/src/workflows.ts#L16 https://github.com/temporalio/samples-typescript/blob/9d9108...
lmilcin 5y ago
It is a problem only if you are mixing up application layers. If you keep your queueing system and business process as separate layers with queueing system serving only as a means of transporting business events then you can make it all to work correctly. Think in terms of IP protocol (as in TCP/IP). It is unsuitable for transmitting financial transactions. Yet, financial transactions can be made to work on top of it if you separate the layers and treat IP only as a component mechanism of getting data from A to B.
- mfateev 5y ago
  I think we are in agreement here. Temporal does exactly what you described. It uses queues to transporting tasks to processes. But it completely hides them from the business process code. The issue is that 99.9% of developers use queues directly in their business applications.

cgio 5y ago

Hiding this complexity is useful if it also means handling it. What are the key patterns you apply in temporal to hide it? I’ve had a look at temporal and find it really interesting.

bergundy 5y ago
Instead of directly using queues in a Temporal Workflow, the Workflow (which is written with plain code), schedules an Activity that the system is responsible for, behind the scenes, the Activity is just an item put on a queue. Activities have retry policies which are also handled by the system. If an Activity attempt fails and should not retry according to the policy, an exception is thrown in the Workflow to be handled using code. Using the TypeScript SDK, you can catch that exception here: https://github.com/temporalio/samples-typescript/blob/9d910866f403ef4bd83e5091d623f1d692ce7d11/activities-examples/src/workflows.ts#L16 https://github.com/temporalio/samples-typescript/blob/9d9108...

lmilcin 5y ago

It is a problem only if you are mixing up application layers. If you keep your queueing system and business process as separate layers with queueing system serving only as a means of transporting business events then you can make it all to work correctly. Think in terms of IP protocol (as in TCP/IP). It is unsuitable for transmitting financial transactions. Yet, financial transactions can be made to work on top of it if you separate the layers and treat IP only as a component mechanism of getting data from A to B.

mfateev 5y ago
I think we are in agreement here. Temporal does exactly what you described. It uses queues to transporting tasks to processes. But it completely hides them from the business process code. The issue is that 99.9% of developers use queues directly in their business applications.