Every long-running business process eventually becomes the same artifact: a status column, a cron job that scans for stuck rows, a pile of compensating updates, and one engineer who understands the state machine. It works, in the sense that a car with no brakes works if you plan the route carefully.
Durable execution attacks that directly. You write the process as ordinary sequential code — call this service, wait three days, call that one, compensate if it fails — and the engine persists every step so the function can resume on another machine after a crash, a deploy, or a week of waiting. The state machine stops being something you maintain and becomes something the runtime derives.
What it actually guarantees
The guarantee is worth stating precisely, because it is narrower than the marketing and more useful than the skeptics assume: the workflow's progress is durable, and each activity is retried until it succeeds or you give up. It is not exactly-once execution of side effects. An activity can run twice — the process can die after the payment call succeeded but before the result was recorded — so activities still need to be idempotent. Durable execution removes the bookkeeping, not the need for idempotency keys.
@workflow.defn
class OnboardTenant:
@workflow.run
async def run(self, tenant: TenantSpec) -> str:
account = await workflow.execute_activity(
provision_account, tenant,
start_to_close_timeout=timedelta(minutes=5),
retry_policy=RetryPolicy(maximum_attempts=5),
)
try:
await workflow.execute_activity(seed_reference_data, account, ...)
# Days can pass here. The process is free to die; the workflow is not.
await workflow.wait_condition(lambda: self.contract_signed, timeout=timedelta(days=14))
except Exception:
await workflow.execute_activity(deprovision_account, account, ...)
raise
return account.id
@workflow.signal
def contract_signed_signal(self) -> None:
self.contract_signed = TrueThat fourteen-day wait is the part that changes how you design. No cron scan, no pending table, no reconciliation job. The wait is a line of code, and a signal from a webhook resumes it.
Determinism is the tax
Resumption works by replaying the workflow function against a recorded history, so the function must be deterministic: no wall-clock reads, no random values, no network calls, no iteration over an unordered set, all of it delegated to activities or the engine's own APIs. This is learnable in an afternoon and forgettable in a sprint, which is why the failure mode is real.
- Everything non-deterministic goes in an activity. The workflow function only orchestrates.
- Version any change to workflow structure, or in-flight executions will fail replay after deploy.
- Keep workflow state small; history size is a real limit and large payloads belong behind a reference.
- Activities still need idempotency keys. Durable execution is not exactly-once side effects.
- Test replay in CI against recorded histories — it catches determinism breaks before they reach running workflows.
When a queue is still the right answer
Durable execution is not a replacement for queues, and reaching for it reflexively adds a stateful cluster to your architecture for no reason. If the work is one hop — resize this image, send this email, reindex this record — a queue with retries and a dead-letter is simpler, cheaper and easier to operate. The value appears when the process has multiple steps, spans systems you do not control, needs compensation, or waits on time or human input.
The question is not 'queue or workflow engine'. It is whether the process has a memory. If it does, something has to hold it, and a status column is the worst available option.
The last argument in its favour is operational. When a multi-step process fails at three in the morning, the difference between reading an execution history that shows exactly which activity failed, with its inputs, and grepping logs across four services to reconstruct the same story, is most of the incident. I have done both. Only one of them is a job I would volunteer for.