Skip to content

Failed-Attempt Recovery Workflow

Recovery workflow — instantiates Endpoint Fan-Out Fulfillment

Turns a failed endpoint attempt into a classified, routed recovery — diagnosing why it failed and sending it to correction, an alternate mode, a reschedule, or escalation — so one miss doesn't become a permanent non-completion.

In a silent system, a first-attempt failure at the endpoint just... disappears — the dashboard reads "dispatched," and the miss becomes a permanent non-completion no one sees. Failed-Attempt Recovery Workflow closes that loop. It catches each attempted-and-failed case, diagnoses why it failed, and routes it down the matching recovery path: correct the data, switch to an alternate mode, reschedule, or escalate. Its defining feature is being the post-failure closed loop — it acts only on things that were tried and missed, and it recovers them without dumping the cost of the failure back onto the recipient. That distinguishes it from verifying prerequisites before an attempt, or from holding cases that never fit a standard path at all.

Example

A social-payments agency disburses monthly cash benefits to millions of recipients by bank transfer. A slice bounce — closed account, wrong number, name mismatch, dormant account — and without recovery that money simply doesn't arrive while the ledger still reads "disbursed." A Failed-Attempt Recovery Workflow catches each bounce, classifies the cause, and routes accordingly: bad account details go to an update channel that reaches the recipient to correct them; a bank rejection retries via an alternate rail such as mobile money or a cash-pickup voucher; a dormant account gets a reschedule with a reactivation nudge. Crucially, the agency initiates the fix and offers an alternate mode rather than making the recipient restart the whole application. Most bounced payments then land within the same cycle instead of vanishing, and the residual truly-unreachable cases escalate for human investigation.

How it works

The distinguishing logic is classify the failure, then route to a matched recovery path — idempotently:

  • Diagnose before retrying. The workflow first classifies why the attempt failed, because a bad address, a capacity miss, and a recipient-side block need different recoveries — blind retry wastes effort on causes it can't fix.
  • Route down a recovery ladder. Each cause maps to a path: correct-and-retry, alternate mode, reschedule, or escalate to a human — a genuine second route to completion, not a repeat of the failed one.
  • Recover without re-burdening. The system carries the recovery — initiating contact, offering fallback modes — so the endpoint isn't made to bear the cost of a failure it didn't cause.

Tuning parameters

  • Failure-taxonomy granularity — how finely failure causes are classified. Finer routing recovers more precisely but costs more to build and maintain.
  • Retry policy — how many retries, how spaced, and whether to back off or switch modes. Aggressive retry recovers more but risks duplicate service and recipient annoyance.
  • Alternate-mode ladder — which fallback modes are offered and in what order (retry same rail → alternate rail → manual). More rungs recover more but add complexity.
  • Idempotency guard — how strongly duplicate completion is prevented on retry; the dial that protects against double-serving.
  • Escalation threshold — when a case stops auto-recovering and goes to human investigation, so it can't churn forever.

When it helps, and when it misleads

Its strength is closing the loop that silent systems leave open — converting a first-attempt miss into eventual completion by the right path, and doing so without making the recipient pay for the system's failure. It is what stops "dispatched" from masking a permanent non-completion. Its failure mode is sharp: a retry loop without an idempotency guard can double-serve — pay twice, deliver twice — and one without an escalation cap can churn forever on a case that will never complete, burning cost and quietly re-burdening the recipient with repeated failed contacts. The classic misuse is leaning on endless automated retries as a substitute for fixing a systematic upstream failure, or counting "recovery attempted" as "recovered." The discipline is to make recovery idempotent so retries can't double-serve, cap retries with escalation to human judgment, bound the recipient's burden explicitly, and feed recurring failure causes back upstream rather than absorbing them forever.[1]

How it implements the components

Failed-Attempt Recovery Workflow realizes the post-failure recovery side of the archetype — the components a recovery loop fills, not the pre-dispatch, batching, or measurement machinery:

  • endpoint_recovery_channel — the workflow is the recovery channel: it supplies the classified paths — correction, alternate mode, reschedule, escalation — by which a failed attempt gets a second route to completion.
  • endpoint_burden_guardrail — it is built so recovery doesn't fall on the recipient: the system initiates the fix, offers alternate modes, and bounds how much the endpoint is asked to do rather than making it restart.

It does not verify prerequisites *before an attempt to prevent the failure (trunk_to_endpoint_boundary) — that is Address or Endpoint Validation; nor does it hold cases that never fit a standard path (local_fit_exception_process), which is Exception Queue.*

Notes

The non-obvious dependency is idempotency. Because recovery retries an operation that may in fact have partly succeeded (the first delivery was made but the confirmation was lost), a recovery workflow bolted on without a duplicate-completion guard will double-serve under retry — the failure mode is silent and expensive. Design the completion operation to be idempotent before scaling retries, not after the first double-payment.

References

[1] In distributed messaging, at-least-once delivery guarantees a message is retried until acknowledged, which means retries can duplicate; the standard remedy is idempotency — designing the operation so that repeating it has no additional effect. A recovery workflow needs the same guard so retries cannot double-serve the endpoint.