Liveness Watchdog¶
Monitor — instantiates Progress-Guarded Livelock Disruption
Arms a deadline against progress and, when the deadline passes with none, forces a reset to a known-good checkpoint before the stall becomes permanent.
Liveness Watchdog holds one predicate — the joint state must strictly progress within each window — and arms a deadline against it. It watches a progress signal (typically a Progress Counter Heartbeat); if that signal fails to advance before the window expires, it declares the system stuck and forces a corrective action: reset the actors to a known-good checkpoint and re-enter, or escalate. The one idea that makes it this mechanism is that it is deliberately ignorant of why — it builds no model of the cycle and understands nothing about the contention. It only knows that nothing progressed in time, and that is enough to act. It is the archetype's dead-man's-switch: the guarantee that something happens when the joint system quietly stops moving.
Example¶
A satellite's attitude-control firmware runs a control loop that must commit a real orientation update every cycle. Now and then two software tasks fall into a mutual-preemption livelock — each keeps yielding to the other's higher-priority interrupt, so the loop runs but never lands an update, and the craft drifts while the CPU sits at 100%. A hardware watchdog timer must be "kicked" every ≈200 ms, but the firmware is wired to kick it only after a committed update, not merely on entering the loop. During the livelock the update never lands, the kick never comes, the timer expires — and the watchdog forces a reset into a safe checkpoint mode, breaking the livelock and re-entering a clean control cycle. The drift is bounded to a single timeout window instead of running open-ended, and the craft self-recovers with no ground intervention.
How it works¶
Two dials and a trigger. A deadline sets how long the progress signal may stay flat before the system is declared stuck. A kick condition governs what resets that deadline — and the whole value hinges on demanding a committed progress step, not mere activity, so a busy livelock cannot keep the timer alive. On expiry the watchdog runs a bounded corrective: reset to the last known-good checkpoint and re-enter, and if resets keep firing, hand off. What distinguishes it from its detector-siblings is that its detection is black-box and timeout-based — no state graph, no cycle signature, just "too long without progress, act now."
Tuning parameters¶
- Timeout length — how long a flat signal is tolerated before firing. Short timeouts catch a stall fast but risk false trips on legitimately slow-but-progressing work; long timeouts are safe but let a livelock burn.
- Kick condition — what the watched actor must do to reset the timer: any activity (weak — a livelock can fool it) versus a committed progress step (strong — it then detects true non-progress, not mere silence).
- Action on expiry — reset-to-checkpoint, a targeted kick of one actor, or escalation to a human or arbiter. Harder actions recover more reliably at more disruption.
- Reset budget before escalation — how many automatic resets to attempt before handing off. Too many and it loops forever on a reset that never sticks; too few and it escalates avoidable blips.
When it helps, and when it misleads¶
Its strength is that it is the guarantee of last resort: it converts an open-ended stall into a bounded one and gives the system a way out even when nothing present understands the cause. It is cheap, robust, and indifferent to the failure's details.
That same blindness is its hazard. A watchdog only masks the symptom, so if the coordination rule is left unchanged it can reset into the very same livelock, again and again — an infinite loop of recoveries that photographs like recovery but never gets anywhere. Set the timeout too short and it destabilizes healthy slow work; kick it on mere activity and it stays silent exactly when it is needed. The classic misuse is trusting the watchdog as the fix rather than the floor. The discipline: kick only on real progress[1], bound the resets and escalate when they stop sticking, and read repeated trips as a signal to change the coordination rule — via a barrier or a desynchronization policy — not just to reset harder.
How it implements the components¶
Liveness Watchdog realizes the archetype's guard-and-recover side — the components that turn "no progress" into a bounded, guaranteed action:
joint_state_progress_invariant— it holds the predicate that the joint state must advance within each window, and treats a deadline passed with a flat signal as the invariant's violation.recovery_checkpoint_and_reentry_rule— on that violation it drives the reset to a known-good checkpoint and the controlled re-entry that follows.
It does not produce the progress signal it watches — that is Progress Counter Heartbeat's liveness_observability_signal — nor identify which states form the cycle (the structural account, with non_progress_cycle_detector and cycle_signature_registry, is State-Machine Cycle Detection's), and it does not change the coordination rule that caused the stall — that falls to Quiescence Barrier or Randomized Retry Desynchronization.
Related¶
- Instantiates: Progress-Guarded Livelock Disruption — the watchdog is the appraisal's liveness guarantee of last resort.
- Consumes: Progress Counter Heartbeat supplies the signal whose flatness it times.
- Sibling mechanisms: Progress Counter Heartbeat · State-Machine Cycle Detection · Quiescence Barrier · Randomized Retry Desynchronization · External Arbitration/Escalation · Circuit Breaker and Cooldown · Bounded Priority Rotation · Joint-State Cycle Trace · Contention Trace Replay · Leader Election or Token Passing · Exponential Backoff with Jitter
Editorial Notes¶
Form Classification¶
Form family: Control, Automation & Runtime
Rationale: Liveness Watchdog operates as a live operational control that automatically routes, enforces, adapts, or responds during execution because it arms a deadline against progress and, when the deadline passes with none, forces a reset to a known-good checkpoint before the stall becomes permanent.
Independent corroboration: The frozen evidence defines Liveness Watchdog as 'Arms a deadline against progress and, when the deadline passes with none, forces a reset to a known-good checkpoint before the stall becomes permanent', so its operative form is Control, Automation & Runtime.
Review outcome: Independent reviewer agreement; high confidence.
Origin Attribution¶
Primary origin: Computer Science & Software Engineering
Origin pattern: Convergent development
Present-day reach: Specialized
Rationale: Watchdog timers and progress monitors are established computing mechanisms for detecting stalled execution and forcing recovery.
Related originating lineages:
- Engineering & Design — Watchdog timers originated as fault-tolerance devices in embedded control and electronics engineering.
Review resolution: Light authoritative research supports computer_science as the primary provenance: Watchdog timers and progress monitors are established computing mechanisms for detecting stalled execution and forcing recovery. NASA's software engineering handbook specifies watchdog timers for detecting loss of progress and restoring processing. The competing reviewed lineage (engineering_design) and other formative traditions remain explicit alternates rather than being erased or confused with downstream applicability. origin_mode=convergent records the relationship among those origin traditions, while domain_reach=specialized separately records how broadly the generalized mechanism can be applied.
Review outcome: Researched adjudication after independent review; high confidence.
Sources consulted:
- https://swehb.nasa.gov/spaces/SWEHBVD/pages/102695807/9.16%2BThread%2BSafety — NASA's software engineering handbook specifies watchdog timers for detecting loss of progress and restoring processing.
Notes¶
A watchdog's reset presumes two things the mechanism cannot supply itself: that a known-good checkpoint exists, and that re-entry from it is safe. Fire it without either and the "recovery" can leave the system in a worse, inconsistent state than the stall it interrupted. This is what separates the watchdog from a Quiescence Barrier: the barrier brings the system to rest and releases it cleanly, whereas the watchdog snaps it back to a saved point — powerful, but only as safe as that saved point.
References¶
[1] National Aeronautics and Space Administration, Ames Research Center. Space Flight System Design and Environmental Test. Ames Technical Standard ARC-STD-8070.1 (2018). Supports kicking the watchdog only after a defined processing path completes and escalating to deeper reinitialization when prior restart levels fail; bounded resets and coordination-rule changes such as barriers or desynchronization are not established. registry ↩