Missing Heartbeat Monitor¶
Software or tool — instantiates Expected-Absence Signal Interpretation
Detects missed keepalives, check-ins, reports, or scheduled signals and routes them through false-absence checks before declaring failure.
Some expectations repeat on a clock: a service that should ping every ten seconds, a field device that should check in hourly, a nightly batch that should report by 6 a.m. Missing Heartbeat Monitor is the running tool that watches those recurring signals and fires the instant one fails to arrive. Its defining trait is inverted, timer-driven detection: unlike ordinary monitoring, which reacts to a bad event that arrived, the heartbeat monitor reacts to the non-arrival of a good one — it maintains a countdown per source, resets it on each heartbeat, and treats an expired countdown as a candidate absence. Crucially, it does not leap from "countdown expired" to "source is dead." It writes an explicit null event and routes the candidate through a false-absence check — verifying the monitor and path are themselves healthy — before letting the miss stand as real. It is the live detector, not the escalation policy that decides what to do once a miss is confirmed.
Example¶
A team runs a fleet of microservices where each instance emits a heartbeat to a central watchdog every fifteen seconds. The Missing Heartbeat Monitor keeps a per-instance timer; when an instance's heartbeat is more than three intervals overdue, the timer expires and a candidate absence is raised. Naively, that would page someone to restart a "dead" service.
But the monitor's defining discipline runs first. On expiry it records a null-event entry (instance X, no heartbeat since 14:02:11) and checks the false-absence guardrail: Is the watchdog itself receiving heartbeats from other instances right now? Is the message bus the heartbeats travel on healthy? Did a deploy just cycle this instance on purpose? One afternoon the monitor sees forty instances go silent at once — and because the guardrail notices the watchdog's own ingestion queue stalled, it recognizes the silence as an artifact of its own blindness, not forty simultaneous crashes. It parks the candidates in a brief hold, suppresses the mass page, and flags the ingestion stall instead. A single real crash, by contrast, is confirmed against a healthy path in seconds and handed off for action.
How it works¶
- Register the expected cadence. Each source declares its heartbeat interval and tolerance; the monitor maintains a live timer per source, reset on every received signal.
- Detect by expiry, not by event. A timer that runs past its tolerance raises a candidate absence and writes an explicit null-event record — absence becomes a logged fact, not an inferred one.
- Gate on channel health. Before the candidate is promoted to a real miss, the guardrail checks that the monitor, the transport path, and the ingestion side are themselves alive — correlated silence across many sources is treated as self-blindness, not mass failure.
- Hold, then hand off. A confirmed miss is briefly parked in a pending state and passed to the response machinery; the monitor itself does not escalate or fail-safe.
Tuning parameters¶
- Interval and tolerance — the expected period and how many missed beats trigger. Tight tolerances detect death fast but false-alarm on jitter; loose ones are calm but slow to notice a real stop.
- Correlation window — how the monitor treats many sources going silent together. Aggressive correlation suppresses mass false alarms but can mask a genuine broad outage.
- Self-health gating strictness — how thoroughly it verifies its own path before trusting a miss. Stricter gating kills false absences but adds latency to real ones.
- Hold duration — how long a confirmed miss waits in pending before hand-off. A short hold catches flapping (dead-then-alive) sources; a long one delays true escalation.
- Null-record verbosity — how much context each null event captures. Rich records aid downstream diagnosis but cost storage and write throughput.
When it helps, and when it misleads¶
Its strength is catching the silent failure — the source that stops without ever emitting an error — and doing so without the classic overreaction of treating its own blindness as the world's catastrophe. It is the software embodiment of a watchdog timer: a countdown that assumes trouble when it stops being reset[1], but here wrapped in a false-absence check so a stalled watchdog can't fake a fleet-wide death.
Its failure mode is the pathological pair of errors at the tolerance boundary: set too tight and normal jitter or a slow-but-alive source triggers phantom deaths; set too loose and a genuinely dead source lingers "green" past the point of harm. And a monitor that over-trusts its own correlation logic can suppress a real broad outage as if it were self-blindness. The guarding discipline is to instrument the monitor's own liveness independently (so its silence is itself detectable), tune tolerances to each source's real jitter, and keep the correlation logic honest by confirming path health directly rather than inferring it from the pattern of silence.
How it implements the components¶
expected_event_model— it encodes each source's expected heartbeat cadence and tolerance as the standard a timer measures against.null_event_channel— on timer expiry it writes an explicit non-occurrence record, making the missed beat a logged event rather than an absent one.false_absence_guardrail— it checks its own path and ingestion health before promoting a candidate, so self-blindness and correlated stalls are not misread as failure.temporary_hold_state— a confirmed miss is briefly parked in pending to absorb flapping before hand-off.
It does not decide the graded follow-up-reroute-escalate steps after a confirmed miss (response_ladder, absence_threshold_rule) — that is No-Response Escalation Protocol; the monitor detects and validates the silence but does not act on it.
Related¶
- Instantiates: Expected-Absence Signal Interpretation — the live detector that turns recurring non-signals into validated candidate misses.
- Consumes: Expected Event Register supplies the cadences and tolerances the monitor watches.
- Sibling mechanisms: Absence Likelihood Dashboard · Confirmation Probe Request · Detection Opportunity Audit · Exception-Lag Review Workflow · Expected Event Register · No-Response Escalation Protocol · Null-Result Power Check · Silence Signal Review Board
Editorial Notes¶
Form Classification¶
Form family: Monitoring, Sensing & Alerting
Rationale: Missing Heartbeat Monitor operates as an ongoing sensing arrangement that repeatedly observes actual state and surfaces changes or alerts because it detects missed keepalives, check-ins, reports, or scheduled signals and routes them through false-absence checks before declaring failure.
Independent corroboration: The frozen evidence defines Missing Heartbeat Monitor as 'Detects missed keepalives, check-ins, reports, or scheduled signals and routes them through false-absence checks before declaring failure', so its operative form is Monitoring, Sensing & Alerting.
Review outcome: Independent reviewer agreement; high confidence.
Origin Attribution¶
Primary origin: Computer Science & Software Engineering
Origin pattern: Cross-disciplinary synthesis
Present-day reach: Multi-domain
Rationale: Heartbeat and keepalive failure detectors are established distributed-systems and network-monitoring mechanisms.
Related originating lineages:
- Engineering & Design — Condition monitoring contributes false-alarm checks before declaring equipment failure.
Review outcome: Independent reviewer agreement; high confidence.
References¶
[1] Kopetz, H. Real-Time Systems: Design Principles for Distributed Embedded Applications. 2nd ed., Springer New York (2011). Defines a watchdog timer as a countdown periodically reset by the monitored system, with a missed reset treated as failure. registry ↩