Failover Architecture

How control authority moves off a failed node, actuator, or model without losing the plant - redundancy that acts within a loop deadline.

Redundancy that switches in time

Failover is redundancy plus a fast, safe switch. Having a spare is not enough; the stack must detect the primary's failure and transfer authority before the controlled phenomenon runs away. In hard-real-time loops this switch must itself complete inside the loop deadline, so the standby is kept hot and synchronized.

Three redundancy styles

• Hot standby: shadow node runs in lockstep, takes over on heartbeat loss (fastest)
• Warm standby: standby has recent state, needs brief catch-up (cheaper)
• Analytical redundancy: no spare hardware; the twin supplies the missing signal or a fallback law substitutes

Fast protection loops use hot standby control nodes. Sensors use analytical redundancy through the twin. Actuators use whichever backup exists - a second gyrotron, a redundant valve - and degrade if none does.

class FailoverPair:
    def __init__(self, primary, standby, hb_timeout_ms):
        self.p, self.s, self.to = primary, standby, hb_timeout_ms
    def step(self, state):
        if self.p.heartbeat_age_ms() > self.to or self.p.faulted():
            self.s.promote(self.p.last_state())   # hot handover
            self.p, self.s = self.s, self.p
            flag('failover')
        return self.p.control(state)

Failover is not free

Every failover consumes a redundancy budget; a second failure in the same channel forces degradation or safing. The stack tracks remaining redundancy as a first-class state so operators and the availability model know the true margin. For the burner, where the plug is already the limiting component, most failover is analytical and conservative because there is no spare 26.49 T plug. See Graceful Degradation and Redundancy and Voting.