Safe Exploration in Learned Control
Keeping a learning or learned controller inside physically safe bounds on hardware where mistakes are costly.
Why safety is non-negotiable
Fusion devices are expensive, scarce, and can be damaged in milliseconds. A controller that explores freely, or that behaves unpredictably in an unfamiliar state, is unacceptable on hardware. Safe exploration keeps learning and deployment inside a region where the worst outcome is tolerable.
Where the risk lives
- Exploration during training on hardware
- Distribution shift: states the policy never saw in simulation
- Actuator commands that exceed engineering limits
- Silent failure of the state estimate feeding the policy
Techniques
Most fusion RL avoids on-hardware exploration entirely by training in simulation and deploying a fixed policy. Beyond that, action clipping enforces hard limits; a safety layer or shield overrides commands that would violate constraints; and a fallback controller takes over if the learned policy produces anomalous outputs. Uncertainty estimates let the system defer to a conservative policy when it is unsure.
Verification before deployment
A learned controller is stress-tested across many simulated scenarios, including off-nominal ones, before it touches a device. Its command envelope is checked against engineering limits, and its behavior under sensor faults is examined. The goal is bounded, predictable behavior, not just good average performance.
Design-stage relevance
For design-stage machines such as the Kronos breeder and burner, control studies live entirely in simulation, so exploration risk is a simulation-fidelity question rather than a hardware one. When any concept moves toward hardware, safe-exploration guarantees become a prerequisite, not an afterthought.