Sim-to-Real Transfer
Why a policy or model trained in simulation may fail on hardware, and how to narrow the gap.
The reality gap
Models and control policies for fusion are usually trained in simulation, because hardware time is scarce and exploration is risky. But no simulator is perfect. The mismatch between simulator and device, the reality gap, can cause a policy that excels in simulation to underperform or fail on the real machine.
Sources of the gap
- Approximate physics in the simulator
- Unmodeled actuator dynamics and delays
- Sensor noise and calibration differences
- Regimes the simulator was never validated in
Closing the gap
Domain randomization trains over a distribution of simulator parameters so the policy is robust to which one is true. System identification tunes the simulator to match measured hardware behavior. Residual learning adds a small correction learned from limited real data on top of a simulation-trained base. Each reduces, but does not eliminate, the gap.
Robustness over peak performance
A policy tuned to exploit exact simulator details transfers worse than one trained to be robust across plausible variations. For safety-critical control, robust and predictable beats brilliant-but-brittle. This trade-off is deliberate.
Honest accounting
Simulation performance is an upper bound, not a promise. Claims about a learned system should distinguish simulation results from hardware results. For design-stage devices such as the Kronos machines, all learned-control work is in simulation, and the reality gap is a known, stated caveat rather than a solved problem.