Reinforcement Learning
Learning a policy that maximizes cumulative reward by interacting with an environment through trial and error.
Definition
Reinforcement learning (RL) trains an agent to choose actions in an environment so as to maximize expected cumulative reward over time. The agent observes a state, takes an action, receives a reward, and transitions to a new state, refining its policy from experience.
Reward design is the subtle heart of RL: an agent optimizes exactly the reward it is given, so a poorly specified reward produces behavior that games the metric while missing the intent. This reward-hacking problem makes careful objective design as important as the learning algorithm itself.
Sample efficiency is a central concern: learning by trial and error can require enormous numbers of interactions, which is cheap in a simulator but expensive or unsafe in the real world. This gap motivates training in simulation and transferring to reality, model-based methods that learn a world model to plan against, and offline approaches that learn from logged data without live exploration.
RL is formalized as a Markov decision process defined by states, actions, transition probabilities, and rewards. The core challenge is balancing exploration of new actions against exploitation of known good ones.
Key ideas
- Value functions estimate long-run reward from a state.
- Q-learning learns action values without a model.
- Policy gradients optimize the policy directly.
- Discounting weights near-term reward over distant reward.
Why it matters
RL suits sequential decision and control problems where the right action depends on consequences that unfold over time, such as robotics, game play, and real-time control loops.
Fusion connection
RL is an active research direction for tokamak plasma control, where an agent adjusts coil currents to shape and stabilize the plasma. For Hyperion such controllers are studied in simulation, not on hardware, ahead of construction beginning Q2 2027.