The Agent-Environment Loop

Every reinforcement learning problem is a cycle: observe a state, act, receive reward and next state, repeat.

The interaction cycle

Reinforcement learning is defined by a repeating interaction between two entities. The agent is the decision maker. The environment is everything the agent cannot directly control but can influence through its actions. The boundary between them is drawn at the point where the agent's control ends: the agent controls its actions, not the rewards or dynamics that follow.

One time step

Kronos motion — cycle loop

At step t the environment presents state S_t.
The agent picks action A_t according to its policy.
The environment returns reward R_{t+1} and next state S_{t+1}.
The cycle repeats until an episode ends or forever in continuing tasks.

Episodic and continuing tasks

In episodic tasks interaction breaks into finite episodes that end in a terminal state — a game won or lost, a robot reaching a goal. In continuing tasks there is no natural end and interaction goes on indefinitely; discounting keeps the accumulated reward finite.

Observations versus states

A true state summarizes everything relevant about history for predicting the future — the Markov property. In practice the agent often sees only a partial observation, not the full state. When observations hide relevant information the problem becomes a partially observable MDP, and the agent may need memory of past observations to act well.

Reward is the objective

The reward signal defines the goal. A central design principle, sometimes called the reward hypothesis, is that goals and purposes can be represented as maximization of expected cumulative reward. Getting the reward right is often harder than the learning algorithm itself, which is why reward shaping and reward design receive so much attention.