Pipelining Basics

Pipelining overlaps the stages of instruction execution so several instructions are in progress at once, raising throughput.

The idea

Executing an instruction takes several steps: fetch, decode, execute, memory access, and write-back in a classic design. Without pipelining the processor finishes one instruction before starting the next. Pipelining splits the datapath into stages separated by registers, so while one instruction executes, the next is being decoded and a third is being fetched.

Throughput versus latency

Pipelining does not make a single instruction finish faster; its latency may even rise slightly. What improves is throughput: once the pipeline is full, one instruction completes every clock cycle. An n-stage pipeline can approach an n-fold increase in instruction rate.

Hazards

Overlap creates hazards. A data hazard arises when an instruction needs a result not yet produced by an earlier one. A control hazard arises at branches, when the next instruction is not yet known. A structural hazard arises when two instructions need the same hardware at once.

Handling hazards

Data hazards are handled by forwarding results directly between stages or by stalling. Control hazards are handled by branch prediction and by flushing wrong-path instructions. These techniques keep the pipeline as full as possible.

Clock effect

Shorter stages mean a shorter critical path, so a pipelined design can use a faster clock. The stage registers add overhead, so there is a practical limit to how finely a pipeline can be divided.