Amdahl's and Gustafson's Laws
Amdahl's law bounds speedup when a serial fraction is fixed; Gustafson's law shows scaling improves when the problem grows with the processors.
Amdahl's law
Amdahl's law models a fixed-size problem with a fraction s that is inherently serial and a fraction (1-s) that parallelizes perfectly over P processors. The speedup is 1 / (s + (1-s)/P). As P grows, the parallel term vanishes and speedup approaches the hard ceiling 1/s. If just 5 percent of a program is serial, the maximum possible speedup is 20, no matter how many processors are used. This is the pessimistic view that governs strong scaling.
Gustafson's law
Gustafson's law reframes the question. In practice, people who acquire more processors use them to solve larger problems in the same time, not the same problem faster. If the parallel work grows with P while the serial part stays roughly fixed, the effective speedup is P - s*(P-1), which is nearly linear in P. This optimistic view governs weak scaling and explains why large machines remain useful despite Amdahl's ceiling: the serial fraction shrinks in relative importance as the problem grows.
- Amdahl: fixed problem, speedup capped at 1/s by the serial fraction.
- Gustafson: growing problem, near-linear speedup as work scales with P.
- Both are the same physics viewed under different problem-size assumptions.
- Together they explain why strong scaling saturates but weak scaling persists.
Reconciling them
The laws are not in conflict; they answer different questions. Amdahl asks how much faster a fixed problem can go (strong scaling) and warns that serial work and overhead dominate at high P. Gustafson asks how much bigger a problem can be tackled in fixed time (weak scaling) and shows that this scales far better. Which applies depends on whether the goal is a faster answer to a set problem or a bigger problem in a set time.
In practice
For Hyperion, Amdahl says there is a point beyond which throwing nodes at one fixed case stops helping, set by serial setup, I/O, and global reductions. Gustafson says the same nodes can instead resolve a finer or larger model efficiently. Design targets each law deliberately: shrink serial fractions to push the Amdahl ceiling, and grow resolution to exploit Gustafson scaling.