Vector Norms

Different ways to measure the length of a vector, each suited to a different notion of size or distance.

What a norm is

A norm assigns a nonnegative length to every vector and must satisfy three rules: it is zero only for the zero vector, it scales as |c x| = |c| |x| for any scalar c, and it obeys the triangle inequality |x + y| <= |x| + |y|. Any function meeting these rules defines a valid notion of size and, through |x - y|, a notion of distance.

The common norms

• the 2-norm (Euclidean): square root of the sum of squares, the ordinary length
• the 1-norm: sum of absolute values, the taxicab distance
• the infinity-norm: the largest absolute component
• the p-norm: the p-th root of the sum of p-th powers of absolute values

Why the choice matters

Norms encode what you care about. The 2-norm is smooth and rotation-invariant, natural for physical length and least squares. The 1-norm promotes sparsity and is central to compressed sensing and robust fitting. The infinity-norm captures worst-case error. Switching norms can change which solution an optimization problem prefers.

Unit balls

The set of vectors with norm at most 1 is the norm's unit ball. For the 2-norm it is a round disk or sphere; for the 1-norm it is a diamond; for the infinity-norm it is a square or cube. These shapes visually explain why 1-norm minimization tends to land on the axes, producing sparse solutions.

import numpy as np
x = np.array([3.0, -4.0, 0.0])
print(np.linalg.norm(x, 2))     # 5.0
print(np.linalg.norm(x, 1))     # 7.0
print(np.linalg.norm(x, np.inf)) # 4.0

In numerical analysis, norms measure residuals and errors; reporting that a solver reduced the 2-norm of the residual below a tolerance is the standard convergence criterion for large simulations.