Norms and the Condition Number

Vector and matrix norms measure size and error, and the matrix condition number bounds how much a linear solve can amplify input errors.

Measuring size and error

A norm assigns a nonnegative size to a vector or matrix. Common vector norms are the 1-norm (sum of absolute values), the 2-norm (Euclidean length), and the infinity-norm (largest absolute component). Norms make error precise: the relative error of an approximation is its difference from the true value, measured in a norm, divided by the true value's norm.

Matrix norms

A matrix norm often measures the largest stretching the matrix applies to any vector: the induced 2-norm equals the largest singular value. Induced norms are submultiplicative, meaning the norm of a product is at most the product of the norms, which is the key property for bounding error propagation through computations.

The condition number

The condition number of an invertible matrix is kappa(A) = norm(A) times norm(A^{-1}), equal in the 2-norm to the ratio of largest to smallest singular value. It bounds sensitivity: solving A x = b with a relative input error e can produce a relative solution error as large as kappa(A) times e. A large condition number warns of unavoidable accuracy loss.

import numpy as np
A = np.array([[1.0, 2.0], [2.0, 4.0001]])
print(np.linalg.cond(A))          # 2-norm condition number
print(np.linalg.cond(A, np.inf))  # infinity-norm version

Reading the condition number

A condition number near 1 is ideal; orthogonal matrices have condition number exactly 1 and never amplify error, which is why orthogonal transformations are favored in stable algorithms. A condition number near 1/epsilon means the matrix is numerically singular and the solution may have no correct digits. Losing about log10(kappa) decimal digits is the rule of thumb.

Monitoring condition numbers flags ill-conditioned operators in the discretized systems of breeder Hyperion models, guiding when reformulation or better preconditioning is needed.