Matrix Norms

Measures of a matrix's size, from element-wise sums to the largest amount it can stretch a vector.

Two families

Matrix norms come in two flavors. Entry-wise norms treat the matrix as a long vector, most notably the Frobenius norm, the square root of the sum of squared entries. Operator or induced norms measure how much a matrix can amplify a vector: the induced p-norm is the largest ratio |Ax|_p / |x|_p over all nonzero x.

The important ones

• Frobenius norm: sqrt of sum of squared entries, equals sqrt of sum of squared singular values
• induced 2-norm (spectral norm): the largest singular value
• induced 1-norm: the largest absolute column sum
• induced infinity-norm: the largest absolute row sum

Submultiplicativity

A useful matrix norm is submultiplicative: |AB| <= |A| |B|. This inequality, satisfied by the Frobenius and all induced norms, lets you bound the growth of products and is the foundation of error analysis for algorithms that chain many matrix operations together.

The spectral norm and singular values

The spectral norm equals the largest singular value of the matrix, the maximum stretching factor of the transformation. It appears in the condition number, in stability analysis of iterations, and in bounds on how perturbations propagate. Because it requires the top singular value, it is more expensive to compute than the Frobenius norm.

import numpy as np
A = np.array([[1.0, 2.0], [0.0, 2.0]])
print(np.linalg.norm(A, 'fro'))   # Frobenius
print(np.linalg.norm(A, 2))       # spectral = largest singular value

The nuclear norm, the sum of singular values, is the tightest convex surrogate for matrix rank and drives low-rank recovery methods used to denoise and compress large simulation datasets.