The Singular Value Decomposition
A factorization that reveals the fundamental geometry of any matrix and underlies least squares, rank, and low-rank approximation.
Every matrix factors this way
The singular value decomposition (SVD) writes any matrix as a product of three matrices: an orthogonal matrix U, a diagonal matrix of nonnegative singular values, and another orthogonal matrix V transposed. Geometrically it says every linear map is a rotation, followed by a scaling along orthogonal axes, followed by another rotation. Unlike the eigendecomposition, the SVD exists for every matrix, rectangular or singular, and its factors are always orthogonal, making it exceptionally stable.
What the singular values tell you
The singular values, ordered from largest to smallest, measure how much the matrix stretches space along each principal direction. The largest and smallest singular values give the matrix's norm and, for square matrices, its condition number (their ratio). The number of nonzero singular values is the rank, and small singular values reveal near-rank-deficiency that is invisible to naive methods.
- Rank and null space: revealed cleanly by which singular values are effectively zero
- Least squares: the SVD gives the minimum-norm solution even for rank-deficient problems (the pseudoinverse)
- Low-rank approximation: keeping the largest singular values gives the best low-rank fit (Eckart-Young theorem)
- Principal component analysis is the SVD of a centered data matrix
Computing it
The SVD is computed by first reducing the matrix to bidiagonal form with orthogonal transformations, then applying an implicit QR-type iteration to the bidiagonal matrix. This two-phase approach is backward stable and is the standard routine in every linear algebra library. For very large matrices where only the leading singular values are needed, randomized and Krylov-based partial SVDs are used instead.
Why it is central
The SVD is the most informative matrix factorization: it diagnoses conditioning, solves rank-deficient least squares, compresses data, and reduces model order. In simulation it supports model reduction, where a high-dimensional state is projected onto the few dominant modes captured by the leading singular vectors, dramatically shrinking the problem while preserving the essential dynamics.