The Pseudoinverse

A generalized inverse that gives the least-squares, minimum-norm solution for any matrix, invertible or not.

Motivation

Most matrices are not square or not invertible, yet we still want a canonical way to solve Ax = b. The Moore-Penrose pseudoinverse, written A^+, provides it. For an invertible square matrix it equals the ordinary inverse. For any other matrix it returns the best possible answer: the least-squares solution, and among all least-squares solutions, the one of smallest norm.

Construction from the SVD

If A = U S V^T is the singular value decomposition, then A^+ = V S^+ U^T, where S^+ is formed by transposing S and replacing each nonzero singular value by its reciprocal, leaving zeros in place. This construction makes clear that the pseudoinverse inverts the invertible part of the transformation and ignores the null directions.

The four defining conditions

• A A^+ A = A
• A^+ A A^+ = A^+
• A A^+ is symmetric
• A^+ A is symmetric

These four Penrose conditions determine A^+ uniquely for any matrix, giving a rigorous foundation independent of the SVD construction.

Two special cases

When A has full column rank, A^+ = (A^T A)^{-1} A^T, the left inverse that produces the least-squares solution. When A has full row rank, A^+ = A^T (A A^T)^{-1}, the right inverse that produces the minimum-norm solution of an underdetermined system. The general pseudoinverse unifies both.

import numpy as np
A = np.array([[1.0, 2.0], [2.0, 4.0], [1.0, 1.0]])  # rank-deficient-ish
Apinv = np.linalg.pinv(A)
b = np.array([1.0, 2.0, 0.5])
print(Apinv @ b)   # minimum-norm least-squares solution

When inverting an operator whose true rank is uncertain, as in reconstructing a field from limited diagnostics, the pseudoinverse via SVD gives a stable answer by suppressing directions the data cannot constrain.