Subspaces

Subsets closed under addition and scaling; the natural stage for column spaces, null spaces, and projections.

The closure test

A subset W of a vector space is a subspace if it is nonempty and closed under the two vector-space operations: for any u and v in W and any scalar c, both u + v and c u remain in W. From these, W automatically contains the zero vector and all inverses, so it is a vector space in its own right. Every subspace must pass through the origin.

Familiar subspaces

• lines and planes through the origin in R^3
• the column space: the span of a matrix's columns
• the null space: all vectors sent to zero by a matrix
• the row space and the left null space
• solution sets of homogeneous linear equations

Operations on subspaces

The intersection of two subspaces is always a subspace. Their union usually is not; instead one takes the sum, the set of all u + w with u in the first and w in the second, which is the smallest subspace containing both. When the intersection is only the zero vector, the sum is called a direct sum, and every element decomposes uniquely.

Orthogonal complements

Given a subspace W, its orthogonal complement is the set of all vectors perpendicular to everything in W. A space splits as the direct sum of any subspace and its orthogonal complement, which is exactly what makes orthogonal projection well defined and unique.

import numpy as np
# The null space is a subspace; find it via SVD
A = np.array([[1.0, 2.0, 3.0], [2.0, 4.0, 6.0]])
u, s, vt = np.linalg.svd(A)
null = vt[np.sum(s > 1e-10):]
print(null.shape[0], 'basis vectors for the null space')

The four fundamental subspaces of a matrix organize the entire theory of linear systems, splitting both the input and output spaces into complementary pieces.