GMRES

GMRES solves general nonsymmetric systems by minimizing the residual over a growing Krylov subspace, at the cost of storing the accumulated basis.

Least-squares over a Krylov subspace

The generalized minimal residual method (GMRES) solves general, possibly nonsymmetric, systems A x = b. At step k it finds the vector in the Krylov subspace spanned by b, Ab, ..., A^{k-1}b that minimizes the residual norm. It builds an orthonormal basis for that subspace with the Arnoldi process and solves a small least-squares problem each step.

Monotone residual, growing cost

Because GMRES minimizes the residual over an expanding subspace, the residual norm never increases from one iteration to the next. The drawback is that Arnoldi must store and orthogonalize against all previous basis vectors, so both memory and work per step grow with the iteration count.

Restarting

To bound the growing cost, restarted GMRES, written GMRES(m), throws away the basis after m steps and restarts from the current approximation. This caps memory but can stall on hard problems because it discards accumulated information. Choosing m trades robustness against resource use, and a good preconditioner reduces the needed m.

import numpy as np
from scipy.sparse.linalg import gmres, LinearOperator
# solve A x = b with restart every 30 iterations
# A can be given as a matrix-vector product only
# x, info = gmres(A, b, restart=30, rtol=1e-8, M=preconditioner)

Where it fits

GMRES is the standard Krylov solver when A is nonsymmetric, as arises from advection-dominated or non-self-adjoint operators. Alternatives such as BiCGSTAB and QMR use short recurrences to bound memory but can behave erratically. GMRES is more robust; its cost is managed by preconditioning and restarts.

Preconditioned GMRES solves the nonsymmetric sparse systems from advective transport and coupled multiphysics operators in breeder Hyperion simulations.