Alternating Least Squares
ALS fits matrix-factorization models by alternately solving closed-form least-squares problems for each factor.
Turning a hard problem into easy ones
The matrix factorization objective is non-convex when both user factors P and item factors Q vary together, but it becomes a convex least-squares problem in one factor when the other is held fixed. Alternating least squares (ALS) exploits this: fix Q and solve for every user vector, then fix P and solve for every item vector, and repeat.
The closed-form step
With item factors fixed, each user vector p_u is the solution of a ridge regression against the items that user rated: p_u = (Q_u^T Q_u + lambda I)^{-1} Q_u^T r_u, where Q_u stacks the factors of the rated items. Each such solve is a small linear system of size equal to the latent dimension, independent across users, so the whole step is embarrassingly parallel.
Why it is popular
- Each half-step is a global optimum, so the objective decreases monotonically
- User and item updates parallelize across machines with no shared state
- No learning rate to tune, unlike stochastic gradient descent
- Handles the implicit-feedback formulation efficiently with a weighted variant
Implicit-feedback ALS
For implicit signals such as plays or clicks, ALS is adapted by treating every entry as observed with a confidence that grows with the interaction count, and a preference of one for any positive interaction and zero otherwise. A precomputation trick keeps the per-user solve efficient despite the matrix now being dense in principle. This weighted ALS is a workhorse for large-scale implicit-feedback recommendation.
The main trade-offs are memory for the factor matrices and the cost of the matrix inversions, which grow with the latent dimension cubed but stay cheap for the modest dimensions typical in practice.