Matrix Factorization for Recommendation

Matrix factorization embeds users and items in a shared latent space so their dot product predicts preference.

Low-rank structure in preferences

Matrix factorization assumes the large, sparse user-item rating matrix R is approximately the product of two thin matrices: R ~ U V^T, where each user u has a latent vector p_u and each item i has a latent vector q_i. The predicted rating is their dot product p_u . q_i, optionally plus a global mean and user and item biases that capture how generous a user is or how well-liked an item is.

The objective

Learning minimizes the squared error over observed entries only, plus L2 regularization: sum over known (u,i) of (r_ui - mu - b_u - b_i - p_u . q_i)^2 + lambda(||p_u||^2 + ||q_i||^2). Crucially the sum runs over observed ratings, not the full matrix, so missing entries are ignored during fitting rather than treated as zero.

How it is solved

Two optimizers dominate. Stochastic gradient descent walks through observed ratings, nudging the relevant p_u and q_i after each. Alternating least squares fixes one factor and solves a ridge regression for the other in closed form, then swaps, which parallelizes cleanly. Both converge to a low-rank fit; the latent dimension trades expressiveness against overfitting.

# SGD update for one observed rating
err = r_ui - (mu + b_u + b_i + p[u] @ q[i])
p[u] += lr * (err * q[i] - lam * p[u])
q[i] += lr * (err * p[u] - lam * q[i])

What the factors capture

The learned latent dimensions often correspond to interpretable taste axes discovered automatically, such as genre or tone, though no such labels are supplied. Because the model is low-rank, it generalizes to unobserved pairs, which is exactly the recommendation task. It remains a strong, efficient baseline underlying much of collaborative filtering, and can be extended with side features to ease cold start.