t-SNE
t-SNE is a nonlinear method that maps high-dimensional data to 2-D or 3-D, preserving local neighborhoods for visualization.
Visualizing neighborhoods
t-distributed stochastic neighbor embedding (t-SNE) is a nonlinear technique built for visualization. It converts pairwise distances into probabilities of being neighbors, then arranges points in 2-D or 3-D so that the low-dimensional neighbor probabilities match the high-dimensional ones as closely as possible.
How it works
- In high-D, each point's neighbors get a Gaussian similarity, tuned by the perplexity parameter.
- In low-D, similarities use a heavy-tailed Student-t distribution to avoid crowding.
- Gradient descent minimizes the KL divergence between the two similarity distributions.
The heavy tail is the key trick: it lets distant points spread out, so clusters separate cleanly instead of collapsing into a crowded blob.
Reading t-SNE plots carefully
t-SNE faithfully preserves local structure but distorts global structure. Cluster sizes and the distances between clusters in a t-SNE plot are not meaningful; only which points sit together is reliable. Different perplexity values and random seeds produce different layouts, so never over-interpret a single run.
from sklearn.manifold import TSNE
Z = TSNE(n_components=2, perplexity=30, init='pca').fit_transform(X)When to use it
Use t-SNE to inspect whether classes or clusters separate, not as input to a downstream model, and not to measure distances. It is slow on large datasets; run PCA first to a few dozen dimensions to speed it up. For a faster method that better preserves global layout, prefer UMAP.