Support Vector Machine
A classifier that finds the boundary with the widest margin between classes, extended to nonlinear cases with kernels.
Definition
A support vector machine (SVM) finds the decision boundary that maximizes the margin, the distance to the nearest points of each class. Those nearest points, the support vectors, are the only ones that determine the boundary.
The kernel trick works because the algorithm depends on the data only through inner products, which a kernel function can compute in the higher-dimensional space implicitly. Choosing the kernel and its parameters is where most of the modeling effort and risk of overfitting lie.
Scaling is a practical necessity, and the method's memory and time cost grow with the number of training points because the kernel compares pairs of examples, which limits it on very large datasets. For such cases, linear SVMs and approximate kernel methods restore scalability. Where data is limited and clean, however, an SVM with a well-chosen kernel is hard to beat.
The kernel trick maps inputs into a higher-dimensional space implicitly, letting an SVM draw nonlinear boundaries without ever computing the mapping explicitly.
Components
- Margin: the gap the model tries to widen.
- Kernel: linear, polynomial, or radial basis function.
- Regularization parameter C: trades margin width against training errors.
Why it matters
SVMs are effective in high dimensions and with limited data, and they have strong theoretical grounding in margin maximization. They were the leading classifier before deep learning and remain useful for small, clean datasets.
Fusion connection
With kernels, SVMs classify Hyperion operating points into stable and unstable regimes from modest labeled sets, where a large network would overfit.