Multiclass Classification

Multiclass classification assigns one of several mutually exclusive labels, using softmax or by decomposing into binary problems.

More than two classes

Multiclass classification chooses one label from K greater than two mutually exclusive categories, digit 0-9, species, fault type. It differs from multilabel classification, where an example may carry several labels at once. Some algorithms handle many classes natively; others are built by combining binary classifiers.

Native multiclass

Trees, random forests, gradient boosting, k-NN, and naive Bayes handle multiple classes directly. Neural networks and multinomial logistic regression use a softmax output layer, which turns K scores into a probability distribution over the classes that sums to one, trained with cross-entropy loss.

import numpy as np
def softmax(z):
    z = z - z.max(axis=1, keepdims=True)   # numerical stability
    e = np.exp(z)
    return e / e.sum(axis=1, keepdims=True)

Decomposition strategies

• One-vs-rest: train K binary classifiers, each class against all others; pick the highest score.
• One-vs-one: train a classifier for every pair, K(K-1)/2 of them; vote; more models but each on less data.
• These wrap inherently binary methods such as SVM for multiclass use.

Evaluation across classes

The confusion matrix generalizes to K by K and shows which classes are confused. Summaries average per-class precision, recall, and F1: macro-averaging weights every class equally (surfacing poor performance on rare classes), micro-averaging weights by frequency, and weighted averaging scales by class size. On imbalanced multiclass data, macro-F1 is usually the honest headline metric.

Softmax and one-vs-rest can disagree in practice. Softmax couples the class scores through a shared normalization, so raising one class's probability lowers the others, which suits truly exclusive labels. One-vs-rest trains each boundary independently, which can leave a point claimed strongly by two classes or by none; the scores are then compared, but they were never calibrated against each other. When exclusivity matters and the model supports it, a native softmax head is usually the cleaner choice.