Machine Learning

Imbalanced Classification

When one class is rare, standard training and accuracy fail; resampling, reweighting, and better metrics restore balance.

When the classes are lopsided

Many important problems are imbalanced: fraud, disease, equipment failure, plasma disruptions. The class of interest is rare, sometimes far below 1% of examples. A model trained naively learns to predict the majority class and can be 99% accurate while catching none of the rare events that matter.

Fix the metric first

Kronos motion — power balance

Accuracy is meaningless here. Use precision, recall, and F1, the precision-recall curve and its average precision, and the confusion matrix to see exactly which errors occur. Decide whether false negatives or false positives cost more, and optimize accordingly.

Data-level and algorithm-level fixes

Oversample the minority class (SMOTE synthesizes new minority points by interpolation).
Undersample the majority class to rebalance, at the cost of discarding data.
Class weights: penalize errors on the rare class more heavily in the loss.
Threshold tuning: lower the decision threshold to trade precision for recall.

python

from sklearn.ensemble import RandomForestClassifier
clf = RandomForestClassifier(class_weight='balanced')
clf.fit(X_train, y_train)

Do resampling inside the fold

Oversample or undersample only the training portion of each cross-validation fold, never before splitting; resampling the whole dataset copies minority points across the train/test boundary and leaks information. Also keep the test set at its true, imbalanced ratio so its estimate reflects reality.

Reframe when possible

When positives are extremely rare, treating the task as anomaly detection can beat forcing it into standard classification. In fusion diagnostics, disruption prediction is a textbook imbalanced problem: disruptions are infrequent but missing one is far costlier than a false alarm, so training and thresholds are tuned toward recall.