Naive Bayes

Naive Bayes applies Bayes' rule with a strong independence assumption, giving a fast, surprisingly effective probabilistic classifier.

Bayes with an assumption

Naive Bayes classifies by Bayes' rule: the posterior probability of a class given the features is proportional to the class prior times the likelihood of the features. The 'naive' part is assuming all features are conditionally independent given the class, so the joint likelihood factorizes into a simple product of per-feature terms.

The math, compactly

P(class | x) is proportional to P(class) * product over j of P(x_j | class). We pick the class with the largest value. The independence assumption is almost never true, yet the classifier often works well because it only needs the correct class to score highest, not the probabilities to be exact.

Variants by feature type

• Gaussian: continuous features modeled as normal per class.
• Multinomial: count features, classic for text (word counts).
• Bernoulli: binary presence/absence features.
• Laplace smoothing adds a pseudo-count so unseen feature values do not zero out a probability.

from sklearn.naive_bayes import MultinomialNB
clf = MultinomialNB(alpha=1.0)   # alpha = Laplace smoothing
clf.fit(X_counts, y)

Why it endures

Naive Bayes trains in a single pass, needs little data, handles very high-dimensional inputs, and is a strong baseline for text classification and spam filtering. Its weaknesses follow from the assumption: correlated features are double-counted, and its probability estimates are often poorly calibrated even when its decisions are right. Treat its outputs as rankings rather than trustworthy probabilities.