Channel Capacity

Channel capacity is the maximum rate at which information can be sent reliably over a noisy channel.

Definition

The capacity C of a channel is the largest mutual information between its input and output, maximized over all input distributions. It is the greatest number of bits per use that can be transmitted with vanishing error.

The noisy-channel coding theorem

Kronos motion — which application

Shannon proved that for any rate below capacity, codes exist that make the error probability as small as desired; above capacity, reliable communication is impossible. Capacity is a sharp threshold, not a gradual decline.

The binary symmetric channel

A channel that flips each bit with probability p has capacity 1 − H(p) bits per use, where H is the binary entropy. At p = 0.5 the output is independent of the input and capacity drops to 0.

Bandwidth and noise

For a continuous channel with bandwidth B and signal-to-noise ratio S/N, the Shannon–Hartley theorem gives capacity B log₂(1 + S/N) bits per second, linking capacity to physical bandwidth and noise power.

Why it matters

Capacity separates the achievable from the impossible. Modern codes such as turbo and LDPC codes operate remarkably close to capacity, which is why the limit is a practical design target, not just a theoretical bound.

python

import math
def bsc_capacity(p):
    if p in (0,1): return 1.0
    H = -(p*math.log2(p) + (1-p)*math.log2(1-p))
    return 1 - H
print(bsc_capacity(0.1))  # ~0.531