Dynamic Routing Between Capsules

Dynamic routing iteratively assigns lower-level capsule outputs to the higher-level capsules whose predictions they most agree with.

Routing by agreement

In a capsule network, each lower-level capsule must decide which higher-level capsule to send its output to. Rather than fixing this with pooling or learned weights alone, dynamic routing decides it at inference time based on agreement. A lower capsule's prediction for a higher capsule is its output multiplied by a learned transformation matrix. The routing algorithm then strengthens the connection when a prediction aligns with the higher capsule's actual output and weakens it otherwise.

The iterative procedure

Routing runs for a few iterations. Coupling coefficients between each lower and higher capsule start uniform and are normalized with a softmax. Each higher capsule computes its output as the coupling-weighted sum of the predictions reaching it, then squashes it. The agreement between each prediction and the resulting higher-capsule output, measured by a dot product, is added to the raw coupling logits, so agreeing predictions gain influence in the next iteration.

b = zeros(num_lower, num_higher)          # routing logits
for it in range(routing_iters):
    c = softmax(b, dim=1)                  # couplings per lower capsule
    s = (c[..., None] * predictions).sum(0)  # weighted sum per higher
    v = squash(s)                          # higher-capsule outputs
    b = b + (predictions * v).sum(-1)      # agreement update

• Couplings are computed per input, not fixed after training
• A few iterations, typically three, are enough in practice
• Agreement concentrates each part on the whole it best explains
• The transformation matrices are the only learned routing parameters

Interpretation and cost

Dynamic routing implements a soft clustering: predictions that cluster together in the higher capsule's output space reinforce each other, so the network favors interpretations where many parts agree on a consistent whole. This is what gives capsule networks their part-whole reasoning. The drawback is that iterative routing must run for every forward pass and does not parallelize as cleanly as a single matrix multiply, which is one reason the approach has been hard to scale relative to transformers and convolutional networks.