Spiking Neural Networks

Spiking neural networks communicate with discrete timed events rather than continuous values, modeling neurons that integrate inputs and fire when a threshold is crossed.

Event-driven computation

Spiking neural networks (SNNs) are inspired closely by biological neurons. Instead of passing continuous activations, neurons communicate with spikes: discrete, all-or-nothing events at particular moments in time. A neuron integrates the spikes it receives into a membrane potential, and when that potential crosses a threshold it emits a spike and resets. Information is carried by which neurons spike and when, so the network is inherently temporal and event-driven rather than evaluated once per input.

The leaky integrate-and-fire model

The most common neuron model is leaky integrate-and-fire (LIF). Its membrane potential rises with incoming weighted spikes and decays, or leaks, toward rest over time. When the potential exceeds a threshold, the neuron fires and the potential is reset. This captures the essential dynamics, temporal integration, leakage, thresholding, and reset, while remaining cheap to simulate, and it is the workhorse for most SNN research.

# leaky integrate-and-fire step
V = beta * V + I                 # decay then integrate input
spike = (V >= threshold).float()
V = V - spike * threshold        # reset by subtraction

The training difficulty

The spike-generating threshold is a step function whose derivative is zero almost everywhere and undefined at the threshold, so ordinary backpropagation cannot flow gradients through it. This is the central obstacle to training SNNs, and it is addressed by surrogate gradients, which substitute a smooth approximation of the step during the backward pass while keeping the true spike in the forward pass.

• Sparse, event-driven activity can be very energy-efficient on suitable hardware
• Neuromorphic chips execute SNNs with low power by processing only spikes
• Time is a first-class dimension, natural for temporal and sensor data
• Discrete spikes make gradient-based training harder than for standard networks

Where they fit

SNNs are most attractive where energy and latency matter and where inputs are naturally event-based, such as data from event cameras or continuous sensor streams. On neuromorphic hardware the sparse spiking activity can dramatically reduce power compared with dense continuous networks. They remain less mature than mainstream deep learning for large-scale accuracy, and much current work focuses on closing that gap through better training methods and architectures.