Neural Network Basics
The building blocks of the models used across fusion ML, from layers to training.
What a neural network is
A neural network is a function built from layers of simple units. Each unit computes a weighted sum of its inputs, adds a bias, and applies a nonlinear activation. Stacking layers lets the network represent complex mappings. Training adjusts the weights to fit data.
How training works
- A loss function measures prediction error
- Backpropagation computes the loss gradient with respect to every weight
- An optimizer, such as stochastic gradient descent or Adam, updates weights
- Repeated over many batches, the network converges toward a good fit
Architectures by data type
Fully connected networks handle fixed-size vectors. Convolutional networks handle spatial data like images and 1D signals. Recurrent networks and transformers handle sequences such as diagnostic time series. Matching architecture to data structure is a first design choice.
Why nonlinearity matters
Without nonlinear activations, stacked layers collapse to a single linear map. The activation functions (ReLU and its variants, and others) are what let networks represent the nonlinear relationships that dominate plasma physics, from transport thresholds to instability onset.
Strengths and cautions
Neural networks are flexible universal approximators, strong when data are plentiful and structure is complex. But they are data-hungry, opaque, prone to overfitting on small datasets, and overconfident out of distribution, all acute concerns in fusion. They are one tool among many; the following pages on regularization, uncertainty, and validation describe the discipline that makes them trustworthy in this domain.