Neural Network Architectures for Sequences and Fields
Choosing recurrent, convolutional, attention, or graph models to match plasma data structure.
Matching structure to data
The right architecture reflects the structure of the data. Fusion data come as scalars, time series, images, and quantities on irregular meshes, and different network families are suited to each. Choosing well improves both accuracy and data efficiency.
Recurrent networks
Recurrent networks (LSTM, GRU) process sequences step by step, carrying a hidden state. They suit real-time diagnostic streams where causal, bounded-latency processing is needed, and they naturally handle variable-length inputs. Their weakness is difficulty with very long dependencies and sequential compute that resists parallelism.
Convolutional and temporal-convolutional networks
- 1D convolutions over time windows, efficient and parallel
- 2D convolutions over images and spectrograms
- Dilated temporal convolutions for long receptive fields
Attention and transformers
Transformers use attention to relate every part of a sequence to every other, capturing long-range structure and parallelizing well. They are data-hungry, which limits them on small fusion datasets, but are increasingly used where enough data or pretraining exist.
Graph networks
Graph neural networks operate on data with irregular connectivity, such as quantities on an unstructured mesh or sensors with a spatial layout. They respect the geometry of the problem rather than forcing it onto a grid.
Choosing
There is no universally best architecture. The choice weighs data structure, dataset size, latency needs, and interpretability. On small datasets, simpler architectures often outperform complex ones, and a matched, modest model beats a fashionable, oversized one.