Workflow Orchestration Engines
Orchestration engines schedule, connect, and manage the many code executions that make up an integrated fusion simulation.
Coordinating many codes
An integrated simulation is not one program but a graph of many code executions with data dependencies between them. A workflow orchestration engine manages this graph: it launches each component in the right order, passes outputs to inputs, handles failures and restarts, and tracks what ran with which parameters. It is the conductor that turns a collection of codes into a repeatable pipeline.
Orchestration separates the science, which code computes what, from the mechanics, where it runs and how data moves, so scientists can compose complex studies without hand-managing every execution.
Directed acyclic graphs
Most workflows are expressed as a directed acyclic graph of tasks, where edges are data dependencies. The engine determines which tasks can run in parallel, dispatches them to available compute resources, and blocks dependents until their inputs are ready. Iterative loops, as in coupled equilibrium-transport convergence, are handled by repeating subgraphs until a criterion is met.
Resilience and scale
On large runs, individual tasks fail, from hardware faults or non-convergence. A good engine checkpoints progress, retries or skips failed tasks according to policy, and resumes without repeating completed work. It also brokers execution across heterogeneous resources, part of the HPC coupling challenge.
Design relevance
The whole-device studies behind the Hyperion breeder and the Aegis and MetroVolt burner rely on orchestrated pipelines so that a design change can be re-simulated end to end reproducibly. Orchestration is also where provenance is captured, supporting the honest, auditable simulation record maintained before construction.
- Manages the graph of code executions
- Schedules parallel tasks by data dependency
- Checkpoints, retries, and resumes on failure
- Foundation for reproducible pipelines