Task-Based Runtimes
Task-based programming expresses work as many small tasks with data dependencies, letting a runtime schedule them dynamically for parallelism and overlap.
Tasks and dependencies
Instead of writing explicit parallel loops and barriers, a task-based program declares units of work (tasks) and the data each reads and writes. A runtime builds a dependency graph from these declarations and executes tasks as soon as their inputs are ready, on whatever worker is free. The programmer specifies what depends on what; the runtime decides when and where each task runs.
Why it helps
This model naturally exposes irregular and dynamic parallelism that static loops handle poorly. It tolerates load imbalance because idle workers pull ready tasks from a queue (work stealing), and it overlaps computation with communication because a communication task and an independent compute task can run at once without hand-coded orchestration. It also adapts to heterogeneous hardware, since the runtime can route a task to a CPU or GPU worker.
- Declare tasks plus their read/write data; the runtime infers ordering.
- Work stealing balances load across idle and busy workers.
- Communication and computation overlap without manual choreography.
- Maps cleanly onto heterogeneous CPU/GPU resources.
The cost
Task runtimes add overhead: building and traversing the dependency graph, and scheduling each task, costs cycles. If tasks are too fine-grained, this overhead dominates; too coarse, and parallelism and load balance suffer. Choosing task granularity is the central tuning knob. Debugging is also harder because execution order varies run to run, so reproducibility requires care in how reductions and shared state are handled.
In practice
Expressing a Hyperion solver's timestep as tasks, halo exchange, interior update, boundary update, diagnostics, lets the runtime overlap the halo communication with the interior computation automatically and rebalance if some subdomains carry more work. This recovers overlap and balance that a rigid bulk-synchronous loop would leave on the table.