Task Parallelism
Task parallelism runs different operations on different processors, coordinating them through dependencies rather than a single shared loop.
Different work in parallel
Where data parallelism repeats one operation over many elements, task parallelism executes distinct operations concurrently. A weather model might advance dynamics on one set of cores while a chemistry module runs on another; a build system compiles independent source files at once. The tasks may consume and produce different data.
Dependencies and the task graph
Tasks rarely run in complete isolation. Their relationships form a directed acyclic graph (DAG): an edge from task A to task B means B cannot start until A finishes. A scheduler walks this graph, launching any task whose inputs are ready. The critical path, the longest dependency chain, bounds the best possible completion time no matter how many processors exist.
Runtimes that support it
- OpenMP tasks with
dependclauses express fine-grained DAGs - Thread pools and futures in general-purpose languages
- Workflow engines that schedule coarse tasks across a cluster
Load balance is the challenge
Tasks vary in size, so a static assignment can leave some processors idle while others are overloaded. Dynamic scheduling and work stealing keep processors busy by handing idle workers tasks from a shared queue. The trade-off is scheduling overhead against balance.
When to use it
Task parallelism suits irregular or heterogeneous workloads: adaptive mesh refinement, tree traversals, and multi-physics couplings where different physical models advance different parts of a simulation. In practice it is layered with data parallelism, so each task itself runs across many lanes or cores.