HPC & Compute

OpenMP: Shared-Memory Directives

OpenMP adds parallelism to C, C++, and Fortran through compiler directives, letting a single program use all cores of a node with little rewriting.

Parallelism by annotation

OpenMP is a directive-based model for shared-memory parallelism. The programmer marks regions and loops with #pragma omp directives; the compiler and runtime create and manage a team of threads. Because the directives are ignored by a non-OpenMP compiler, the same source can build serial or parallel.

The workhorse: parallel for

python

# C-style pseudocode
# pragma omp parallel for reduction(+:sum)
# for (i = 0; i < n; i++)
#     sum += a[i] * b[i];
# The loop iterations are split across threads;
# the reduction clause safely combines partial sums.

Key concepts

Work sharing: split loop iterations across a thread team
Reduction: combine per-thread partial results without a race
Scheduling: static, dynamic, or guided assignment of iterations
Tasks: express irregular, dependency-driven parallelism

Correctness concerns

Shared variables must be classified as shared or private, and updates to shared state protected. Forgetting a reduction or a private clause causes a race condition. False sharing, where threads update different variables that land on the same cache line, silently degrades performance without being a correctness bug.

OpenMP and the wider stack

OpenMP handles parallelism within a node; across nodes it is paired with MPI in the common MPI+OpenMP hybrid. Recent OpenMP versions also target GPUs through target offload directives, giving one directive-based model for both CPU threads and accelerators. Its incremental nature is a practical strength: a serial code can be parallelized one loop at a time, checking correctness at each step, rather than being rewritten all at once.