Cache Memory
Small, fast memory close to the processor that holds recently used data to hide slow main-memory access.
Definition
Cache memory is a small, fast layer of storage between the processor and main memory. It holds recently and frequently accessed data so the processor avoids the long wait of fetching from slower RAM.
False sharing is a subtle parallel pitfall: when two threads update different variables that happen to share a cache line, the hardware keeps invalidating that line between them, quietly destroying performance. Padding data to separate lines is the usual remedy.
Because processor speed has far outpaced memory speed, many programs are limited by how well they use the cache rather than by arithmetic, and reorganizing data to improve locality can yield order-of-magnitude speedups. Pitfalls such as false sharing, where threads contend over the same cache line, can silently destroy parallel performance. Cache-aware design, blocking, tiling, and careful data layout, is central to fast numerical code.
The hierarchy
- Registers: fastest, inside the processor.
- L1, L2, L3 caches: progressively larger and slower.
- Main memory (RAM): large but comparatively slow.
- Caches exploit locality: reuse in time and in nearby addresses.
Why it matters
Processor speed has outpaced memory speed, so many programs are limited by how well they use the cache rather than by arithmetic. Data layouts and access patterns that keep the working set in cache can improve performance by an order of magnitude.
Fusion connection
Cache-aware data layouts in Kronos simulation kernels keep the active portion of the plasma grid in fast memory, cutting the memory-bandwidth stalls that otherwise dominate runtime.