Memory-Side Prefetching for Linked Data Structures

This paper studies a memory-side prefetching technique to hide latency incurred by inherently serial accesses to linked data structures (LDS). A programmable prefetch engine sits close to memory and traverses LDS independently from the processor. The prefetch engine can run ahead of the processor because of its low latency, high bandwidth path to memory. This allows the prefetch engine to initiate data transfers earlier than the processor and pipeline multiple such transfers over the network. We evaluate the proposed memory-side prefetching scheme for the pointer-intensive Olden benchmark suite, comparing both to a system without any prefetching and one with a state-of-the-art processor-side software prefetching scheme for LDS. For the six benchmarks where LDS memory stall time is significant, the memory-side scheme reduces execution time by an average of 27\% (range of 0\% to 62\%) compared to a system without any prefetching. Compared to processor-side prefetching, the memory-side scheme reduces execution time in the range of 20\% to 50\% for three of the six applications, is about the same for two applications, and is worse by 18\% for one application. We conclude that memory-side prefetching is effective, but a combination of processor- and memory-side prefetching is best and provide a qualitative framework to determine when either scheme should be used. Our results differ from a previous memory-side prefetching study in significant ways, primarily because we perform our comparisons with a state-of-the-art processor-side scheme.