Exploring Speculative Techniques to Improve the Memory System Performance

As processor clock speeds have increased along with microarchitectural innovations, the gap between processor and memory performance has become a greater bottleneck and improvements in memory system design have become more important. This dissertation focuses on improving memory performance through the addition of novel functionalities in the memory system. Specifically, we have proposed two different techniques to hide the latency for memory accesses: Incorrect Speculation and Address Correlation. Both techniques, while based on different ideas, try to reduce or eliminate the misses in the data cache either by prefetching or data forwarding. The two techniques of prefetching and data forwarding by correlated addresses are complementary, one trying to bring the data into the cache before it is requested by the processor, and the other trying to forward the data that is already residing in the cache at one or more correlated addresses on a miss of a requested address. The speculated execution of threads in a multithreaded architecture, plus the branch prediction used in each thread execution units, allows many instructions to be executed speculatively, that is, before it is known whether they actually will be needed by the program. We have examined how the load instructions executed on what turn out to be incorrectly executed program paths impact the memory system performance. We have