Exploiting Load Latency Tolerance in Dynamically Scheduled Processors

This paper provides quantitative measurements of load latency tolerance in a dynamically scheduled processor and presents one cache management technique that exploits this information to improve overall performance. We determine the latency of each memory load operation such that the number of instructions issued per cycle (IPC) is comparable to an ideal memory system that satisfies all requests in a single cycle. Our measurements reveal that to produce IPC values within 16% of the ideal memory system, between 50% and 90% of loads need to be satisfied within a single cycle and that up to 50% can be satisfied in as many as 8 cycles (an artificially imposed upper limit), depending on the benchmark and processor configuration. Load latency tolerance is largely determined by the number of dependent operations and whether a branch is dependent on the load. This paper presents an all hardware approach to obtain this information and to utilize it in determining cache replacement decisions. Simulation results indicate this technique can improve IPC values by up to 8%.