Tools and techniques for memory system design and analysis

As processor cycle times decrease, memory system performance becomes ever more critical to overall performance. Continually changing technology and workloads create a moving target for computer architects in their effort to design cost-effective memory systems. Meeting the demands of ever changing workloads and technology requires the following: (1) Efficient techniques for evaluating memory system performance, (2) Tuning programs to better use the memory system, and (3) New memory system designs. This thesis makes contributions in each of these areas. Hardware and software developers rely on simulation to evaluate new ideas. In this thesis, I present a new interface for writing memory system simulators--the active memory abstraction--designed specifically for simulators that process memory references as the application executes and avoids storing them to tape or disk. Active memory allows simulators to optimize for the common case, e.g., cache hits, achieving simulation times only 2-6 times slower than the original uninstrumented application. The efficiency of the active memory abstraction can be used by software designers to obtain information about their program's memory system behavior--called a cache profile. In this thesis, using the CProf cache profiling system, I show that cache profiling is an effective means of improving uniprocessor program performance by focusing a programmer's attention on problematic code sections and providing insight into the type of program transformation to apply. Execution time speedups for the programs studied range from 1.02 to 3.46, depending on the machine's memory system. The third contribution of this thesis is dynamic self-invalidation (DSI), a new technique for reducing coherence overhead in shared-memory multiprocessors. The fundamental concept of DSI is for processors to automatically replace a block from their cache before another processor wants to access the block, allowing the directory to immediately respond with the data. My results show that, under sequential consistency DSI can reduce execution time by as much as 41% and under weak consistency by as much as 18%. Under weak consistency, DSI can also exploit tear-off blocks--which eliminate both invalidation and acknowledgment messages--for a total reduction in messages of up to 26%.