Hierarchical Coherency Management for Shared Virtual Memory Multicomputers

Abstract For many applications of scalable multicomputers with distributed memory it is desirable to provide transparent shared virtual memory. For such applications, the hardware and system software must maintain coherency among the local memories. Most existing coherency schemes for multicomputers manage memory uniformly at a single granularity of fixed size pages or cache blocks, leading to unnecessarily high storage overhead and poor performance. Coherency management at the granularity of pages (thousands of bytes) results in unnecessary network traffic. At the granularity of cache blocks (tens of bytes), unacceptably large mapping tables are needed for large systems. We propose a solution to this problem using hierarchical management, where mapping and transfer at the block level are done only when necessary—during the time that the page is actually shared. When pages are not shared, they do not require more space for mapping tables than uniform page-level schemes. A detailed description of the scheme is presented, together with an evaluation of its performance and area overheads. The results of trace-driven simulations of the hierarchical coherency scheme are reported. It is shown that the hierarchical scheme makes it possible to achieve the performance advantage of uniform block-based schemes without their large storage overhead.