Logical time coherence maintenance

We apply techniques based on isotach logical time to the problem of maintaining a coherent shared memory. In isotach logical time systems, processes can predict and control the logical times at which their messages are received. This control over the logical receive time of messages provides a powerful basis for implementing coherence protocols. Existing isotach-based memory coherence protocols are more concurrent than other protocols, but are limited in the topologies on which they work and the reference patterns for which they are suited. We define a new framework for isotach shared memory systems that supports protocols that work for arbitrary topologies and are suited to a wide range of reference patterns. By extending isotach protocols to a wider class of applications and networks, we contribute to the solution of the memory coherence problem. In addition to extending isotach-based coherence protocols, we advance the theory of isotach systems. We redefine isotach systems to be consistent with potential causality, a new relation among events that captures causality in a less conservative way than Lamport's happens before relation. This redefinition expands the class of correct implementations of isotach systems. We introduce the flex algorithm, a new implementation of isotach logical time that allows different links to be assigned different logical distances. We expect that increased flexibility in assigning logical distances will improve the performance of isotach systems in cases in which links have significantly different real time latencies.

[1]  Kanad Ghose,et al.  A Cache Coherency Mechanism with Limited Combining Capabilities for MIN-Based Multiprocessors , 1991, ICPP.

[2]  Divyakant Agrawal,et al.  Type-specific coherence protocols for distributed shared memory , 1992, [1992] Proceedings of the 12th International Conference on Distributed Computing Systems.

[3]  Alan L. Cox,et al.  Evaluation of release consistent software distributed shared memory on emerging network technology , 1993, ISCA '93.

[4]  Vivek Khera,et al.  An Architecture-Independent Analysis of False Sharing , 1993 .

[5]  Joonwon Lee,et al.  Cache-Based Synchronization in Shared Memory Multiprocessors , 1996, J. Parallel Distributed Comput..

[6]  Bronis R. de Supinski,et al.  Performance Evaluation of the Late Delta Cache Coherence Protocol , 1996 .

[7]  Alan L. Cox,et al.  Software versus hardware shared-memory implementation: a case study , 1994, ISCA '94.

[8]  Alexander Thomasian,et al.  Database Concurrency Control , 1996, The Springer International Series on Advances in Database Systems.

[9]  S.K. Reinhardt,et al.  Decoupled Hardware Support for Distributed Shared Memory , 1996, 23rd Annual International Symposium on Computer Architecture (ISCA'96).

[10]  Randy H. Katz,et al.  Implementing a cache consistency protocol , 1985, ISCA 1985.

[11]  Sang Lyul Min,et al.  Design and Analysis of a Scalable Cache Coherence Scheme Based on Clocks and Timestamps , 1992, IEEE Trans. Parallel Distributed Syst..

[12]  Shreekant S. Thakkar,et al.  Synchronization algorithms for shared-memory multiprocessors , 1990, Computer.

[13]  Ajay D. Kshemkalyani,et al.  An Efficient Implementation of Vector Clocks , 1992, Inf. Process. Lett..

[14]  Janak H. Patel,et al.  Memory Reference Behavior of Compiler Optimized Programs on High Speed Architectures , 1993, 1993 International Conference on Parallel Processing - ICPP'93.

[15]  Anoop Gupta,et al.  Reducing Memory and Traffic Requirements for Scalable Directory-Based Cache Coherence Schemes , 1990, ICPP.

[16]  Brett D. Fleisch,et al.  Mirage: a coherent distributed shared memory design , 1989, SOSP '89.

[17]  Abhiram G. Ranade,et al.  How to emulate shared memory (Preliminary Version) , 1987, FOCS.

[18]  James R. Larus,et al.  Mechanisms for cooperative shared memory , 1993, ISCA '93.

[19]  Mukesh Singhal,et al.  Logical Time: Capturing Causality in Distributed Systems , 1996, Computer.

[20]  Alexander V. Veidenbaum,et al.  A cache coherence scheme with fast selective invalidation , 1988, ISCA '88.

[21]  Anant Agarwal,et al.  LimitLESS directories: A scalable cache coherence scheme , 1991, ASPLOS IV.

[22]  Anant Agarwal,et al.  Evaluating the performance of software cache coherence , 1989, ASPLOS 1989.

[23]  Mark Horowitz,et al.  An evaluation of directory schemes for cache coherence , 1998, ISCA '98.

[24]  Michael L. Scott,et al.  False sharing and its effect on shared memory performance , 1993 .

[25]  Friedemann Mattern,et al.  Efficient Algorithms for Distributed Snapshots and Global Virtual Time Approximation , 1993, J. Parallel Distributed Comput..

[26]  David R. Cheriton,et al.  Preliminary thoughts on problem-oriented shared memory: a decentralized approach to distributed systems , 1985, OPSR.

[27]  Willy Zwaenepoel,et al.  Adaptive software cache management for distributed shared memory architectures , 1990, ISCA '90.

[28]  Paul Hudak,et al.  Memory coherence in shared virtual memory systems , 1989, TOCS.

[29]  Willy Zwaenepoel,et al.  Munin: distributed shared memory based on type-specific memory coherence , 1990, PPOPP '90.

[30]  Alan L. Cox,et al.  An integrated compile-time/run-time software distributed shared memory system , 1996, ASPLOS VII.

[31]  Laxmi N. Bhuyan,et al.  Design and Analysis of Cache Coherent Multistage Interconnection Networks , 1993, IEEE Trans. Computers.

[32]  Anant Agarwal,et al.  Software-extended coherent shared memory: performance and cost , 1994, ISCA '94.

[33]  Michael L. Scott,et al.  Distributed Shared Memory for New Generation Networks , 1995 .

[34]  Sarita V. Adve,et al.  Shared Memory Consistency Models: A Tutorial , 1996, Computer.

[35]  André Schiper,et al.  Lightweight causal and atomic group multicast , 1991, TOCS.

[36]  David L. Black,et al.  Competitive algorithms for replication and migration problems , 1989 .

[37]  Michel Raynal,et al.  An Efficient Implementation of Sequentially Consistent Distributed Shared Memories , 1994, Applications in Parallel and Distributed Computing.

[38]  H. Grahn,et al.  Efficient strategies for software-only directory protocols in shared-memory multiprocessors , 1995, Proceedings 22nd Annual International Symposium on Computer Architecture.

[39]  Mary K. Vernon,et al.  Efficient synchronization primitives for large-scale cache-coherent multiprocessors , 1989, ASPLOS 1989.

[40]  Leslie Lamport,et al.  Time, clocks, and the ordering of events in a distributed system , 1978, CACM.

[41]  James Archibald,et al.  An economical solution to the cache coherence problem , 1984, ISCA 1984.

[42]  James R. Larus,et al.  Application-specific protocols for user-level shared memory , 1994, Proceedings of Supercomputing '94.

[43]  Sandeep N. Bhatt,et al.  The fluent abstract machine , 1988 .

[44]  Michael L. Scott,et al.  A Trace-Based Comparison of Shared Memory Multiprocessor Architectures , 1992 .

[45]  Lawrence C. Stewart,et al.  Firefly: a multiprocessor workstation , 1987, ASPLOS 1987.

[46]  Jeffery R. Westbrook.  Randomized Algorithms for Multiprocessor Page Migration , 1994, SIAM J. Comput..

[47]  Timothy Mark Pinkston,et al.  Parallel Processor Memory Reference Analysis: Examining Locality and Clustering Potential , 1991, PPSC.

[48]  A. Richard Newton,et al.  An empirical evaluation of two memory-efficient directory methods , 1990, ISCA '90.

[49]  Leslie Lamport,et al.  How to Make a Multiprocessor Computer That Correctly Executes Multiprocess Programs , 2016, IEEE Transactions on Computers.

[50]  Baruch Awerbuch,et al.  Complexity of network synchronization , 1985, JACM.

[51]  Thomas J. LeBlanc,et al.  Adjustable block size coherent caches , 1992, ISCA '92.

[52]  James R. Goodman,et al.  Cache Consistency and Sequential Consistency , 1991 .

[53]  Michel Dubois,et al.  Combined performance gains of simple cache protocol extensions , 1994, ISCA '94.

[54]  Paul F. Reynolds,et al.  Isotach Networks , 1997, IEEE Trans. Parallel Distributed Syst..

[55]  K. Mani Chandy,et al.  Distributed Simulation: A Case Study in Design and Verification of Distributed Programs , 1979, IEEE Transactions on Software Engineering.

[56]  D. B. Lomet.  Process structuring, synchronization, and recovery using atomic actions , 1977 .

[57]  Mustaque Ahamad,et al.  Implementing and programming causal distributed shared memory , 1991, [1991] Proceedings. 11th International Conference on Distributed Computing Systems.

[58]  Livio Ricciulli,et al.  The detection and elimination of useless misses in multiprocessors , 1993, ISCA '93.

[59]  Yehuda Afek,et al.  Lazy caching , 1993, TOPL.

[60]  Kai Li,et al.  Understanding Application Performance on Shared Virtual Memory Systems , 1996, 23rd Annual International Symposium on Computer Architecture (ISCA'96).

[61]  Henri E. Bal,et al.  Parallel programming using shared objects and broadcasting , 1992, Computer.

[62]  Abhinav Gupta,et al.  Analysis of cache invalidation patterns in multiprocessors , 1989, ASPLOS 1989.

[63]  Kirk L. Johnson,et al.  CRL: high-performance all-software distributed shared memory , 1995, SOSP.

[64]  Pen-Chung Yew,et al.  Compiler and Hardware Support for Cache Coherence in Large-Scale Multiprocessors: Design Considerations and Performance Study , 1996, 23rd Annual International Symposium on Computer Architecture (ISCA'96).

[65]  Carolyn Craig Williams,et al.  Concurrency control in asynchronous computations , 1993 .

[66]  John L. Hennessy,et al.  SoftFLASH: analyzing the performance of clustered distributed virtual shared memory , 1996, ASPLOS VII.

[67]  Philip J. Woest,et al.  The Wisconsin multicube: a new large-scale cache-coherent multiprocessor , 1988, ISCA '88.

[68]  Kourosh Gharachorloo,et al.  Shasta: a low overhead, software-only approach for supporting fine-grain shared memory , 1996, ASPLOS VII.

[69]  Amos Fiat,et al.  Competitive distributed file allocation , 1993, STOC '93.

[70]  Ricardo Bianchini,et al.  Hiding communication latency and coherence overhead in software DSMs , 1996, ASPLOS VII.

[71]  James Lyle Peterson,et al.  Petri net theory and the modeling of systems , 1981 .

[72]  Yuval Tamir,et al.  Hierarchical Coherency Management for Shared Virtual Memory Multicomputers , 1992, J. Parallel Distributed Comput..

[73]  Robert J. Fowler,et al.  A performance evaluation of optimal hybrid cache coherency protocols , 1992, ASPLOS V.

[74]  A. Agarwal,et al.  MGS: A Multigrain Shared Memory System , 1996, 23rd Annual International Symposium on Computer Architecture (ISCA'96).

[75]  Robert J. Fowler,et al.  The implementation of a coherent memory abstraction on a NUMA multiprocessor: experiences with platinum , 1989, SOSP '89.

[76]  Richard P. LaRowe,et al.  Hiding Shared Memory Reference Latency on the Galactica Net Distributed Shared Memory Architecture , 1992, J. Parallel Distributed Comput..

[77]  Philip Heidelberger,et al.  Multiple reservations and the Oklahoma update , 1993, IEEE Parallel & Distributed Technology: Systems & Applications.

[78]  Roy Friedman,et al.  Using Virtual Synchrony to Develop Efficient Fault Tolerant Distributed Shared Memories , 1995 .

[79]  Sarita V. Adve,et al.  Designing memory consistency models for shared-memory multiprocessors , 1993 .

[80]  Bernadette Charron-Bost,et al.  Concerning the Size of Logical Clocks in Distributed Systems , 1991, Inf. Process. Lett..

[81]  Abhiram G. Ranade,et al.  Fluent parallel computation , 1989 .

[82]  Harjinder S. Sandhu,et al.  The shared regions approach to software cache coherence on multiprocessors , 1993, PPOPP '93.

[83]  Willy Zwaenepoel,et al.  Implementation and performance of Munin , 1991, SOSP '91.

[84]  Erik Hagersten,et al.  The Cache Coherence Protocol of the Data Diffusion Machine , 1989, PARLE.

[85]  Michel Dubois,et al.  Memory access buffering in multiprocessors , 1998, ISCA '98.

[86]  Susan J. Eggers,et al.  Eliminating False Sharing , 1991, ICPP.

[87]  Randall L. Hyde,et al.  An Analysis of Degenerate Sharing and False Coherence , 1996, J. Parallel Distributed Comput..

[88]  Yuval Rabani,et al.  Competitive Algorithms for Distributed Data Management , 1995, J. Comput. Syst. Sci..

[89]  Umakishore Ramachandran,et al.  The Quest for a Zero Overhead Shared Memory Parallel Machine , 1995, ICPP.

[90]  Yuval Rabani,et al.  Competitive algorithms for distributed data management (extended abstract) , 1992, STOC '92.

[91]  Robert J. Fowler,et al.  Adaptive cache coherency for detecting migratory shared data , 1993, ISCA '93.

[92]  Alan L. Cox,et al.  TreadMarks: shared memory computing on networks of workstations , 1996 .

[93]  Liviu Iftode,et al.  Improving release-consistent shared virtual memory using automatic update , 1996, Proceedings. Second International Symposium on High-Performance Computer Architecture.

[94]  Alvin M. Despain,et al.  Multiprocessor cache synchronization: issues, innovations, evolution , 1986, ISCA 1986.

[95]  Larry Rudolph,et al.  Dynamic decentralized cache schemes for mimd parallel processors , 1984, ISCA 1984.

[96]  David V. James,et al.  SCI (Scalable Coherent Interface) Cache Coherence , 1990 .

[97]  Roy Friedman,et al.  A Correctness Condition for High-Performance Multiprocessors , 1998, SIAM J. Comput..

[98]  Robert E. Tarjan,et al.  Amortized efficiency of list update and paging rules , 1985, CACM.

[99]  Alan L. Cox,et al.  Lazy release consistency for software distributed shared memory , 1992, ISCA '92.

[100]  Paul F. Reynolds,et al.  Combining Atomic Actions , 1995, J. Parallel Distributed Comput..

[101]  David J. Lilja,et al.  The Potential of Compile-Time Analysis to Adapt the Cache Coherence Enforcement Strategy to the Data Sharing Characteristics , 1995, IEEE Trans. Parallel Distributed Syst..

[102]  John Regehr.  An Isotach Implementation for Myrinet , 1997 .

[103]  Laxmi N. Bhuyan,et al.  Design of an Adaptive Cache Coherence Protocol for Large Scale Multiprocessors , 1992, IEEE Trans. Parallel Distributed Syst..

[104]  Thomas J. LeBlanc,et al.  Eliminating Useless Messages in Write-Update Protocols on Scalable Multiprocessors , 1994 .

[105]  Partha Dasgupta,et al.  The Clouds distributed operating system: functional description, implementation details and related work , 1988, [1988] Proceedings. The 8th International Conference on Distributed.

[106]  André Schiper,et al.  From Causal Consistency to Sequential Consistency in Shared Memory Systems , 1995, FSTTCS.

[107]  Paul Feautrier,et al.  A New Solution to Coherence Problems in Multicache Systems , 1978, IEEE Transactions on Computers.

[108]  Michael J. Flynn,et al.  Update-based cache coherence protocols for scalable shared-memory multiprocessors , 1994, 1994 Proceedings of the Twenty-Seventh Hawaii International Conference on System Sciences.

[109]  Liviu Iftode,et al.  Performance evaluation of two home-based lazy release consistency protocols for shared virtual memory systems , 1996, OSDI '96.

[110]  Mustaque Ahamad,et al.  Slow memory: weakening consistency to enhance concurrency in distributed shared memories , 1990, Proceedings.,10th International Conference on Distributed Computing Systems.

[111]  Kourosh Gharachorloo,et al.  Memory consistency models for shared-memory multiprocessors , 1995 .

[112]  Janak H. Patel,et al.  A low-overhead coherence solution for multiprocessors with private cache memories , 1984, ISCA '84.

[113]  Jorge L. C. Sanz,et al.  A Simple Mechanism for Efficient Barrier Synchronization in MIMD Machines , 1990, ICPP.

[114]  James R. Goodman.  Using cache memory to reduce processor-memory traffic , 1998, ISCA '98.

[115]  Anoop Gupta,et al.  The SPLASH-2 programs: characterization and methodological considerations , 1995, ISCA.

[116]  Josep Torrellas,et al.  False Sharing ans Spatial Locality in Multiprocessor Caches , 1994, IEEE Trans. Computers.

[117]  Mats Brorsson,et al.  An adaptive cache coherence protocol optimized for migratory sharing , 1993, ISCA '93.

[118]  Anoop Gupta,et al.  Memory-reference characteristics of multiprocessor applications under MACH , 1988, SIGMETRICS 1988.

[119]  Hagit Attiya,et al.  Sequential consistency versus linearizability , 1994, TOCS.

[120]  Jih-Kwon Peir,et al.  Inter-Section Locality of Shared Data in Parallel Programs , 1991, ICPP.

[121]  David R. Jefferson,et al.  Virtual time , 1985, ICPP.

[122]  Anoop Gupta,et al.  Competitive management of distributed shared memory , 1989, Digest of Papers. COMPCON Spring 89. Thirty-Fourth IEEE Computer Society International Conference: Intellectual Leverage.

[123]  David Abramson,et al.  Implementing a large virtual memory in a Distributed Computing , 1985 .

[124]  Anoop Gupta,et al.  Memory consistency and event ordering in scalable shared-memory multiprocessors , 1990, ISCA '90.

[125]  James R. Larus,et al.  Fine-grain access control for distributed shared memory , 1994, ASPLOS VI.

[126]  Leslie Lamport,et al.  Proving Liveness Properties of Concurrent Programs , 1982, TOPL.

[127]  Colin J. Fidge,et al.  Logical time in distributed computing systems , 1991, Computer.

[128]  William W. Collier,et al.  Reasoning about parallel architectures , 1992 .

[129]  Mustaque Ahamad,et al.  Evaluation of Causal Distributed Shared Memory for Data-race-free Programs , 1994 .

[130]  Brian N. Bershad,et al.  Midway : shared memory parallel programming with entry consistency for distributed memory multiprocessors , 1991 .