Wormhole routing techniques for directly connected multicomputer systems

Wormhole routing has emerged as the most widely used switching technique in massively parallel computers. We present a detailed survey of various techniques for enhancing the performance and reliability of wormhole-routing schemes in directly connected networks. We start with an overview of the direct network topologies and a comparison of various switching techniques. Next, the characteristics of the wormhole routing mechanism are described in detail along with the theory behind deadlock-free routing. The performance of routing algorithms depends on the selection of the path between the source and the destination, the network traffic, and the router design. The routing algorithms are implemented in the router chips. We outline the router characteristics and describe the functionality of various elements of the router. Depending on the usage of paths between the source and the destination, routing algorithms are classified as deterministic, fully adaptive, and partially adaptive. We discuss several representative algorithms for all these categories. The algorithms within each category vary in terms of resource requirements and performance under various traffic conditions. The main difference among various adaptive routing schemes is the technique used to avoid deadlocks. We also discuss a few algorithms based on deadlock recovery techniques. Along with performance, fault tolerance is essential for message routing in multicomputers, and we thus discuss several fault-tolerant wormhole routing algorithms along with their fault-handling capabilities. These routing schemes enable a message to reach its destination even in the presence of faults in the network. The implementation details of wormhole routing algorithms in contemporary commercial systems are also discussed. We conclude by itemizing several future directions and open issues.

[1]  Lionel M. Ni,et al.  Fault-tolerant wormhole routing in meshes , 1993, FTCS-23 The Twenty-Third International Symposium on Fault-Tolerant Computing.

[2]  Suresh Chalasani,et al.  Fault-Tolerant Wormhole Routing Algorithms for Mesh Networks , 1995, IEEE Trans. Computers.

[3]  Mary K. Vernon,et al.  Performance Analysis of Mesh Interconnection Networks with Deterministic Routing , 1994, IEEE Trans. Parallel Distributed Syst..

[4]  Sudhakar Yalamanchili,et al.  Configurable flow control mechanisms for fault-tolerant routing , 1995, ISCA.

[5]  Anant Agarwal,et al.  Limits on Interconnection Network Performance , 1991, IEEE Trans. Parallel Distributed Syst..

[6]  José Duato,et al.  A theory of fault-tolerant routing in wormhole networks , 1994, Proceedings of 1994 International Conference on Parallel and Distributed Systems.

[7]  Andrew A. Chien,et al.  A Cost and Speed Model for k-ary n-Cube Wormhole Routers , 1998, IEEE Trans. Parallel Distributed Syst..

[8]  Jack Dongarra,et al.  An Introduction to the MPI Standard , 1995 .

[9]  William J. Dally,et al.  Deadlock-Free Adaptive Routing in Multicomputer Networks Using Virtual Channels , 1993, IEEE Trans. Parallel Distributed Syst..

[10]  Cevdet Aykanat,et al.  Routing Algorithms for IBM SP1 , 1994, PCRCW.

[11]  Chita R. Das,et al.  Modeling wormhole routing in a hypercube , 1991, [1991] Proceedings. 11th International Conference on Distributed Computing Systems.

[12]  Vara Varavithya,et al.  An EfficientFault-Tolerant Routing Scheme for Two Dimensional Meshes , 1995 .

[13]  Xiaola Lin,et al.  The Message Flow Model for Routing in Wormhole-Routed Networks , 1995, IEEE Trans. Parallel Distributed Syst..

[14]  Luciano Lenzini,et al.  Architecture and Implementation of RPCNET , 1976, ICCC.

[15]  Loren Schwiebert,et al.  A Necessary and Sufficient Condition for Deadlock-Free Wormhole Routing , 1996, J. Parallel Distributed Comput..

[16]  Pedro López,et al.  Deadlock-Free Adaptive Routing Algorithms for the 3D-Torus: Limitations and Solutions , 1993, PARLE.

[17]  Young-Joo Suh,et al.  Software Based Fault-Tolerant Oblivious Routing in Pipelined Networks , 1995, ICPP.

[18]  Sudhakar Yalamanchili,et al.  A Family of Fault-Tolerant Routing Protocols for Direct Multiprocessor Networks , 1995, IEEE Trans. Parallel Distributed Syst..

[19]  S. Konstantinidou,et al.  Chaos router: architecture and performance , 1991, [1991] Proceedings. The 18th Annual International Symposium on Computer Architecture.

[20]  Chita R. Das,et al.  A Class of Partially Adaptive Routing Algorithms for n_dimensional Meshes , 1993, 1993 International Conference on Parallel Processing - ICPP'93.

[21]  Sheldon B. Akers,et al.  A Group-Theoretic Model for Symmetric Interconnection Networks , 1989, IEEE Trans. Computers.

[22]  Luis Gravano,et al.  Adaptive deadlock- and livelock-free routing with all minimal paths in Torus networks , 1992, SPAA '92.

[23]  José Duato.  A Theory of Fault-Tolerant Routing in Wormhole Networks , 1997, IEEE Trans. Parallel Distributed Syst..

[24]  Andrew A. Chien,et al.  Network Performance under Bimodal Traffic Loads , 1995, J. Parallel Distributed Comput..

[25]  R. E. Kessler,et al.  Cray T3D: a new dimension for Cray Research , 1993, Digest of Papers. Compcon Spring.

[26]  Timothy Mark Pinkston,et al.  An efficient, fully adaptive deadlock recovery scheme: DISHA , 1995, ISCA.

[27]  Xiaola Lin,et al.  The Message Flow Model for Routing in Wormhole-Routed Networks , 1993, 1993 International Conference on Parallel Processing - ICPP'93.

[28]  Andrew A. Chien,et al.  The Cost of Adaptivity and Virtual Lanes in aWormhole Router , 1995 .

[29]  José Duato,et al.  994 International Conference on Parallel Processing a Necessary and Sufficient Condition for Deadlock-free Adaptive Routing in Wormhole Networks , 2022 .

[30]  Suresh Chalasani,et al.  A Framework for Designing Deadlock-Free Wormhole Routing Algorithms , 1996, IEEE Trans. Parallel Distributed Syst..

[31]  José Duato,et al.  A New Theory of Deadlock-Free Adaptive Routing in Wormhole Networks , 1993, IEEE Trans. Parallel Distributed Syst..

[32]  Dhiraj K. Pradhan,et al.  The De Bruijn Multiprocessor Network: A Versatile Parallel Processing and Sorting Network for VLSI , 1989, IEEE Trans. Computers.

[33]  José Duato,et al.  Scouting: fully adaptive, deadlock-free routing in faulty pipelined networks , 1994, Proceedings of 1994 International Conference on Parallel and Distributed Systems.

[34]  Andrew A. Chien,et al.  Compressionless routing: a framework for adaptive and fault-tolerant routing , 1994, ISCA '94.

[35]  José Duato.  A Necessary and Sufficient Condition for Deadlock-Free Adaptive Routing in Wormhole Networks , 1995, IEEE Trans. Parallel Distributed Syst..

[36]  Kang G. Shin,et al.  Adaptive Deadlock-Free Routing in Multicomputers Using Only One Extra Virtual Channel , 1993, 1993 International Conference on Parallel Processing - ICPP'93.

[37]  Thorsten von Eicken,et al.  技術解説 IEEE Computer , 1999 .

[38]  Lionel M. Ni,et al.  A survey of wormhole routing techniques in direct networks , 1993, Computer.

[39]  A. A. Chein,et al.  A cost and speed model for k-ary n-cube wormhole routers , 1998 .

[40]  Philip K. McKinley,et al.  Collective Communication in Wormhole-Routed Massively Parallel Computers , 1995, Computer.

[41]  Prasant Mohapatra,et al.  A Traffic-Balanced Adaptive Wormhole-Routing Scheme for Two-Dimensional Meshes , 1997, IEEE Trans. Computers.

[42]  Daniel H. Linder,et al.  An Adaptive and Fault Tolerant Wormhole Routing Strategy for k-Ary n-Cubes , 1994, IEEE Trans. Computers.

[43]  Andrew A. Chien,et al.  Planar-adaptive routing: low-cost adaptive networks for multiprocessors , 1992, ISCA '92.

[44]  T.M. Pinkston,et al.  On Deadlocks In Interconnection Networks , 1997, Conference Proceedings. The 24th Annual International Symposium on Computer Architecture.

[45]  J DallyWilliam,et al.  Performance Analysis of k-ary n-cube Interconnection Networks , 1990 .

[46]  K. Anjan,et al.  An efficient, fully adaptive deadlock recovery scheme: DISHA , 1995, Proceedings 22nd Annual International Symposium on Computer Architecture.

[47]  Tilak Agerwala,et al.  SP2 System Architecture , 1999, IBM Syst. J..

[48]  William J. Dally,et al.  Virtual-channel flow control , 1990, [1990] Proceedings. The 17th Annual International Symposium on Computer Architecture.

[49]  José Duato.  Improving the Efficiency of Virtual Channels with Time-Dependent Selection Functions , 1992, PARLE.

[50]  Chita R. Das,et al.  Efficient fully adaptive wormhole routing in n-dimensional meshes , 1994, 14th International Conference on Distributed Computing Systems.

[51]  William J. Dally,et al.  The Reliable Router: A Reliable and High-Performance Communication Substrate for Parallel Computers , 1994, PCRCW.

[52]  William J. Dally,et al.  Deadlock-Free Message Routing in Multiprocessor Interconnection Networks , 1987, IEEE Transactions on Computers.

[53]  William J. Dally,et al.  Performance Analysis of k-Ary n-Cube Interconnection Networks , 1987, IEEE Trans. Computers.

[54]  Lionel M. Ni,et al.  Performance Evaluation of Switch-Based Wormhole Networks , 1997, IEEE Trans. Parallel Distributed Syst..

[55]  SchwiebertLoren.  A Necessary and Sufficient Condition for Deadlock-Free Wormhole Routing , 1996 .

[56]  Pedro López,et al.  Performance Evaluation of Adaptive Routing Algorithms for k-ary-n-cubes , 1994, PCRCW.

[57]  Steven L. Scott,et al.  Optimized Routing in the Cray T3D , 1994, PCRCW.

[58]  Vara Varavithya,et al.  Routing Algorithms for Torus Networks , 1995 .

[59]  Prasant Mohapatra,et al.  Efficient and balanced adaptive routing in two-dimensional meshes , 1995, Proceedings of 1995 1st IEEE Symposium on High Performance Computer Architecture.

[60]  Michael D. Noakes,et al.  The J-machine multicomputer: an architectural evaluation , 1993, ISCA '93.

[61]  W. Daniel Hillis,et al.  The Network Architecture of the Connection Machine CM-5 , 1996, J. Parallel Distributed Comput..

[62]  Luis Gravano,et al.  Adaptive Deadlock- and Livelock-Free Routing with All Minimal Paths in Torus Networks , 1994, IEEE Trans. Parallel Distributed Syst..

[63]  Joydeep Ghosh,et al.  A Comprehensive Analytical Model for Wormhole Routng in Multicomputer Systems , 1994, J. Parallel Distributed Comput..

[64]  Lionel M. Ni,et al.  Adaptive routing in mesh-connected networks , 1992, [1992] Proceedings of the 12th International Conference on Distributed Computing Systems.

[65]  C. R. Jesshope,et al.  High performance communications in processor networks , 1989, ISCA '89.

[66]  W. Daniel Hillis,et al.  The network architecture of the Connection Machine CM-5 (extended abstract) , 1992, SPAA '92.

[67]  Loren Schwiebert,et al.  Optimal Fully Adaptive Minimal Wormhole Routing for Meshes , 1995, J. Parallel Distributed Comput..

[68]  Inder S. Gopal.  Prevention of Store-and-Forward Deadlock in Computer Networks , 1985, IEEE Trans. Commun..

[69]  Chris R. Jesshope,et al.  High Performance Communications In Processor Networks , 1989, The 16th Annual International Symposium on Computer Architecture.

[70]  Dharma P. Agrawal,et al.  Efficient deadlock-free wormhole routing in shuffle based networks , 1995, Proceedings.Seventh IEEE Symposium on Parallel and Distributed Processing.

[71]  Kang G. Shin,et al.  Support for Multiple Classes of Traffic in Multicomputer Routers , 1994, PCRCW.

[72]  Ran Libeskind-Hadas,et al.  Origin-based fault-tolerant routing in the mesh , 1995, Future Gener. Comput. Syst..

[73]  Anoop Gupta,et al.  The Stanford Dash multiprocessor , 1992, Computer.

[74]  Leonard Kleinrock,et al.  Virtual Cut-Through: A New Computer Communication Switching Technique , 1979, Comput. Networks.

[75]  Suresh Chalasani,et al.  A comparison of adaptive wormhole routing algorithms , 1993, ISCA '93.

[76]  José Duato.  Improving the efficiency of virtual channels with time-dependent selection functions , 1994, Future Gener. Comput. Syst..

[77]  José Duato,et al.  Generalized theory for deadlock-free adaptive wormhole routing and its application to Disha Concurrent , 1996, Proceedings of International Conference on Parallel Processing.