The Cray T3E Network: Adaptive Routing in a High Performance 3D Torus

This paper describes the interconnection network used in the Cray T3E multiprocessor. The network is a bidirectional 3D torus with fully adaptive routing, optimized virtual channel assignments, integrated barrier synchronization support and considerable fault tolerance. The routers are built with LSI’s 500K ASIC technology with custom transmitters/ receivers driving low-voltage differential signals at 375 MHz, for a link data payload capacity of approximately 500 MB/s.

[1]  DAVID GELERNTER.  A DAG-Based Algorithm for Prevention of Store-and-Forward Deadlock in Packet Networks , 1981, IEEE Transactions on Computers.

[2]  Inder S. Gopal.  Prevention of Store-and-Forward Deadlock in Computer Networks , 1985, IEEE Trans. Commun..

[3]  William J. Dally,et al.  Deadlock-Free Message Routing in Multiprocessor Interconnection Networks , 1987, IEEE Transactions on Computers.

[4]  Charles L. Seitz,et al.  A Framework for Adaptive Routing , 1987 .

[5]  Chris R. Jesshope,et al.  High Performance Communications In Processor Networks , 1989, The 16th Annual International Symposium on Computer Architecture.

[6]  William J. Dally,et al.  Virtual-channel flow control , 1990, [1990] Proceedings. The 17th Annual International Symposium on Computer Architecture.

[7]  Daniel H. Linder,et al.  An Adaptive and Fault Tolerant Wormhole Routing Strategy for k-Ary n-Cubes , 1994, IEEE Trans. Computers.

[8]  Charles L. Seitz,et al.  A framework for adaptive routing in multicomputer networks , 1989, CARN.

[9]  Anant Agarwal,et al.  Limits on Interconnection Network Performance , 1991, IEEE Trans. Parallel Distributed Syst..

[10]  Kevin Bolding.  Non-Uniformities Introduced by Virtual Channel Deadlock Prevention , 1992 .

[11]  Andrew A. Chien,et al.  Planar-adaptive routing: low-cost adaptive networks for multiprocessors , 1992, ISCA '92.

[12]  José Duato,et al.  A New Theory of Deadlock-Free Adaptive Routing in Wormhole Networks , 1993, IEEE Trans. Parallel Distributed Syst..

[13]  Steven L. Scott,et al.  Optimized Routing in the Cray T3D , 1994, PCRCW.

[14]  Steven L. Scott,et al.  Synchronization and communication in the T3E multiprocessor , 1996, ASPLOS VII.

[15]  The Turn Model for Adaptive Routing , 1998, 25 Years ISCA: Retrospectives and Reprints.