HARAQ: Congestion-Aware Learning Model for Highly Adaptive Routing Algorithm in On-Chip Networks

The occurrence of congestion in on-chip networks can severely degrade the performance due to increased message latency. In mesh topology, minimal methods can propagate messages over two directions at each switch. When shortest paths are congested, sending more messages through them can deteriorate the congestion condition considerably. In this paper, we present an adaptive routing algorithm for on-chip networks that provide a wide range of alternative paths between each pair of source and destination switches. Initially, the algorithm determines all permitted turns in the network including 180-degree turns on a single channel without creating cycles. The implementation of the algorithm provides the best usage of all allowable turns to route messages more adaptively in the network. On top of that, for selecting a less congested path, an optimized and scalable learning method is utilized. The learning method is based on local and global congestion information and can estimate the latency from each output channel to the destination region.

[1]  Vijay Laxmi,et al.  C-Routing: An adaptive hierarchical NoC routing methodology , 2011, 2011 IEEE/IFIP 19th International Conference on VLSI and System-on-Chip.

[2]  Hannu Tenhunen,et al.  Memory-Efficient On-Chip Network With Adaptive Interfaces , 2012, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[3]  Andrew A. Chien,et al.  Planar-adaptive routing: low-cost adaptive networks for multiprocessors , 1992, ISCA '92.

[4]  Ming Li,et al.  DyXY - a proximity congestion-aware deadlock-free dynamic routing method for network on chip , 2006, 2006 43rd ACM/IEEE Design Automation Conference.

[5]  Stephen W. Keckler,et al.  Regional congestion awareness for load balance in networks-on-chip , 2008, 2008 IEEE 14th International Symposium on High Performance Computer Architecture.

[6]  Taisook Han,et al.  Fault-Tolerant Adaptive Wormhole Routing in 2D Mesh , 1998 .

[7]  Siamak Mohammadi,et al.  Adaptive Input-Output Selection Based On-Chip Router Architecture , 2012, J. Low Power Electron..

[8]  Michael L. Littman,et al.  Packet Routing in Dynamically Changing Networks: A Reinforcement Learning Approach , 1993, NIPS.

[9]  Alain Greiner,et al.  A reconfigurable routing algorithm for a fault-tolerant 2D-Mesh Network-on-Chip , 2008, 2008 45th ACM/IEEE Design Automation Conference.

[10]  Milo M. K. Martin,et al.  Multifacet's general execution-driven multiprocessor simulator (GEMS) toolset , 2005, CARN.

[11]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[12]  Chi-Kwong Li,et al.  An approach to tune fuzzy controllers based on reinforcement learning for autonomous vehicle control , 2005, IEEE Transactions on Intelligent Transportation Systems.

[13]  Nacer-Eddine Zergainoh,et al.  A fault-tolerant deadlock-free adaptive routing for on chip interconnects , 2011, 2011 Design, Automation & Test in Europe.

[14]  Natalie D. Enright Jerger,et al.  DBAR: An efficient routing algorithm to support multiple concurrent applications in networks-on-chip , 2011, 2011 38th Annual International Symposium on Computer Architecture (ISCA).

[15]  Jürgen Teich,et al.  Packet routing in dynamically changing networks on chip , 2005, 19th IEEE International Parallel and Distributed Processing Symposium.

[16]  Vincenzo Catania,et al.  Implementation and Analysis of a New Selection Strategy for Adaptive Routing in Networks-on-Chip , 2008, IEEE Transactions on Computers.

[17]  Richard Sutton,et al.  Learning to Maximize Rewards: A Review of "Reinforcement Learning: An Introduction , 2000 .

[18]  Anoop Gupta,et al.  The SPLASH-2 programs: characterization and methodological considerations , 1995, ISCA.

[19]  Camel Tanougast,et al.  A new deadlock-free fault-tolerant routing algorithm for NoC interconnections , 2009, 2009 International Conference on Field Programmable Logic and Applications.

[20]  Masoud Dehyadegari,et al.  An adaptive fuzzy logic-based routing algorithm for networks-on-chip , 2011, 2011 NASA/ESA Conference on Adaptive Hardware and Systems (AHS).

[21]  Shailesh Kumar and Risto Miikkulainen.  Dual Reinforcement Q-Routing: An On-Line Adaptive Routing Algorithm , 1997 .

[22]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[23]  Chung-Ta King,et al.  Turn Grouping for Multicast in Wormhole-Routed Mesh Networks Supporting the Turn Model , 2004, The Journal of Supercomputing.

[24]  Jie Wu,et al.  Fault-tolerant and deadlock-free routing in 2-D meshes using rectilinear-monotone polygonal fault blocks , 2005, Parallel Algorithms Appl..

[25]  Axel Jantsch,et al.  A reconfigurable fault-tolerant deflection routing algorithm based on reinforcement learning for network-on-chip , 2010, NoCArc '10.

[26]  David A. Wood,et al.  Managing Wire Delay in Large Chip-Multiprocessor Caches , 2004, 37th International Symposium on Microarchitecture (MICRO-37'04).

[27]  David Blaauw,et al.  A highly resilient routing algorithm for fault-tolerant NoCs , 2009, 2009 Design, Automation & Test in Europe Conference & Exhibition.