Using recurrent neural networks to learn the structure of interconnection networks

A modified Recurrent Neural Network (RNN) is used to learn a Self-Routing Interconnection Network (SRIN) from a set of routing examples. The RNN is modified so that it has several distinct initial states. This is equivalent to a single RNN learning multiple different synchronous sequential machines. We define such a sequential machine structure as augmented and show that a SRIN is essentially an Augmented Synchronous Sequential Machine (ASSM). As an example, we learn a small six-switch SRIN. After training we extract the network's internal representation of the ASSM and corresponding SRIN.

[1]  Mark William Goudreau.  Neural network applications for interconnection networks , 1993 .

[2]  Ronald L. Rivest,et al.  Diversity-based inference of finite automata , 1994, 28th Annual Symposium on Foundations of Computer Science (sfcs 1987).

[3]  Srimat T. Chakradhar,et al.  First-order versus second-order single-layer recurrent neural networks , 1994, IEEE Trans. Neural Networks.

[4]  Yoshiyasu Takefuji,et al.  A neural network model for traffic controls in multistage interconnection networks , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[5]  W. Daniel Hillis,et al.  Co-evolving parasites improve simulated evolution as an optimization procedure , 1990 .

[6]  A. Marrakchi,et al.  A neural net arbitrator for large crossbar packet-switches , 1989 .

[7]  Kevin J. Lang.  Random DFA's can be approximately learned from sparse uniform examples , 1992, COLT '92.

[8]  Michael C. Mozer,et al.  Discovering the Structure of a Reactive Environment by Exploration , 1990, Neural Computation.

[9]  M. Goudreau,et al.  First-order vs. Second-order Single Layer Recurrent Neural Networks , 1994 .

[10]  C. Lee Giles,et al.  Routing in Random Multistage Interconnection Networks , 1994 .

[11]  C. Lee Giles,et al.  Experimental Comparison of the Effect of Order in Recurrent Neural Networks , 1993, Int. J. Pattern Recognit. Artif. Intell..

[12]  C. Lee Giles,et al.  Learning a class of large finite state machines with a recurrent neural network , 1995, Neural Networks.

[13]  Robert E. Schapire,et al.  A new approach to unsupervised learning in deterministic environments , 1990 .

[14]  T.X. Brown,et al.  Neural networks for switching , 1989, IEEE Communications Magazine.

[15]  C. Lee Giles,et al.  Extraction, Insertion and Refinement of Symbolic Rules in Dynamically Driven Recurrent Neural Networks , 1993 .

[16]  Leslie G. Valiant,et al.  Cryptographic limitations on learning Boolean formulae and finite automata , 1994, JACM.

[17]  Raymond L. Watrous,et al.  Induction of Finite-State Languages Using Second-Order Recurrent Networks , 1992, Neural Computation.

[18]  Panagiotis Manolios,et al.  First-Order Recurrent Neural Networks and Deterministic Finite State Automata , 1994, Neural Computation.

[19]  Boris A. Trakhtenbrot,et al.  Finite automata : behavior and synthesis , 1973 .

[20]  N. Z. Hakim,et al.  A neural network approach to set up the Benes switch , 1990, Proceedings. IEEE INFOCOM '90: Ninth Annual Joint Conference of the IEEE Computer and Communications Societies@m_The Multiple Facets of Integration.

[21]  Joachim Diederich,et al.  Survey and critique of techniques for extracting rules from trained artificial neural networks , 1995, Knowl. Based Syst..

[22]  James L. McClelland,et al.  Finite State Automata and Simple Recurrent Networks , 1989, Neural Computation.

[23]  W ShavlikJude,et al.  Using Knowledge-Based Neural Networks to Improve Algorithms , 1993 .

[24]  C. E. Rohrs,et al.  A neural network solutions for routing in three stage interconnection networks , 1990, IEEE International Symposium on Circuits and Systems.

[25]  E. Mark Gold,et al.  Complexity of Automaton Identification from Given Data , 1978, Inf. Control..

[26]  Jeffrey D. Ullman,et al.  Introduction to Automata Theory, Languages and Computation , 1979 .

[27]  DANA ANGLUIN,et al.  On the Complexity of Minimum Inference of Regular Sets , 1978, Inf. Control..

[28]  C. E. Rohrs,et al.  A neural network solution for call routing with preferential call placement , 1990, [Proceedings] GLOBECOM '90: IEEE Global Telecommunications Conference and Exhibition.

[29]  Ronald L. Rivest,et al.  Diversity-Based Inference of Finite Automata (Extended Abstract) , 1987, FOCS.

[30]  Martin T. Hagan,et al.  Neural network design , 1995 .

[31]  C. Lee Giles,et al.  Learning and Extracting Finite State Automata with Second-Order Recurrent Neural Networks , 1992, Neural Computation.

[32]  Howard Jay Siegel,et al.  Interconnection networks for large-scale parallel processing: theory and case studies (2nd ed.) , 1985 .

[33]  Yoshiyasu Takefuji,et al.  Comparisons of Seven Neural Network Models on Traffic Control Problems in Multistage Interconnection Networks , 1993, IEEE Trans. Computers.

[34]  C. L. Giles,et al.  Pruning recurrent neural networks for improved generalization performance , 1994, Proceedings of IEEE Workshop on Neural Networks for Signal Processing.

[35]  S.C.A. Thomopoulos,et al.  Neural network implementation of the shortest path algorithm for traffic routing in communication networks , 1989, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks.

[36]  Stephen M. Walters,et al.  Neural network architecture for crossbar switch control , 1991 .

[37]  C. Lee Giles,et al.  Pruning recurrent neural networks for improved generalization performance , 1994, IEEE Trans. Neural Networks.

[38]  Michael C. Mozer,et al.  SLUG: A Connectionist Architecture for Inferring the Structure of Finite-State Environments , 1991, Mach. Learn..

[39]  Ronald J. Williams,et al.  A Learning Algorithm for Continually Running Fully Recurrent Neural Networks , 1989, Neural Computation.

[40]  Robert E. Schapire,et al.  A new approach to unsupervised learning in deterministic environments , 1990 .

[41]  S.C.A. Thomopoulos,et al.  Neural network implementation of the shortest path algorithm for traffic routing in communication networks , 1989, International 1989 Joint Conference on Neural Networks.

[42]  Su-Ling Lee,et al.  Neural networks for routing of communication networks with unreliable components , 1993, IEEE Trans. Neural Networks.

[43]  Timothy X. Brown,et al.  Neural Network Design of a Banyan Network Controller , 1990, IEEE J. Sel. Areas Commun..

[44]  Padhraic Smyth,et al.  Learning Finite State Machines With Self-Clustering Recurrent Networks , 1993, Neural Computation.

[45]  C. Lee Giles,et al.  Routing in Random Multistage Interconnections Networks: Comparing Exhaustive Search, Greedy and Neural Network Approaches , 1992, Int. J. Neural Syst..

[46]  R. J. Nelson,et al.  Introduction to Automata , 1968 .

[47]  C. Lee Giles,et al.  Extracting and Learning an Unknown Grammar with Recurrent Neural Networks , 1991, NIPS.