Identification of Finite State Automata With a Class of Recurrent Neural Networks

A class of recurrent neural networks is proposed and proven to be capable of identifying any discrete-time dynamical system. The application of the proposed network is addressed in the encoding, identification, and extraction of finite state automata (FSAs). Simulation results show that the identification of FSAs using the proposed network, trained by the hybrid greedy simulated annealing with a modified cost function in the training stage, generally exhibits better performance than the conventional identification procedures.

[1]  S. Hyakin,et al.  Neural Networks: A Comprehensive Foundation , 1994 .

[2]  P. Frasconi,et al.  Representation of Finite State Automata in Recurrent Radial Basis Function Networks , 1996, Machine Learning.

[3]  Sun-Young Lee,et al.  Improving generalization capability of neural networks based on simulated annealing , 2007, 2007 IEEE Congress on Evolutionary Computation.

[4]  Andrej Dobnikar,et al.  On-line identification and reconstruction of finite automata with generalized recurrent neural networks , 2003, Neural Networks.

[5]  Maria del Carmen Pegalajar Jiménez,et al.  A multiobjective genetic algorithm for obtaining the optimal size of a recurrent neural network for grammatical inference , 2005, Pattern Recognit..

[6]  Zhang Yi,et al.  Multistability of discrete-time recurrent neural networks with unsaturating piecewise linear activation functions , 2004, IEEE Transactions on Neural Networks.

[7]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[8]  Mikel L. Forcada,et al.  Stable Encoding of Finite-State Machines in Discrete-Time Recurrent Neural Nets with Sigmoid Units , 2000, Neural Computation.

[9]  Giovanni Soda,et al.  Inductive inference from noisy examples using the hybrid finite state filter , 1998, IEEE Trans. Neural Networks.

[10]  Stefan C. Kremer,et al.  On the computational power of Elman-style recurrent networks , 1995, IEEE Trans. Neural Networks.

[11]  Jeffrey D. Ullman,et al.  Introduction to Automata Theory, Languages and Computation , 1979 .

[12]  Kwang Y. Lee,et al.  Diagonal recurrent neural networks for dynamic systems control , 1995, IEEE Trans. Neural Networks.

[13]  D. Marquardt.  An Algorithm for Least-Squares Estimation of Nonlinear Parameters , 1963 .

[14]  W S McCulloch,et al.  A logical calculus of the ideas immanent in nervous activity , 1990, The Philosophy of Artificial Intelligence.

[15]  Miguel Pinzolas,et al.  Neighborhood based Levenberg-Marquardt algorithm for neural network training , 2002, IEEE Trans. Neural Networks.

[16]  Kenneth Levenberg.  A METHOD FOR THE SOLUTION OF CERTAIN NON – LINEAR PROBLEMS IN LEAST SQUARES , 1944 .

[17]  Panagiotis Manolios,et al.  First-Order Recurrent Neural Networks and Deterministic Finite State Automata , 1994, Neural Computation.

[18]  W. Graham Richards,et al.  Art of electronics , 1983, Nature.

[19]  Mengjie Zhang,et al.  Building Subcomponents in the Cooperative Coevolution Framework for Training Recurrent Neural Networks: Technical Report , 2009 .

[20]  Jing Peng,et al.  An Efficient Gradient-Based Algorithm for On-Line Training of Recurrent Network Trajectories , 1990, Neural Computation.

[21]  Richard D. Braatz,et al.  On the "Identification and control of dynamical systems using neural networks" , 1997, IEEE Trans. Neural Networks.

[22]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[23]  J.A. Ramirez,et al.  Hybrid Estimation of Distribution Algorithm Using Local Function Approximations , 2009, IEEE Transactions on Magnetics.

[24]  Emile H. L. Aarts,et al.  Simulated Annealing: Theory and Applications , 1987, Mathematics and Its Applications.

[25]  John F. Kolen,et al.  Field Guide to Dynamical Recurrent Networks , 2001 .

[26]  Padhraic Smyth,et al.  Learning Finite State Machines With Self-Clustering Recurrent Networks , 1993, Neural Computation.

[27]  Y. Baram,et al.  An information theoretic approach to dynamical systems modeling and identification , 1977, 1977 IEEE Conference on Decision and Control including the 16th Symposium on Adaptive Processes and A Special Symposium on Fuzzy Set Theory and Applications.

[28]  Stephen A. Billings,et al.  Lattice Dynamical Wavelet Neural Networks Implemented Using Particle Swarm Optimization for Spatio–Temporal System Identification , 2009, IEEE Transactions on Neural Networks.

[29]  C. Lee Giles,et al.  Learning and Extracting Finite State Automata with Second-Order Recurrent Neural Networks , 1992, Neural Computation.

[30]  Michael C. Mozer,et al.  Dynamic On-line Clustering and State Extraction: An Approach to Symbolic Learning , 1998, Neural Networks.

[31]  Eduardo D. Sontag,et al.  NEURAL NETS AS SYSTEMS MODELS AND CONTROLLERS , 1992 .

[32]  Ronald J. Williams,et al.  A Learning Algorithm for Continually Running Fully Recurrent Neural Networks , 1989, Neural Computation.

[33]  Jonas Sjöberg,et al.  Efficient training of neural nets for nonlinear adaptive filtering using a recursive Levenberg-Marquardt algorithm , 2000, IEEE Trans. Signal Process..

[34]  R. Ghanem,et al.  A wavelet-based approach for model and parameter identification of non-linear systems , 2001 .

[35]  Dongkyung Nam,et al.  n-Dimensional Cauchy Neighbor Generation for the Fast Simulated Annealing , 2004, IEICE Trans. Inf. Syst..

[36]  H. Troy Nagle,et al.  Performance of the Levenberg–Marquardt neural network training method in electronic nose applications , 2005 .

[37]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .