Fuzzy finite-state automata can be deterministically encoded into recurrent neural networks

There has been an increased interest in combining fuzzy systems with neural networks because fuzzy neural systems merge the advantages of both paradigms. On the one hand, parameters in fuzzy systems have clear physical meanings and rule-based and linguistic information can be incorporated into adaptive fuzzy systems in a systematic way. On the other hand, there exist powerful algorithms for training various neural network models. However, most of the proposed combined architectures are only able to process static input-output relationships; they are not able to process temporal input sequences of arbitrary length. Fuzzy finite-state automats (FFAs) can model dynamical processes whose current state depends on the current input and previous states. Unlike in the case of deterministic finite-state automats (DFAs), FFAs are not in one particular state, rather each state is occupied to some degree defined by a membership function. Based on previous work on encoding DFAs in discrete-time second-order recurrent neural networks, we propose an algorithm that constructs an augmented recurrent neural network that encodes a FFA and recognizes a given fuzzy regular language with arbitrary accuracy. We then empirically verify the encoding methodology by correct string recognition of randomly generated FFAs. In particular, we examine how the networks' performance varies as a function of synaptic weight strengths.

[1]  M. W. Shields An Introduction to Automata Theory , 1988 .

[2]  James L. McClelland,et al.  Finite State Automata and Simple Recurrent Networks , 1989, Neural Computation.

[3]  Li-Xin Wang,et al.  Adaptive fuzzy systems and control - design and stability analysis , 1994 .

[4]  Lotfi A. Zadeh,et al.  Information Title : Fuzzy languages and their relation to human and machine intelligence , 2022 .

[5]  Oscar H. IBARm Information and Control , 1957, Nature.

[6]  Noga Alon,et al.  Efficient simulation of finite automata by neural nets , 1991, JACM.

[7]  C. Lee Giles,et al.  Constructing deterministic finite-state automata in recurrent neural networks , 1996, JACM.

[8]  Jeffrey D. Ullman,et al.  Introduction to Automata Theory, Languages and Computation , 1979 .

[9]  S. Chiu,et al.  Fuzzy logic for control of roll and moment for a flexible wing aircraft , 1991, IEEE Control Systems.

[10]  Antonio Torralba,et al.  FASY: a fuzzy-logic based tool for analog synthesis , 1996, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst..

[11]  Lotfi A. Zadeh,et al.  Fuzzy Sets , 1996, Inf. Control..

[12]  C. L. Giles,et al.  Constructing deterministic finite-state automata in sparse recurrent neural networks , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[13]  Bing J. Sheu,et al.  Neural information processing and VLSI , 1995 .

[14]  Yoichi Hayashi,et al.  Fuzzy neural expert system with automated extraction of fuzzy If-Then rules from a trained neural network , 1990, [1990] Proceedings. First International Symposium on Uncertainty Modeling and Analysis.

[15]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1989, Math. Control. Signals Syst..

[16]  Terry L. Hardy Multi-objective decision-making under uncertainty: Fuzzy logic methods , 1994 .

[17]  King-Sun Fu,et al.  A Formulation of Fuzzy Automata and Its Application as a Model of Learning Systems , 1969, IEEE Trans. Syst. Sci. Cybern..

[18]  Michael C. Mozer,et al.  A Unified Gradient-Descent/Clustering Architecture for Finite State Machine Induction , 1993, NIPS.

[19]  Eugene S. Santos,et al.  Maximin Automata , 1968, Inf. Control..

[20]  Jennie Si,et al.  Analysis and synthesis of a class of discrete-time neural networks with multilevel threshold neurons , 1995, IEEE Trans. Neural Networks.

[21]  C. P. Pappis,et al.  A Fuzzy Logic Controller for a Trafc Junction , 1977, IEEE Transactions on Systems, Man, and Cybernetics.

[22]  C. Lee Giles,et al.  Rule Revision With Recurrent Neural Networks , 1996, IEEE Trans. Knowl. Data Eng..

[23]  C. Lee Giles,et al.  Stable Encoding of Large Finite-State Automata in Recurrent Neural Networks with Sigmoid Discriminants , 1996, Neural Computation.

[24]  Bart Kosko,et al.  Fuzzy Systems as Universal Approximators , 1994, IEEE Trans. Computers.

[25]  Richard Maclin,et al.  Refining algorithms with knowledge-based neural networks: improving the Chou-Fasman algorithm for protein folding , 1994, COLT 1994.

[26]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[27]  F. A. Unal,et al.  A fuzzy finite state machine implementation based on a neural fuzzy system , 1994, Proceedings of 1994 IEEE 3rd International Fuzzy Systems Conference.

[28]  Michael H. Freedman,et al.  Computation in discrete-time dynamical systems , 1995 .

[29]  H T Siegelmann,et al.  Dating and Context of Three Middle Stone Age Sites with Bone Points in the Upper Semliki Valley, Zaire , 2007 .

[30]  Chuen-Chien Lee,et al.  Fuzzy logic in control systems: fuzzy logic controller. I , 1990, IEEE Trans. Syst. Man Cybern..

[31]  Teresa B. Ludermir Logical networks capable of computing weighted regular languages , 1991, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks.

[32]  Peter Tiño,et al.  Learning and Extracting Initial Mealy Automata with a Modular Neural Network Model , 1995, Neural Comput..

[33]  Alberto Sanfeliu,et al.  An Algebraic Framework to Represent Finite State Machines in Single-Layer Recurrent Neural Networks , 1995, Neural Computation.

[34]  Don R. Hush,et al.  Bounds on the complexity of recurrent neural network implementations of finite state machines , 1993, Neural Networks.

[35]  Mike Casey,et al.  The Dynamics of Discrete-Time Computation, with Application to Recurrent Neural Networks and Finite State Machine Extraction , 1996, Neural Computation.

[36]  T. I. Liu,et al.  Design for machining using expert system and fuzzy logic approach , 1995, Journal of Materials Engineering and Performance.

[37]  Pekka Orponen,et al.  On the Effect of Analog Noise in Discrete-Time Analog Computations , 1996, Neural Computation.

[38]  Hugues Bersini,et al.  Recurrent fuzzy systems , 1994, Proceedings of 1994 IEEE 3rd International Fuzzy Systems Conference.

[39]  Carver Mead,et al.  Analog VLSI and neural systems , 1989 .

[40]  Edward T. Lee,et al.  Fuzzy Neural Networks , 1975 .

[41]  Marc Acheroy,et al.  Fuzzy reasoning and neural networks for decision making problems in uncertain environment , 1994, Proceedings of 1994 IEEE 3rd International Fuzzy Systems Conference.

[42]  Peter Tiňo,et al.  Finite State Machines and Recurrent Neural Networks -- Automata and Dynamical Systems Approaches , 1995 .

[43]  Sankar K. Pal,et al.  Fuzzy multi-layer perceptron, inferencing and rule generation , 1995, IEEE Trans. Neural Networks.

[44]  Hikmet Senay,et al.  Fuzzy command grammars for intelligent interface design , 1992, IEEE Trans. Syst. Man Cybern..

[45]  Karvel K. Thornber,et al.  The fidelity of fuzzy-logic inference , 1993, IEEE Trans. Fuzzy Syst..

[46]  Emdad Khan,et al.  Recurrent Fuzzy Logic Using Neural Network , 1994, IEEE/Nagoya-University World Wisepersons Workshop.

[47]  Hava T. Siegelmann,et al.  On the Computational Power of Neural Nets , 1995, J. Comput. Syst. Sci..

[48]  Padhraic Smyth,et al.  Learning Finite State Machines With Self-Clustering Recurrent Networks , 1993, Neural Computation.

[49]  C. Lee Giles,et al.  Learning and Extracting Finite State Automata with Second-Order Recurrent Neural Networks , 1992, Neural Computation.

[50]  Didier Dubois,et al.  Fuzzy sets and systems ' . Theory and applications , 2007 .

[51]  Marvin Minsky,et al.  Computation : finite and infinite machines , 2016 .

[52]  Amita Pathak,et al.  Fuzzy Grammars in Syntactic Recognition of Skeletal Maturity from X-Rays , 1986, IEEE Transactions on Systems, Man, and Cybernetics.

[53]  H. R. van Nauta Lemke,et al.  Application of a fuzzy controller in a warm water plant , 1976, Autom..

[54]  Giovanni Soda,et al.  Recurrent neural networks and prior knowledge for sequence processing: a constrained nondeterministic approach , 1995, Knowl. Based Syst..

[55]  Masao Nakagawa,et al.  Fuzzy inference neural networks which automatically partition a pattern space and extract fuzzy if-then rules , 1994, Proceedings of 1994 IEEE 3rd International Fuzzy Systems Conference.

[56]  Chuen-Chien Lee FUZZY LOGIC CONTROL SYSTEMS: FUZZY LOGIC CONTROLLER - PART I , 1990 .

[57]  Jude W. Shavlik,et al.  Refining Domain Theories Expressed as Finite-State Automata , 1991, ML.

[58]  Michael, G. Thomason,et al.  Deterministic Acceptors of Regular Fuzzy Languages , 1974, IEEE Trans. Syst. Man Cybern..

[59]  Raymond L. Watrous,et al.  Induction of Finite-State Languages Using Second-Order Recurrent Networks , 1992, Neural Computation.

[60]  J. Grantner,et al.  Synthesis and analysis of fuzzy logic finite state machine models , 1994, Proceedings of 1994 IEEE 3rd International Fuzzy Systems Conference.

[61]  Hans Martin Lipp,et al.  Fuzzy specification of finite state machines , 1990, Proceedings of the European Design Automation Conference, 1990., EDAC..

[62]  Arto Salomaa,et al.  Probabilistic and Weighted Grammars , 1969, Inf. Control..

[63]  B. Gaines,et al.  THE LOGIC OF AUTOMATA , 1975 .

[64]  Mo-Yuen Chow,et al.  A hybrid fuzzy/neural system used to extract heuristic knowledge from a fault detection problem , 1994, Proceedings of 1994 IEEE 3rd International Fuzzy Systems Conference.

[65]  Karvel K. Thornber,et al.  A key to fuzzy-logic inference , 1993, Int. J. Approx. Reason..

[66]  Hamid R. Berenji,et al.  Learning and tuning fuzzy logic controllers through reinforcements , 1992, IEEE Trans. Neural Networks.