Learning Automata from Ordered Examples

Connectionist learning models have had considerable empirical success, but it is hard to characterize exactly what they learn. The learning of finite-state languages (FSL) from example strings is a domain which has been extensively studied and might provide an opportunity to help understand connectionist learning. A major problem is that traditional FSL learning assumes the storage of all examples and thus violates connectionist principles. This paper presents a provably correct algorithm for inferring any minimum-state deterministic finite-state automata (FSA) from a complete ordered sample using limited total storage and without storing example strings. The algorithm is an iterative strategy that uses at each stage a current encoding of the data considered so far, and one single sample string. One of the crucial advantages of our algorithm is that the total amount of space used in the course of learning for encoding any finite prefix of the sample is polynomial in the size of the inferred minimum state deterministic FSA. The algorithm is also relatively efficient in time and has been implemented. More importantly, there is a connectionist version of the algorithm that preserves these properties. The connectionist version requires much more structure than the usual models and has been implemented using the Rochester Connectionist Simulator. We also show that no machine with finite working storage can iteratively identify the FSL from arbitrary presentations.

[1]  Robert E. Schapire,et al.  A new approach to unsupervised learning in deterministic environments , 1990 .

[2]  Tao Jiang,et al.  Learning regular languages from counterexamples , 1988, COLT '88.

[3]  Carl H. Smith,et al.  Inductive Inference: Theory and Methods , 1983, CSUR.

[4]  Jerome A. Feldman,et al.  On the Synthesis of Finite-State Machines from Samples of Their Behavior , 1972, IEEE Transactions on Computers.

[5]  Dana Angluin,et al.  A Note on the Number of Queries Needed to Identify Regular Languages , 1981, Inf. Control..

[6]  Dana Charmian Angluin,et al.  An application of the theory of computational complexity to the study of inductive inference. , 1976 .

[7]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, STOC '84.

[8]  E. Mark Gold,et al.  Language Identification in the Limit , 1967, Inf. Control..

[9]  守屋 悦朗,et al.  J.E.Hopcroft, J.D. Ullman 著, "Introduction to Automata Theory, Languages, and Computation", Addison-Wesley, A5変形版, X+418, \6,670, 1979 , 1980 .

[10]  Jerome A. Feldman,et al.  Connectionist Models and Their Properties , 1982, Cogn. Sci..

[11]  Klaus P. Jantke,et al.  Combining Postulates of Naturalness in Inductive Inference , 1981, J. Inf. Process. Cybern..

[12]  Jerome A. Feldman,et al.  Connectionist models and their implications , 1988 .

[13]  Daniel N. Osherson,et al.  Systems That Learn: An Introduction to Learning Theory for Cognitive and Computer Scientists , 1990 .

[14]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[15]  E. Mark Gold,et al.  System identification via state characterization , 1972 .

[16]  Dana Angluin,et al.  Learning Regular Sets from Queries and Counterexamples , 1987, Inf. Comput..

[17]  DANA ANGLUIN,et al.  On the Complexity of Minimum Inference of Regular Sets , 1978, Inf. Control..

[18]  Geoffrey E. Hinton Connectionist Learning Procedures , 1989, Artif. Intell..

[19]  Dominique Perrin,et al.  Finite Automata , 1958, Philosophy.

[20]  Ronald L. Rivest,et al.  Diversity-Based Inference of Finite Automata (Extended Abstract) , 1987, FOCS.

[21]  James Jay Horning,et al.  A study of grammatical inference , 1969 .

[22]  Leslie G. Valiant,et al.  On the learnability of Boolean formulae , 1987, STOC.

[23]  Balas K. Natarajan,et al.  On learning Boolean functions , 1987, STOC.

[24]  E. Mark Gold,et al.  Complexity of Automaton Identification from Given Data , 1978, Inf. Control..

[25]  R. A. Brooks,et al.  Intelligence without Representation , 1991, Artif. Intell..