A Stochastic Search Approach to Grammar Induction

This paper describes a new sampling-based heuristic for tree search named SAGE and presents an analysis of its performance on the problem of grammar induction. This last work has been inspired by the Abbadingo DFA learning competition [14] which took place between Mars and November 1997. SAGE ended up as one of the two winners in that competition. The second winning algorithm, first proposed by Rodney Price, implements a new evidence-driven heuristic for state merging. Our own version of this heuristic is also described in this paper and compared to SAGE.

[1]  Thomas Bäck,et al.  A Survey of Evolution Strategies , 1991, ICGA.

[2]  Boris A. Trakhtenbrot,et al.  Finite automata : behavior and synthesis , 1973 .

[3]  Mikel L. Forcada,et al.  Learning the Initial State of a Second-Order Recurrent Neural Network during Regular-Language Inference , 1995, Neural Computation.

[4]  Barak A. Pearlmutter,et al.  Ab-badingo one: Dfa learning competition , 1997 .

[5]  Barak A. Pearlmutter,et al.  Results of the Abbadingo One DFA Learning Competition and a New Evidence-Driven State Merging Algorithm , 1998, ICGI.

[6]  E. Mark Gold,et al.  Complexity of Automaton Identification from Given Data , 1978, Inf. Control..

[7]  Raymond L. Watrous,et al.  Induction of Finite-State Languages Using Second-Order Recurrent Networks , 1992, Neural Computation.

[8]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[9]  Padhraic Smyth,et al.  Self-clustering recurrent networks , 1993, IEEE International Conference on Neural Networks.

[10]  Kevin J. Lang.  Random DFA's can be approximately learned from sparse uniform examples , 1992, COLT '92.

[11]  Michael C. Mozer,et al.  A Unified Gradient-Descent/Clustering Architecture for Finite State Machine Induction , 1993, NIPS.

[12]  Hugues Juillé.  Evolution of Non-Deterministic Incremental Algorithms as a New Approach for Search in State Spaces , 1995, ICGA.

[13]  Pang C. Chen.  Heuristic Sampling: A Method for Predicting the Performance of Tree Searching Programs , 1992, SIAM J. Comput..

[14]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[15]  Carl H. Smith,et al.  Inductive Inference: Theory and Methods , 1983, CSUR.