Learning Shallow Context-free Languages under Simple Distributions

In this paper I present the EMILE 3.0 algorithm 1. It can learn shallow context-free grammars eeciently. It does so under circumstances that, from a perspective of complexity, come resonably close to the conditions under which human beings learn a language. A language is shallow in its descriptive length if all the relevant constructions we need to know to learn it have a complexity that is logarithmic in the descriptive length of the grammar as a whole. I claim that natural languages are shallow and that shallowness in itself is an interesting general constraint in the context of formal learning theory. I also introduce the concept of sepa-rability. A language is separable if the validity of rules can be tested by means of memberbership queries using simple substitutions. Although I do not believe that human languages are strictly separable I do think it is an important aspect of eecient human communication. I illustrate the structure of the algorithm by means of an extensive example, and I indicate how a careful analysis of the bias involved in natural language learning may lead to versions of EMILE 3.0 that can be implemented in real-life dialogue systems. I claim that the EMILE approach could serve as a valuable metamodel for evaluating clustering approaches to language learning. The results of EMILE are mainly theoretical. They 1 The original acronym stands for Entity Modeling Intelligent Learning Engine. It refers to earlier versions of EMILE that also had semantic capacities. The name EMILE is also motivated by the book on education by J.-J. Rousseau. In order to distinguishseveral versions of EMILE I started recently to give them version numbers. Versions 1 and 2 are discussed in Adriaans, 1992a. 1 2 / Pieter W. Adriaans could potentially also be used to develop more eecient variants of other approaches to language learning.