Elementary formal systems, intrinsic complexity, and procrastination

Recently, rich subclasses of elementary formal systems (EFS) have been shown to be identifiable in the limit from only positive data. Examples of these classes are Angluin’s pattern languages, unions of pattern languages by Wright and Shinohara, and classes of languages definable by length-bounded elementary formal systems studied by Shinohara. The present paper employs two distinct bodies of abstract studies in the inductive inference literature to analyze the learnability of the these concrete classes. The first approach, introduced by Freivalds and Smith, uses constructive ordinals to bound the number of mind changes. w denotes the first limit ordinal. An ordinal mind change bound of u means that identification can be carried out by a learner that after examining some element(s) of the language announces an upper bound on the number of mind changes it will make before converging; a bound of 2W means that the learner reserves the right to revise this upper bound once; a bound of 3W means the learner reserves the right to revise this upper bound twice, and so on. A bound of W2 means that identification can be carried out by a learner that announces an upper bound on the number of times it may revise its conjectured upper bound on the number of mind changes. It is shown in the present paper that the ordinal mind change complexity for identification of languages formed by unions of up to n pattern languages is Wn. It is also shown that this bound is essential. Similar results are also Permission to make digital/hard copies of all or part of this material for personal or classroom use is granted without fee provided that rhe copies are not made or distributed for profit or commercial advantage, the copyright ootice, the title of the publication and its date appear, and notice ia given that copyright is by permission of the ACM, Inc. To copy otherwise, to republish, to post on servers or to redistribute to lists, requirea apecitic permiaaion and/or fee. COLT’96, Desenzano del Garda, Italy @ 1996 ACM 0-89791-81 1-8/96/06 ..$3.50 shown to hold for classes definable by lengthbounded elementary formal systems with up to n clauses. The second approach, studied by Freivalds, Kinber and Wiehagen and by Jain and Sharma, employs reductions to study the intrinsic complexity of learnable classes. It is shown that the class of languages formed by taking unions of up to n + 1 pattern languages is a strictly more difficult learning problem than the class of languages formed by the union of up to n pattern languages. It is also shown that a similar hierarchy holds for the bound on the number of clauses in the case of languages definable by length-bounded EFS. In addition to building bridges between three distinct areas of inductive inference, viz., learnability of EFS subclasses, ordinal mind change complexity, and intrinsic complexity, this paper also presents results that relate topological properties of learnable classes to that of intrinsic complexity and ordinal mind change complexity. For example, it is shown that a clam that is complete according to the reductions for intrinsic complexity has infinite elasticity, Since EFS languages and their learnability results have counterparts in traditional logic programming, the present paper demonstrates the possibility of using abstract results of inductive inference to gain insights into inductive logic programming.