Intrinsic complexity of learning geometrical concepts from positive data

Intrinsic complexity is used to measure the complexity of learning areas limited by broken-straight lines (called open semi-hulls) and intersections of such areas. Any strategy learning such geometrical concepts can be viewed as a sequence of primitive basic strategies. Thus, the length of such a sequence together with the complexities of the primitive strategies used can be regarded as the complexity of learning the concepts in question. We obtained the best possible lower and upper bounds on learning open semi-hulls, as well as matching upper and lower bounds on the complexity of learning intersections of such areas. Surprisingly, upper bounds in both cases turn out to be much lower than those provided by natural learning strategies. Another surprising result is that learning intersections of open semi-hulls turns out to be easier than learning open semi-hulls themselves.

[1]  Paul Young,et al.  An introduction to the general theory of algorithms , 1978 .

[2]  Paul W. Goldberg,et al.  Learning unions of boxes with membership and equivalence queries , 1994, COLT '94.

[3]  Stephen Kwek,et al.  Agnostic Learning of Geometric Patterns , 2001, J. Comput. Syst. Sci..

[4]  S. Goldman,et al.  PAC Learning of One-Dimensional Patterns , 1996, Machine Learning.

[5]  Efim B. Kinber Monotonicity versus Efficiency for Learning Languages from Texts , 1994, AII/ALT.

[6]  Rolf Wiehagen,et al.  On the Complexity of Program Synthesis from Examples , 1986, J. Inf. Process. Cybern..

[7]  Jerome A. Feldman,et al.  Some Decidability Results on Grammatical Inference and Complexity , 1972, Inf. Control..

[8]  John Case,et al.  Comparison of Identification Criteria for Machine Inductive Inference , 1983, Theor. Comput. Sci..

[9]  Dimitrios Gunopulos,et al.  Concept learning with geometric hypotheses , 1995, COLT '95.

[10]  D. C. Cooper,et al.  Theory of Recursive Functions and Effective Computability , 1969 .

[11]  David Haussler,et al.  Learnability and the Vapnik-Chervonenkis dimension , 1989, JACM.

[12]  John Case,et al.  The Power of Vacillation in Language Learning , 1999, SIAM J. Comput..

[13]  Paul Fischer,et al.  More or less efficient agnostic learning of convex polygons , 1995, COLT '95.

[14]  John Case,et al.  Machine Inductive Inference and Language Identification , 1982, ICALP.

[15]  Zhixiang Chen,et al.  The Learnability of Unions of Two Rectangles in the Two-Dimensional Discretized Space , 1999, J. Comput. Syst. Sci..

[16]  Tibor Hegedüs Geometrical Concept Learning and Convex Polytopes , 1994, COLT.

[17]  Frank Stephan,et al.  Language Learning from Texts: Mindchanges, Limited Memory, and Monotonicity , 1995, Inf. Comput..

[18]  E. Mark Gold,et al.  Language Identification in the Limit , 1967, Inf. Control..

[19]  Thomas Zeugmann,et al.  Learning Recursive Languages with Bounded Mind Changes , 1993, Int. J. Found. Comput. Sci..

[20]  Sally A. Goldman,et al.  A Theoretical and Empirical Study of a Noise-Tolerant Algorithm to Learn Geometric Patterns , 1999, Machine Learning.

[21]  Arun Sharma,et al.  The Intrinsic Complexity of Language Identification , 1996, J. Comput. Syst. Sci..

[22]  守屋 悦朗,et al.  J.E.Hopcroft, J.D. Ullman 著, "Introduction to Automata Theory, Languages, and Computation", Addison-Wesley, A5変形版, X+418, \6,670, 1979 , 1980 .

[23]  Manuel Blum,et al.  A Machine-Independent Theory of the Complexity of Recursive Functions , 1967, JACM.

[24]  Jeffrey D. Ullman,et al.  Introduction to Automata Theory, Languages and Computation , 1979 .

[25]  Zhixiang Chen,et al.  Learning unions of two rectangles in the plane with equivalence queries , 1993, COLT '93.

[26]  Arun Sharma,et al.  On the intrinsic complexity of language identification , 1994, COLT '94.

[27]  David Haussler,et al.  Classifying learnable geometric concepts with the Vapnik-Chervonenkis dimension , 1986, STOC '86.

[28]  Carl H. Smith,et al.  On the Intrinsic Complexity of Learning , 1995, Inf. Comput..

[29]  Zhixiang Chen,et al.  On-line learning of rectangles , 1992, COLT '92.

[30]  Jr. Hartley Rogers Theory of Recursive Functions and Effective Computability , 1969 .

[31]  Stephen Kwek,et al.  Agnostic learning of geometric patterns (extended abstract) , 1997, COLT '97.

[32]  Paul W. Goldberg,et al.  Exact Learning of Discretized Geometric Concepts , 1998, SIAM J. Comput..

[33]  Arun Sharma,et al.  The Structure of Intrinsic Complexity of Learning , 1997, J. Symb. Log..

[34]  Paul W. Goldberg,et al.  Learning one-dimensional geometric patterns under one-sided random misclassification noise , 1994, COLT '94.

[35]  Rolf Wiehagen,et al.  Language Learning from Texts: Degrees of Intrinsic Complexity and Their Characterizations , 2000, J. Comput. Syst. Sci..

[36]  Zhixiang Chen,et al.  On-line learning of rectangles and unions of rectangles , 2004, Machine Learning.

[37]  Carl H. Smith,et al.  On the Intrinsic Complexity of Learning , 1995, Inf. Comput..

[38]  Daniel N. Osherson,et al.  Criteria of Language Learning , 1982, Inf. Control..

[39]  Nader H. Bshouty,et al.  Noise-tolerant parallel learning of geometric concepts , 1995, COLT '95.