Agnostic Learning of Geometric Patterns

P. W. Goldberg, S. A. Goldman, and S. D. Scott (Mach. Learning25, No. 1 (1996), 51?70) discussed how the problem of recognizing a landmark from a one-dimensional visual image might be mapped to that of learning a one-dimensional geometric pattern and gave a PAC algorithm to learn that class. In this paper, we present an efficient online agnostic learning algorithm for learning the class of constant-dimensional geometric patterns. Our algorithm can tolerate both classification and attribute noise. By working in higher dimensional spaces we can represent more features from the visual image in the geometric pattern. Our mapping of the data to a geometric pattern and, hence, our learning algorithm are applicable to any data representable as a constant-dimensional array of values, e.g., sonar data, temporal difference information, amplitudes of a waveform, or other pattern recognition data. To our knowledge, these classes of patterns are more complex than any class of geometric patterns previously studied. Also, our results are easily adapted to learn the union of fixed-dimensional boxes from multiple-instance examples. Finally, our algorithms are tolerant of concept shift, where the target concept that labels the examples can change over time.

[1]  David Haussler,et al.  Decision Theoretic Generalizations of the PAC Model for Neural Net and Other Learning Applications , 1992, Inf. Comput..

[2]  David Haussler,et al.  Equivalence of models for polynomial learnability , 1988, COLT '88.

[3]  Thomas G. Dietterich,et al.  Converting English text to speech: a machine learning approach , 1991 .

[4]  Thomas G. Dietterich,et al.  Solving the Multiple Instance Problem with Axis-Parallel Rectangles , 1997, Artif. Intell..

[5]  F ROSENBLATT,et al.  The perceptron: a probabilistic model for information storage and organization in the brain. , 1958, Psychological review.

[6]  Jörg M. Wills,et al.  Convexity and its applications , 1983 .

[7]  Thomas G. Dietterich,et al.  Improving the Performance of Radial Basis Function Networks by Learning Center Locations , 1991, NIPS.

[8]  Tod S. Levitt,et al.  Qualitative Navigation for Mobile Robots , 1990, Artif. Intell..

[9]  R. Schapire,et al.  Toward Efficient Agnostic Learning , 1994 .

[10]  David Haussler,et al.  Learnability and the Vapnik-Chervonenkis dimension , 1989, JACM.

[11]  Thomas G. Dietterich,et al.  Solving Multiclass Learning Problems via Error-Correcting Output Codes , 1994, J. Artif. Intell. Res..

[12]  Aravind Srinivasan,et al.  Approximating hyper-rectangles: learning and pseudo-random sets , 1997, STOC '97.

[13]  Thomas G. Dietterich,et al.  Error-Correcting Output Coding Corrects Bias and Variance , 1995, ICML.

[14]  Javed A. Aslam,et al.  Specification and simulation of statistical query algorithms for efficiency and noise tolerance , 1995, Annual Conference Computational Learning Theory.

[15]  Edward M. Riseman,et al.  Image-based homing , 1992 .

[16]  Leslie G. Valiant,et al.  Computational limitations on learning from examples , 1988, JACM.

[17]  Nick Littlestone,et al.  Redundant noisy attributes, attribute errors, and linear-threshold learning using winnow , 1991, COLT '91.

[18]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, CACM.

[19]  Manfred K. Warmuth,et al.  Efficient Learning With Virtual Threshold Gates , 1995, Inf. Comput..

[20]  Suguru Arimoto,et al.  Visual control of autonomous mobile robot based on self-organizing model for pattern learning , 1988, J. Field Robotics.

[21]  N. Littlestone.  Learning Quickly When Irrelevant Attributes Abound: A New Linear-Threshold Algorithm , 1987, 28th Annual Symposium on Foundations of Computer Science (sfcs 1987).

[22]  R. Schapire.  Toward Eecient Agnostic Learning , 1992 .

[23]  Oded Maron,et al.  Multiple-Instance Learning for Natural Scene Classification , 1998, ICML.

[24]  W. Maass,et al.  Eecient Learning with Virtual Threshold Gates , 1997 .

[25]  Nick Littlestone,et al.  From on-line to batch learning , 1989, COLT '89.

[26]  P. Gruber.  Approximation of convex bodies , 1983 .

[27]  Thomas G. Dietterich,et al.  Error-Correcting Output Codes: A General Method for Improving Multiclass Inductive Learning Programs , 1991, AAAI.

[28]  Leslie G. Valiant,et al.  Learning Disjunction of Conjunctions , 1985, IJCAI.

[29]  Paul W. Goldberg,et al.  PAC-learning geometrical figures , 1992 .

[30]  Stephen Kwek,et al.  Learning of depth two neural networks with constant fan-in at the hidden nodes (extended abstract) , 1996, COLT '96.

[31]  B. Pinette.  Image-based navigation through large-scale environments , 1994 .

[32]  Peter Auer,et al.  On Learning From Multi-Instance Examples: Empirical Evaluation of a Theoretical Approach , 1997, ICML.

[33]  Linda Sellie,et al.  Toward efficient agnostic learning , 1992, COLT '92.

[34]  Michael Kearns,et al.  Efficient noise-tolerant learning from statistical queries , 1993, STOC.

[35]  Philip M. Long,et al.  PAC Learning Axis-aligned Rectangles with Respect to Product Distributions from Multiple-Instance Examples , 1996, COLT '96.

[36]  Manfred K. Warmuth,et al.  The weighted majority algorithm , 1989, 30th Annual Symposium on Foundations of Computer Science.

[37]  Manfred K. Warmuth,et al.  The Perceptron Algorithm Versus Winnow: Linear Versus Logarithmic Mistake Bounds when Few Input Variables are Relevant (Technical Note) , 1997, Artif. Intell..

[38]  Manfred K. Warmuth,et al.  The perceptron algorithm vs. Winnow: linear vs. logarithmic mistake bounds when few input variables are relevant , 1995, COLT '95.

[39]  Tomás Lozano-Pérez,et al.  A Framework for Multiple-Instance Learning , 1997, NIPS.

[40]  Paul W. Goldberg,et al.  Exact Learning of Discretized Geometric Concepts , 1998, SIAM J. Comput..

[41]  Oded Maron,et al.  Learning from Ambiguity , 1998 .

[42]  Robert E. Schapire,et al.  Using output codes to boost multiclass learning problems , 1997, ICML.