Modeling repetitive patterns: A bridge between pattern theory and data mining

Traditional learning algorithms generate a predictive model by effectively partitioning the input or feature space in search for regions having a dominant single class. In this paper we point to the existence of problems where the relation among these regions corresponds to repetitive patterns that can be mapped to high-level models. We show how a formalism for the representation of patterns, also known as pattern theory, is instrumental to capture such relations. The idea is to verify patterns using mathematical constructs by combining primitive structures. We illustrate our ideas using parity problems, and show how bridging the gap between traditional supervised learning and pattern theory is a challenge that can bring large benefits to the data mining community.

[1]  L. Goddard.  Information Theory , 1962, Nature.

[2]  Robert B. Ash,et al.  Information Theory , 2020, The SAGE International Encyclopedia of Mass Media and Society.

[3]  David G. Stork,et al.  Pattern Classification , 1973 .

[4]  Chris Thornton,et al.  Parity: The Problem that Won't Go Away , 1996, Canadian Conference on AI.

[5]  U. Grenander.  Elements of Pattern Theory , 1996 .

[6]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[7]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[8]  David R. Anderson,et al.  Model selection and multimodel inference : a practical information-theoretic approach , 2003 .

[9]  M. Peruggia.  Model Selection and Multimodel Inference: A Practical Information-Theoretic Approach (2nd ed.) , 2003 .

[10]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis , 2003, ICTAI.

[11]  David J. C. MacKay,et al.  Information Theory, Inference, and Learning Algorithms , 2004, IEEE Transactions on Information Theory.

[12]  J. Franklin,et al.  The elements of statistical learning: data mining, inference and prediction , 2005 .

[13]  Tony Jebara,et al.  Machine learning: Discriminative and generative , 2006 .

[14]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[15]  Michael I. Miller,et al.  Pattern Theory: From Representation to Inference , 2007 .

[16]  Andreas Christmann,et al.  Support vector machines , 2008, Data Mining and Knowledge Discovery Handbook.

[17]  Kaizhu Huang,et al.  Machine Learning: Modeling Data Locally and Globally , 2008 .

[18]  Jacob Feldman,et al.  Conceptual complexity and the bias/variance tradeoff , 2011, Cognition.