Convex sets as prototypes for classifying patterns

We propose a general framework for nonparametric classification of multi-dimensional numerical patterns. Given training points for each class, it builds a set cover with convex sets each of which contains some training points of the class but no points of the other classes. Each convex set has thus an associated class label, and classification of a query point is made to the class of the convex set such that the projection of the query point onto its boundary is minimal. In this sense, the convex sets of a class are regarded as ''prototypes'' for that class. We then apply this framework to two special types of convex sets, minimum enclosing balls and convex hulls, giving algorithms for constructing a set cover with them and for computing the projection length onto their boundaries. For convex hulls, we also give a method for implicitly evaluating whether a point is contained in a convex hull, which can avoid computational difficulty for explicit construction of convex hulls in high-dimensional space.

[1]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[2]  Trevor Hastie,et al.  Prototype Methods and Nearest-Neighbors , 2009 .

[3]  Kim-Chuan Toh,et al.  Efficient Algorithms for the Smallest Enclosing Ball Problem , 2005, Comput. Optim. Appl..

[4]  Don R. Hush,et al.  Machine Learning with Data Dependent Hypothesis Classes , 2002, J. Mach. Learn. Res..

[5]  Dengyong Zhou,et al.  Global Geometry of SVM Classifiers , 2002 .

[6]  A. Karr Random Graphs for Statistical Pattern Recognition , 2006 .

[7]  Lenore Cowen,et al.  Approximation Algorithms for the Class Cover Problem , 2004, Annals of Mathematics and Artificial Intelligence.

[8]  Bernd Gärtner,et al.  Fast and Robust Smallest Enclosing Balls , 1999, ESA.

[9]  Emo Welzl,et al.  Smallest enclosing disks (balls and ellipsoids) , 1991, New Results and New Trends in Computer Science.

[10]  Carey E. Priebe,et al.  Classification Using Class Cover Catch Digraphs , 2003, J. Classif..

[11]  A. Atiya,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2005, IEEE Transactions on Neural Networks.

[12]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[13]  Ying Liu,et al.  The Maximum Box Problem and its Application to Data Analysis , 2002, Comput. Optim. Appl..

[14]  Alexander J. Smola,et al.  Learning with Kernels: support vector machines, regularization, optimization, and beyond , 2001, Adaptive computation and machine learning series.

[15]  Peter L. Hammer,et al.  Spanned patterns for the logical analysis of data , 2006, Discret. Appl. Math..

[16]  Atsuyoshi Nakamura,et al.  The Convex Subclass Method: Combinatorial Classifier Based on a Family of Convex Sets , 2005, MLDM.

[17]  Geoffrey J. McLachlan,et al.  Finite Mixture Models , 2019, Annual Review of Statistics and Its Application.

[18]  Carey E. Priebe,et al.  The class cover problem and its applications in pattern recognition , 2003 .

[19]  Mineichi Kudo,et al.  Construction of class regions by a randomized algorithm: a randomized subclass method , 1996, Pattern Recognit..

[20]  Olvi L. Mangasarian,et al.  Polyhedral Boundary Projection , 1999, SIAM J. Optim..

[21]  Kristin P. Bennett,et al.  Duality and Geometry in SVM Classifiers , 2000, ICML.

[22]  David G. Stork,et al.  Pattern Classification , 1973 .

[23]  Sergios Theodoridis,et al.  A novel SVM Geometric Algorithm based on Reduced Convex Hulls , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[24]  Bernd Gärtner,et al.  Fast Smallest-Enclosing-Ball Computation in High Dimensions , 2003, ESA.

[25]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[26]  Peter Gritzmann,et al.  Computational complexity of inner and outerj-radii of polytopes in finite-dimensional normed spaces , 1993, Math. Program..

[27]  Hans J. H. Tuenter Minimum L1-Distance Projection onto the Boundary of a Convex Set: Simple Characterization , 2002 .

[28]  James B. Orlin,et al.  On the complexity of four polyhedral set containment problems , 2018, Math. Program..

[29]  László Györfi,et al.  A Probabilistic Theory of Pattern Recognition , 1996, Stochastic Modelling and Applied Probability.

[30]  Walter Briec,et al.  Minimum Distance to the Complement of a Convex Set: Duality Result , 1997 .