On the smallest possible dimension and the largest possible margin of linear arrangements representing given concept classes

This paper discusses theoretical limitations of classification systems that are based on feature maps and use a separating hyperplane in the feature space. In particular, we study the embeddability of a given concept class into a class of Euclidean half spaces of low dimension, or of arbitrarily large dimension but realizing a large margin. New bounds on the smallest possible dimension or on the largest possible margin are presented. In addition, we present new results on the rigidity of matrices and briefly mention applications in complexity and learning theory.

[1]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[2]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[3]  Jürgen Forster.  A linear lower bound on the unbounded error probabilistic communication complexity , 2002, J. Comput. Syst. Sci..

[4]  Hans Ulrich Simon,et al.  Estimating the Optimal Margins of Embeddings in Euclidean Half Spaces , 2001, COLT/EuroCOLT.

[5]  Shai Ben-David,et al.  Limitations of Learning Via Embeddings in Euclidean Half Spaces , 2003, J. Mach. Learn. Res..

[6]  Janos Simon,et al.  Probabilistic Communication Complexity , 1986, J. Comput. Syst. Sci..

[7]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[8]  A. Hoffman,et al.  The variation of the spectrum of a normal matrix , 1953 .

[9]  Santosh S. Vempala,et al.  An algorithmic theory of learning: Robust concepts and random projection , 1999, Machine Learning.

[10]  A. Razborov,et al.  Improved lower bounds on the rigidity of Hadamard matrices , 1998 .

[11]  Nello Cristianini,et al.  An introduction to Support Vector Machines , 2000 .

[12]  Satyanarayana V. Lokam.  Spectral methods for matrix rigidity with applications to size-depth tradeoffs and communication complexity , 1995, Proceedings of IEEE 36th Annual Foundations of Computer Science.

[13]  Gene H. Golub,et al.  Matrix computations , 1983 .

[14]  Peter Frankl,et al.  The Johnson-Lindenstrauss lemma and the sphericity of some graphs , 1987, J. Comb. Theory, Ser. B.

[15]  Charles R. Johnson,et al.  Topics in Matrix Analysis , 1991 .

[16]  W. B. Johnson,et al.  Extensions of Lipschitz mappings into Hilbert space , 1984 .