A Close Look to Margin Complexity and Related Parameters

Concept classes can canonically be represented by sign-matrices, i.e., by matrices with entries 1 and 1. The question whether a sign-matrix (concept class) A can be learned by a machine that performs large margin classication is closely related to the \margin complexity" associated with A. We consider several variants of margin complexity, reveal how they are related to each other, and we reveal how they are related to other notions of learning-theoretic relevance like SQ-dimension, CSQ-dimension, and the Forster bound.

[1]  Michael Kearns,et al.  Efficient noise-tolerant learning from statistical queries , 1993, STOC.

[2]  Vitaly Feldman,et al.  On using extended statistical queries to avoid membership queries , 2002 .

[3]  Hans Ulrich Simon,et al.  Dimension and Margin Bounds for Reflection-invariant Kernels , 2008, COLT.

[4]  Vitaly Feldman,et al.  Evolvability from learning algorithms , 2008, STOC.

[5]  Alexander A. Sherstov.  Halfspace Matrices , 2007, Twenty-Second Annual IEEE Conference on Computational Complexity (CCC'07).

[6]  Hans Ulrich Simon,et al.  On the smallest possible dimension and the largest possible margin of linear arrangements representing given concept classes , 2006, Theor. Comput. Sci..

[7]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, STOC '84.

[8]  Nathan Linial,et al.  Complexity measures of sign matrices , 2007, Comb..

[9]  Farid Alizadeh,et al.  Interior Point Methods in Semidefinite Programming with Applications to Combinatorial Optimization , 1995, SIAM J. Optim..

[10]  Javed A. Aslam,et al.  General bounds on statistical query learning and PAC learning with noise via hypothesis boosting , 1993, Proceedings of 1993 IEEE 34th Annual Foundations of Computer Science.

[11]  Vitaly Feldman,et al.  On Using Extended Statistical Queries to Avoid Membership Queries , 2001, J. Mach. Learn. Res..

[12]  Santosh S. Vempala,et al.  An algorithmic theory of learning: Robust concepts and random projection , 1999, Machine Learning.

[13]  Shai Ben-David,et al.  Limitations of Learning Via Embeddings in Euclidean Half Spaces , 2003, J. Mach. Learn. Res..

[14]  Javed A. Aslam,et al.  General Bounds on Statistical Query Learning and PAC Learning with Noise via Hypothesis Boosting , 1998, Inf. Comput..

[15]  Zvi Galil,et al.  Lower bounds on communication complexity , 1984, STOC '84.