Complexity versus Agreement for Many Views

The paper considers the problem of semi-supervised multi-view classification, where each view corresponds to a Reproducing Kernel Hilbert Space. An algorithm based on co-regularization methods with extra penalty terms reflecting smoothness and general agreement properties is proposed. We first provide explicit tight control on the Rademacher (L 1) complexity of the corresponding class of learners for arbitrary many views, then give the asymptotic behavior of the bounds when the co-regularization term increases, making explicit the relation between consistency of the views and reduction of the search space. Since many views involve many parameters, we third provide a parameter selection procedure, based on the stability approach with clustering and localization arguments. To this aim, we give an explicit bound on the variance (L 2-diameter) of the class of functions. Finally we illustrate the algorithm through simulations on toy examples.

[1]  Vikas Sindhwani,et al.  On Manifold Regularization , 2005, AISTATS.

[2]  V. Koltchinskii.  Local Rademacher complexities and oracle inequalities in risk minimization , 2006, 0708.0083.

[3]  Tong Zhang,et al.  Learning on Graph with Laplacian Regularization , 2006, NIPS.

[4]  Mikhail Belkin,et al.  A Co-Regularization Approach to Semi-supervised Learning with Multiple Views , 2005 .

[5]  Jason Weston,et al.  Semi-supervised Protein Classification Using Cluster Kernels , 2003, NIPS.

[6]  M. Talagrand,et al.  Probability in Banach Spaces: Isoperimetry and Processes , 1991 .

[7]  Bernhard Schölkopf,et al.  Learning with Local and Global Consistency , 2003, NIPS.

[8]  Maria-Florina Balcan,et al.  Co-Training and Expansion: Towards Bridging Theory and Practice , 2004, NIPS.

[9]  Vikas Sindhwani,et al.  An RKHS for multi-view learning and manifold co-regularization , 2008, ICML '08.

[10]  Alain Celisse,et al.  Model selection via cross-validation in density estimation, regression, and change-points detection , 2008 .

[11]  Shai Ben-David,et al.  A Sober Look at Clustering Stability , 2006, COLT.

[12]  Gene H. Golub,et al.  Matrix computations , 1983 .

[13]  Peter L. Bartlett,et al.  The Rademacher Complexity of Co-Regularized Kernel Classes , 2007, AISTATS.

[14]  Werner Linde,et al.  Approximation, metric entropy and small ball estimates for Gaussian measures , 1999 .

[15]  Avrim Blum,et al.  The Bottleneck , 2021, Monopsony Capitalism.

[16]  Zhan Shi,et al.  Small ball estimates for Brownian motion under a weighted sup-norm , 2000 .

[17]  Sham M. Kakade,et al.  An Information Theoretic Framework for Multi-view Learning , 2008, COLT.

[18]  Alexander J. Smola,et al.  Kernels and Regularization on Graphs , 2003, COLT.

[19]  Maria-Florina Balcan,et al.  A PAC-Style Model for Learning from Labeled and Unlabeled Data , 2005, COLT.