Online Co-regularized Algorithms

We propose an online co-regularized learning algorithm for classification and regression tasks. We demonstrate that by sequentially co-regularizing prediction functions on unlabeled data points, our algorithm provides improved performance in comparison to supervised methods on several UCI benchmarks and a real world natural language processing dataset. The presented algorithm is particularly applicable to learning tasks where large amounts of (unlabeled) data are available for training. We also provide an easy to set-up and use Python implementation of our algorithm.

[1]  Johan A. K. Suykens,et al.  Advances in learning theory : methods, models and applications , 2003 .

[2]  Jing Peng,et al.  SVM vs regularized least squares classification , 2004, ICPR 2004.

[3]  Daniel Dominic Sleator,et al.  Parsing English with a Link Grammar , 1995, IWPT.

[4]  Peter A. Flach,et al.  Evaluation Measures for Multi-class Subgroup Discovery , 2009, ECML/PKDD.

[5]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis , 2004 .

[6]  Yoram Singer,et al.  Pegasos: primal estimated sub-gradient solver for SVM , 2007, ICML '07.

[7]  Vikas Sindhwani,et al.  An RKHS for multi-view learning and manifold co-regularization , 2008, ICML '08.

[8]  Tapio Salakoski,et al.  Evaluation of two dependency parsers on biomedical corpus targeted at protein-protein interactions , 2006, Int. J. Medical Informatics.

[9]  T. Poggio,et al.  Regularized Least-Squares Classification 133 In practice , although , 2007 .

[10]  Johan A. K. Suykens,et al.  Least Squares Support Vector Machine Classifiers , 1999, Neural Processing Letters.

[11]  Jari Björne,et al.  BioInfer: a corpus for information extraction in the biomedical domain , 2007, BMC Bioinformatics.

[12]  Alexander Gammerman,et al.  Ridge Regression Learning Algorithm in Dual Variables , 1998, ICML.

[13]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[14]  A. Atiya,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2005, IEEE Transactions on Neural Networks.

[15]  Ming Li,et al.  Online Manifold Regularization: A New Learning Setting and Empirical Study , 2008, ECML/PKDD.

[16]  Chia-Hua Ho,et al.  Recent Advances of Large-Scale Linear Classification , 2012, Proceedings of the IEEE.

[17]  Avishek Saha,et al.  Co-regularization Based Semi-supervised Domain Adaptation , 2010, NIPS.

[18]  Peter L. Bartlett,et al.  The Rademacher Complexity of Co-Regularized Kernel Classes , 2007, AISTATS.

[19]  D. Sculley,et al.  Large Scale Learning to Rank , 2009 .

[20]  Thomas Gärtner,et al.  Efficient co-regularised least squares regression , 2006, ICML.

[21]  Ulf Brefeld,et al.  Co-EM support vector learning , 2004, ICML.

[22]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[23]  Mikhail Belkin,et al.  A Co-Regularization Approach to Semi-supervised Learning with Multiple Views , 2005 .

[24]  Glenn Fung,et al.  Proximal support vector machine classifiers , 2001, KDD '01.

[25]  Avrim Blum,et al.  The Bottleneck , 2021, Monopsony Capitalism.