Classifier learning with hidden information

Traditional data-driven classifier learning approaches become limited when the training data is inadequate either in quantity or quality. To address this issue, in this paper we propose to combine hidden information and data to enhance classifier learning. Hidden information represents information that is only available during training but not available during testing. It often exists in many applications yet has not been thoroughly exploited, and existing methods to utilize hidden information are still limited. To this end, we propose two general approaches to exploit different types of hidden information to improve different classifiers. We also extend the proposed methods to deal with incomplete hidden information. Experimental results on different applications demonstrate the effectiveness of the proposed methods for exploiting hidden information and their superior performance to existing methods.

[1]  Lawrence Carin,et al.  Multi-Task Learning for Classification with Dirichlet Process Priors , 2007, J. Mach. Learn. Res..

[2]  Masao Fukushima,et al.  A new multi-class support vector algorithm , 2006, Optim. Methods Softw..

[3]  Shiliang Sun,et al.  Multitask multiclass privileged information support vector machines , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[4]  Jean-Philippe Vert,et al.  Clustered Multi-Task Learning: A Convex Formulation , 2008, NIPS.

[5]  Peter Tiño,et al.  Learning Using Privileged Information in Prototype Based Models , 2012, ICANN.

[6]  Qiang Ji,et al.  Anatomically Constrained Decoding of Finger Flexion from Electrocorticographic Signals , 2011, NIPS.

[7]  Yong Shi,et al.  Learning Using Privileged Information with L-1 Support Vector Machine , 2012, 2012 IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology.

[8]  David S. Rosenberg,et al.  The rademacher complexity of coregularized kernel classes , 2007 .

[9]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[10]  Uwe Aickelin,et al.  Privileged information for data clustering , 2012, Inf. Sci..

[11]  Bernt Schiele,et al.  Learning using privileged information: SV M+ and weighted SVM , 2013, Neural Networks.

[12]  Massimiliano Pontil,et al.  Regularized multi--task learning , 2004, KDD.

[13]  Mikhail Belkin,et al.  A Co-Regularization Approach to Semi-supervised Learning with Multiple Views , 2005 .

[14]  Vladimir Cherkassky,et al.  Generalized SMO Algorithm for SVM-Based Multitask Learning , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[15]  Vladimir Vapnik,et al.  A new learning paradigm: Learning using privileged information , 2009, Neural Networks.

[16]  Masashi Sugiyama,et al.  Multi-Task Learning via Conic Programming , 2007, NIPS.

[17]  Takeo Kanade,et al.  The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[18]  V. Vapnik,et al.  On the theory of learning with Privileged Information , 2010, NIPS 2010.

[19]  Christoph H. Lampert,et al.  Attribute-Based Classification for Zero-Shot Visual Object Categorization , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Qiang Ji,et al.  Learning with Hidden Information , 2014, 2014 22nd International Conference on Pattern Recognition.

[21]  John Shawe-Taylor,et al.  Two view learning: SVM-2K, Theory and Practice , 2005, NIPS.

[22]  Qiang Ji,et al.  Learning with Hidden Information Using a Max-Margin Latent Variable Model , 2014, 2014 22nd International Conference on Pattern Recognition.

[23]  Dit-Yan Yeung,et al.  A Convex Formulation for Learning Task Relationships in Multi-Task Learning , 2010, UAI.

[24]  Christoph H. Lampert,et al.  Learning to Rank Using Privileged Information , 2013, 2013 IEEE International Conference on Computer Vision.

[25]  Xiaoming Liu,et al.  Boosting with Side Information , 2012, ACCV.

[26]  Peter L. Bartlett,et al.  The Rademacher Complexity of Co-Regularized Kernel Classes , 2007, AISTATS.