Information Bottleneck Learning Using Privileged Information for Visual Recognition

We explore the visual recognition problem from a main data view when an auxiliary data view is available during training. This is important because it allows improving the training of visual classifiers when paired additional data is cheaply available, and it improves the recognition from multi-view data when there is a missing view at testing time. The problem is challenging because of the intrinsic asymmetry caused by the missing auxiliary view during testing. We account for such view during training by extending the information bottleneck method, and by combining it with risk minimization. In this way, we establish an information theoretic principle for leaning any type of visual classifier under this particular setting. We use this principle to design a large-margin classifier with an efficient optimization in the primal space. We extensively compare our method with the state-of-the-art on different visual recognition datasets, and with different types of auxiliary data, and show that the proposed framework has a very promising potential.

[1]  Vladimir Vapnik,et al.  A new learning paradigm: Learning using privileged information , 2009, Neural Networks.

[2]  John Shawe-Taylor,et al.  Canonical Correlation Analysis: An Overview with Application to Learning Methods , 2004, Neural Computation.

[3]  Gal Chechik,et al.  Extracting Relevant Structures with Side Information , 2002, NIPS.

[4]  Xiaoming Liu,et al.  Boosting with Side Information , 2012, ACCV.

[5]  Gang Hua,et al.  Multi-View Visual Recognition of Imperfect Testing Data , 2015, ACM Multimedia.

[6]  Andrew W. Fitzgibbon,et al.  Efficient Object Category Recognition Using Classemes , 2010, ECCV.

[7]  Naftali Tishby,et al.  Multivariate Information Bottleneck , 2001, Neural Computation.

[8]  Christoph H. Lampert,et al.  Attribute-Based Classification for Zero-Shot Visual Object Categorization , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Jeff Donahue,et al.  Annotator rationales for visual recognition , 2011, 2011 International Conference on Computer Vision.

[10]  Qiang Ji,et al.  Learning with Hidden Information , 2014, 2014 22nd International Conference on Pattern Recognition.

[11]  Bernt Schiele,et al.  Learning using privileged information: SV M+ and weighted SVM , 2013, Neural Networks.

[12]  Trevor Darrell,et al.  Adapting Visual Category Models to New Domains , 2010, ECCV.

[13]  Trevor Darrell,et al.  Hidden Conditional Random Fields , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Isabelle Guyon,et al.  The ChaLearn gesture dataset (CGD 2011) , 2014, Machine Vision and Applications.

[15]  V. Vapnik,et al.  On the theory of learning with Privileged Information , 2010, NIPS 2010.

[16]  Thomas Serre,et al.  HMDB: A large video database for human motion recognition , 2011, 2011 International Conference on Computer Vision.

[17]  Naftali Tishby,et al.  Agglomerative Information Bottleneck , 1999, NIPS.

[18]  Jan Feyereisl,et al.  Object Localization based on Structural SVM using Privileged Information , 2014, NIPS.

[19]  Shiqian Ma,et al.  Fast alternating linearization methods for minimizing the sum of two convex functions , 2009, Math. Program..

[20]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[21]  Vladimir Cherkassky,et al.  Connection between SVM+ and multi-task learning , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[22]  Sebastian Nowozin,et al.  On feature combination for multiclass object classification , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[23]  Naftali Tishby,et al.  The information bottleneck method , 2000, ArXiv.

[24]  Dong Xu,et al.  Recognizing RGB Images by Learning from RGB-D Data , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  John Shawe-Taylor,et al.  Two view learning: SVM-2K, Theory and Practice , 2005, NIPS.

[26]  Qiang Ji,et al.  Learning with Hidden Information Using a Max-Margin Latent Variable Model , 2014, 2014 22nd International Conference on Pattern Recognition.

[27]  Gérard Bloch,et al.  Incorporating prior knowledge in support vector machines for classification: A review , 2008, Neurocomputing.

[28]  Yi Yao,et al.  Boosting for transfer learning with multiple sources , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[29]  Qiang Ji,et al.  Classifier learning with hidden information , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Zhuowen Tu,et al.  Harvesting Mid-level Visual Concepts from Large-Scale Internet Images , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[31]  Ali Farhadi,et al.  Describing objects by their attributes , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Juhan Nam,et al.  Multimodal Deep Learning , 2011, ICML.

[33]  Yurii Nesterov,et al.  Smooth minimization of non-smooth functions , 2005, Math. Program..

[34]  Chih-Jen Lin,et al.  Projected Gradient Methods for Nonnegative Matrix Factorization , 2007, Neural Computation.

[35]  Saeid Motiian,et al.  Information Bottleneck Domain Adaptation with Privileged Information for Visual Recognition , 2016, ECCV.

[36]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[37]  Andrea Vedaldi,et al.  Vlfeat: an open and portable library of computer vision algorithms , 2010, ACM Multimedia.

[38]  Dong Xu,et al.  Distance Metric Learning Using Privileged Information for Face Verification and Person Re-Identification , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[39]  David A. McAllester,et al.  Object Detection with Discriminatively Trained Part Based Models , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Dong Xu,et al.  Exploiting Privileged Information from Web Data for Image Categorization , 2014, ECCV.

[41]  Lior Wolf,et al.  The SVM-Minus Similarity Score for Video Face Recognition , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[42]  TaoDacheng,et al.  Large-Margin Multi-ViewInformation Bottleneck , 2014 .

[43]  Dacheng Tao,et al.  Large-Margin Multi-ViewInformation Bottleneck , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  Andrew Zisserman,et al.  Multiple kernels for object detection , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[45]  Christoph H. Lampert,et al.  Learning to Rank Using Privileged Information , 2013, 2013 IEEE International Conference on Computer Vision.

[46]  Peter Tiño,et al.  Incorporating Privileged Information Through Metric Learning , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[47]  Andrew Zisserman,et al.  Learning Visual Attributes , 2007, NIPS.

[48]  Xindong Wu,et al.  NESVM: A Fast Gradient Method for Support Vector Machines , 2010, 2010 IEEE International Conference on Data Mining.

[49]  Thorsten Joachims,et al.  Making large scale SVM learning practical , 1998 .

[50]  Koby Crammer,et al.  A theory of learning from different domains , 2010, Machine Learning.

[51]  Thorsten Joachims,et al.  Training linear SVMs in linear time , 2006, KDD '06.

[52]  Gang Hua,et al.  Can Visual Recognition Benefit from Auxiliary Information in Training? , 2014, ACCV.

[53]  Erkki Oja,et al.  Kullback-Leibler Divergence for Nonnegative Matrix Factorization , 2011, ICANN.