Multiview Matrix Completion for Multilabel Image Classification

There is growing interest in multilabel image classification due to its critical role in web-based image analytics-based applications, such as large-scale image retrieval and browsing. Matrix completion (MC) has recently been introduced as a method for transductive (semisupervised) multilabel classification, and has several distinct advantages, including robustness to missing data and background noise in both feature and label space. However, it is limited by only considering data represented by a single-view feature, which cannot precisely characterize images containing several semantic concepts. To utilize multiple features taken from different views, we have to concatenate the different features as a long vector. However, this concatenation is prone to over-fitting and often leads to very high time complexity in MC-based image classification. Therefore, we propose to weightedly combine the MC outputs of different views, and present the multiview MC (MVMC) framework for transductive multilabel image classification. To learn the view combination weights effectively, we apply a cross-validation strategy on the labeled set. In particular, MVMC splits the labeled set into two parts, and predicts the labels of one part using the known labels of the other part. The predicted labels are then used to learn the view combination coefficients. In the learning process, we adopt the average precision (AP) loss, which is particular suitable for multilabel image classification, since the ranking-based criteria are critical for evaluating a multilabel classification system. A least squares loss formulation is also presented for the sake of efficiency, and the robustness of the algorithm based on the AP loss compared with the other losses is investigated. Experimental evaluation on two real-world data sets (PASCAL VOC' 07 and MIR Flickr) demonstrate the effectiveness of MVMC for transductive (semisupervised) multilabel image classification, and show that MVMC can exploit complementary properties of different features and output-consistent labels for improved multilabel image classification.

[1]  M. Kloft,et al.  l p -Norm Multiple Kernel Learning , 2011 .

[2]  Yoram Singer,et al.  BoosTexter: A Boosting-based System for Text Categorization , 2000, Machine Learning.

[3]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2008, Found. Comput. Math..

[4]  Christopher D. Manning,et al.  Introduction to Information Retrieval , 2010, J. Assoc. Inf. Sci. Technol..

[5]  Bernhard Rinner,et al.  Vehicle Classification on Multi-Sensor Smart Cameras Using Feature- and Decision-Fusion , 2007, 2007 First ACM/IEEE International Conference on Distributed Smart Cameras.

[6]  Yong Luo,et al.  Group Sparse Multiview Patch Alignment Framework With View Consistency for Image Classification , 2014, IEEE Transactions on Image Processing.

[7]  Cordelia Schmid,et al.  Multimodal semi-supervised learning for image classification , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[8]  TaoDacheng,et al.  Large-Margin Multi-ViewInformation Bottleneck , 2014 .

[9]  Avrim Blum,et al.  The Bottleneck , 2021, Monopsony Capitalism.

[10]  Dacheng Tao,et al.  Large-Margin Multi-ViewInformation Bottleneck , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Arvind Ganesh,et al.  Fast Convex Optimization Algorithms for Exact Recovery of a Corrupted Low-Rank Matrix , 2009 .

[12]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[13]  Trevor Darrell,et al.  Co-training with noisy perceptual observations , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Xuelong Li,et al.  Asymmetric bagging and random subspace for support vector machines-based relevance feedback in image retrieval , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Shunsuke Ono,et al.  Cartoon-Texture Image Decomposition Using Blockwise Low-Rank Texture Characterization , 2014, IEEE Transactions on Image Processing.

[16]  Fabio Roli,et al.  A theoretical and experimental analysis of linear combiners for multiple classifier systems , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[18]  Yong Luo,et al.  Multiview Vector-Valued Manifold Regularization for Multilabel Image Classification , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[19]  Rayid Ghani,et al.  Analyzing the effectiveness and applicability of co-training , 2000, CIKM '00.

[20]  Martha White,et al.  Convex Multi-view Subspace Learning , 2012, NIPS.

[21]  Yufeng Liu,et al.  Robust Truncated Hinge Loss Support Vector Machines , 2007 .

[22]  Jiebo Luo,et al.  Learning multi-label scene classification , 2004, Pattern Recognit..

[23]  Michal Wozniak,et al.  Some Remarks on Chosen Methods of Classifier Fusion Based on Weighted Voting , 2009, HAIS.

[24]  Cordelia Schmid,et al.  Coloring Local Feature Extraction , 2006, ECCV.

[25]  Bo Wang,et al.  Dynamic Label Propagation for Semi-supervised Multi-class Multi-label Classification , 2013, ICCV.

[26]  Stéphane Marchand-Maillet,et al.  Information Fusion in Multimedia Information Retrieval , 2007, Adaptive Multimedia Retrieval.

[27]  Zhixun Su,et al.  Linearized Alternating Direction Method with Adaptive Penalty for Low-Rank Representation , 2011, NIPS.

[28]  Hervé Glotin,et al.  Cooperative Sparse Representation in Two Opposite Directions for Semi-Supervised Image Annotation , 2012, IEEE Transactions on Image Processing.

[29]  Thorsten Joachims,et al.  Cutting-plane training of structural SVMs , 2009, Machine Learning.

[30]  Ingo Steinwart,et al.  Consistency and robustness of kernel-based regression in convex risk minimization , 2007, 0709.0626.

[31]  Tom Fawcett,et al.  ROC Graphs: Notes and Practical Considerations for Researchers , 2007 .

[32]  Dacheng Tao,et al.  Multi-View Intact Space Learning , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Andrea Montanari,et al.  Matrix completion from a few entries , 2009, 2009 IEEE International Symposium on Information Theory.

[34]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[35]  Nello Cristianini,et al.  Learning the Kernel Matrix with Semidefinite Programming , 2002, J. Mach. Learn. Res..

[36]  Shiqian Ma,et al.  Fixed point and Bregman iterative methods for matrix rank minimization , 2009, Math. Program..

[37]  Xuelong Li,et al.  Principal Component 2-D Long Short-Term Memory for Font Recognition on Single Chinese Characters , 2016, IEEE Transactions on Cybernetics.

[38]  Vikas Sindhwani,et al.  Vector-valued Manifold Regularization , 2011, ICML.

[39]  John Shawe-Taylor,et al.  Two view learning: SVM-2K, Theory and Practice , 2005, NIPS.

[40]  Shie Mannor,et al.  Robustness and generalization , 2010, Machine Learning.

[41]  Alexandre Bernardino,et al.  Matrix Completion for Multi-label Image Classification , 2011, NIPS.

[42]  Andreas E. Savakis,et al.  LGE-KSVD: Robust Sparse Representation Classification , 2014, IEEE Transactions on Image Processing.

[43]  Steffen Bickel,et al.  Multi-view clustering , 2004, Fourth IEEE International Conference on Data Mining (ICDM'04).

[44]  Massimiliano Pontil,et al.  $K$ -Dimensional Coding Schemes in Hilbert Spaces , 2010, IEEE Transactions on Information Theory.

[45]  Henry Arguello,et al.  Rank Minimization Code Aperture Design for Spectrally Selective Compressive Imaging , 2013, IEEE Transactions on Image Processing.

[46]  Lihi Zelnik-Manor,et al.  Large Scale Max-Margin Multi-Label Classification with Priors , 2010, ICML.

[47]  Alexander Zien,et al.  lp-Norm Multiple Kernel Learning , 2011, J. Mach. Learn. Res..

[48]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[49]  Jieping Ye,et al.  Canonical Correlation Analysis for Multilabel Classification: A Least-Squares Formulation, Extensions, and Analysis , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[50]  Zhi-Hua Zhou,et al.  ML-KNN: A lazy learning approach to multi-label learning , 2007, Pattern Recognit..

[51]  Dapeng Tao,et al.  Local structure preserving discriminative projections for RGB-D sensor-based scene classification , 2015, Inf. Sci..

[52]  Ulf Brefeld,et al.  Co-EM support vector learning , 2004, ICML.

[53]  John Shawe-Taylor,et al.  Canonical Correlation Analysis: An Overview with Application to Learning Methods , 2004, Neural Computation.

[54]  Mikhail Belkin,et al.  A Co-Regularization Approach to Semi-supervised Learning with Multiple Views , 2005 .

[55]  Filip Radlinski,et al.  A support vector method for optimizing average precision , 2007, SIGIR.

[56]  Yong Luo,et al.  Manifold Regularized Multitask Learning for Semi-Supervised Multilabel Image Classification , 2013, IEEE Transactions on Image Processing.

[57]  Bhavani Raskutti,et al.  Optimising area under the ROC curve using gradient descent , 2004, ICML.

[58]  Byung-Woo Hong,et al.  FAST LABEL: Easy and Efficient Solution of Joint Multi-label and Estimation Problems , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[59]  Thorsten Joachims,et al.  A support vector method for multivariate performance measures , 2005, ICML.

[60]  Mark J. Huiskes,et al.  The MIR flickr retrieval evaluation , 2008, MIR '08.

[61]  Mikhail Belkin,et al.  Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples , 2006, J. Mach. Learn. Res..

[62]  Chiou-Shann Fuh,et al.  Dimensionality Reduction for Data in Multiple Feature Representations , 2008, NIPS.

[63]  Gert R. G. Lanckriet,et al.  Learning Multi-modal Similarity , 2010, J. Mach. Learn. Res..

[64]  Felipe Cucker,et al.  On the mathematical foundations of learning , 2001 .

[65]  Yong Luo,et al.  Pre-Trained Multi-View Word Embedding Using Two-Side Neural Network , 2014, AAAI.

[66]  Cheng Soon Ong,et al.  Multiclass multiple kernel learning , 2007, ICML '07.

[67]  Grigorios Tsoumakas,et al.  Multi-Label Classification: An Overview , 2007, Int. J. Data Warehous. Min..

[68]  Robert D. Nowak,et al.  Transduction with Matrix Completion: Three Birds with One Stone , 2010, NIPS.

[69]  B. V. K. Vijaya Kumar,et al.  Maximum Margin Correlation Filter: A New Approach for Localization and Classification , 2013, IEEE Transactions on Image Processing.

[70]  Yong Luo,et al.  Vector-Valued Multi-View Semi-Supervsed Learning for Multi-Label Image Classification , 2013, AAAI.

[71]  Yong Luo,et al.  Decomposition-Based Transfer Distance Metric Learning for Image Classification , 2014, IEEE Transactions on Image Processing.

[72]  Luc Van Gool,et al.  The 2005 PASCAL Visual Object Classes Challenge , 2005, MLCW.

[73]  Jianguo Zhang,et al.  The PASCAL Visual Object Classes Challenge , 2006 .

[74]  Joakim Lindblad,et al.  Linear Time Distances Between Fuzzy Sets With Applications to Pattern Matching and Classification , 2014, IEEE Transactions on Image Processing.

[75]  Thomas Hofmann,et al.  Large Margin Methods for Structured and Interdependent Output Variables , 2005, J. Mach. Learn. Res..

[76]  Trevor Darrell,et al.  Co-training with noisy perceptual observations , 2009, CVPR.

[77]  Lawrence Carin,et al.  Semi-Supervised Classification , 2004, Encyclopedia of Database Systems.