Weighted hybrid fusion with rank consistency

Abstract This paper proposes a weighted hybrid multi-view fusion method for the semi-supervised classification problem. Instead of getting access to the features from different views directly, this method utilizes the square losses of the multi-view classifiers to exploit the between-view relationship, which preserves the privacy of data. Considering the different prediction capability of classifiers on multiple views, an objective function with the constraint of rank consistency is constructed to weight view-specific learners adaptively, where the constraint makes each view-specific learner improve its performance by exploring the predicted results of other learners. Furthermore, an iterative algorithm based on the Variant Alternating Splitting Augmented Lagrangian Method (VASALM) and the quadratic programming method is developed to optimize the objective function. Experimental results on different real-world datasets demonstrate the effectiveness of the proposed method for multi-view learning. The experiments also analyze parameter sensitivity and convergency of the optimization algorithm.

[1]  Qinghua Zheng,et al.  Adaptive Semi-Supervised Learning with Discriminative Least Squares Regression , 2017, IJCAI.

[2]  Mohan S. Kankanhalli,et al.  Multimodal fusion for multimedia analysis: a survey , 2010, Multimedia Systems.

[3]  Fabio A. González,et al.  Multimodal representation, indexing, automated annotation and retrieval of image collections via non-negative matrix factorization , 2012, Neurocomputing.

[4]  Ling Shao,et al.  Dynamic Multi-View Hashing for Online Image Retrieval , 2017, IJCAI.

[5]  Feiping Nie,et al.  Unsupervised Single and Multiple Views Feature Extraction with Structured Graph , 2017, IEEE Transactions on Knowledge and Data Engineering.

[6]  Xuelong Li,et al.  Matrix completion by Truncated Nuclear Norm Regularization , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[8]  Michael I. Jordan,et al.  Multiple kernel learning, conic duality, and the SMO algorithm , 2004, ICML.

[9]  Zhao Wang,et al.  Adaptive multi-view feature selection for human motion retrieval , 2016, Signal Process..

[10]  Lise Getoor,et al.  Collective Classification in Network Data , 2008, AI Mag..

[11]  Xuelong Li,et al.  Fast and Accurate Matrix Completion via Truncated Nuclear Norm Regularization , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Michael K. Ng,et al.  Feature weight estimation for gene selection: a local hyperlinear learning approach , 2014, BMC Bioinformatics.

[13]  Jing Liu,et al.  Partially Shared Latent Factor Learning With Multiview Data , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[14]  Lei Gao,et al.  Discriminative Multiple Canonical Correlation Analysis for Information Fusion , 2018, IEEE Transactions on Image Processing.

[15]  Ivor W. Tsang,et al.  Co-Labeling for Multi-View Weakly Labeled Learning , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Feiping Nie,et al.  Discriminatively Embedded K-Means for Multi-view Clustering , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Ling Guan,et al.  Joint intermodal and intramodal correlation preservation for semi-paired learning , 2018, Pattern Recognit..

[18]  Louis-Philippe Morency,et al.  Multimodal Machine Learning: A Survey and Taxonomy , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Michael I. Jordan,et al.  A Probabilistic Interpretation of Canonical Correlation Analysis , 2005 .

[20]  Ling Guan,et al.  Information Fusion for Human Action Recognition via Biset/Multiset Globality Locality Preserving Canonical Correlation Analysis , 2018, IEEE Transactions on Image Processing.

[21]  Chenping Hou,et al.  Robust auto-weighted multi-view subspace clustering with common subspace representation matrix , 2017, PloS one.

[22]  Mehryar Mohri,et al.  Two-Stage Learning Kernel Algorithms , 2010, ICML.

[23]  Qingming Huang,et al.  GOMES: A group-aware multi-view fusion approach towards real-world image clustering , 2015, 2015 IEEE International Conference on Multimedia and Expo (ICME).

[24]  Feiping Nie,et al.  A Unified Weight Learning Paradigm for Multi-view Learning , 2019, AISTATS.

[25]  Xiaoming Yuan,et al.  Recovering Low-Rank and Sparse Components of Matrices from Incomplete and Noisy Observations , 2011, SIAM J. Optim..

[26]  Xiaoming Yuan,et al.  A splitting method for separable convex programming , 2015 .

[27]  Wei Liu,et al.  Multimedia classification and event detection using double fusion , 2013, Multimedia Tools and Applications.

[28]  H. Hotelling Relations Between Two Sets of Variates , 1936 .

[29]  Zi Huang,et al.  Multi-Feature Fusion via Hierarchical Regression for Multimedia Analysis , 2013, IEEE Transactions on Multimedia.

[30]  Ronald Phlypo,et al.  Adaptive Feature Selection and Feature Fusion for Semi-supervised Classification , 2019, J. Signal Process. Syst..

[31]  Meng Wang,et al.  Optimizing multi-graph learning: towards a unified video annotation scheme , 2007, ACM Multimedia.

[32]  Ling Guan,et al.  Graph-Based Safe Support Vector Machine for Multiple Classes , 2018, IEEE Access.

[33]  Limin Wang,et al.  Multi-view Super Vector for Action Recognition , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[34]  Zhi-Hua Zhou,et al.  One-Pass Multi-View Learning , 2015, ACML.

[35]  Chang-Dong Wang,et al.  Weighted Multi-view Clustering with Feature Selection , 2016, Pattern Recognit..

[36]  Zhihong Zeng,et al.  A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions , 2009, IEEE Trans. Pattern Anal. Mach. Intell..

[37]  Yi Yang,et al.  Semi-Supervised Multiple Feature Analysis for Action Recognition , 2014, IEEE Transactions on Multimedia.

[38]  Masayuki Karasuyama,et al.  Multiple Graph Label Propagation by Sparse Integration , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[39]  Rongrong Ji,et al.  Cross-Modality Binary Code Learning via Fusion Similarity Hashing , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[40]  Yunming Ye,et al.  A new weighting k-means type clustering framework with an l2-norm regularization , 2018, Knowl. Based Syst..

[41]  Xuelong Li,et al.  Multi-View Clustering and Semi-Supervised Classification with Adaptive Neighbours , 2017, AAAI.

[42]  Ethem Alpaydin,et al.  Localized multiple kernel learning , 2008, ICML '08.

[43]  Lei Gao,et al.  A Novel Discriminative Framework Integrating Kernel Entropy Component Analysis and Discriminative Multiple Canonical Correlation for Information Fusion , 2016, 2016 IEEE International Symposium on Multimedia (ISM).

[44]  Xiaojun Wu,et al.  Graph Regularized Nonnegative Matrix Factorization for Data Representation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[46]  Cordelia Schmid,et al.  Multimodal semi-supervised learning for image classification , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[47]  Dong Liu,et al.  Sample-Specific Late Fusion for Visual Category Recognition , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[48]  Dong Liu,et al.  Robust late fusion with rank minimization , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[49]  Chang-Dong Wang,et al.  Weighted Multi-view On-Line Competitive Clustering , 2016, 2016 IEEE Second International Conference on Big Data Computing Service and Applications (BigDataService).

[50]  Tat-Seng Chua,et al.  Fusion of AV features and external information sources for event detection in team sports video , 2006, TOMCCAP.

[51]  Shao-Yuan Li,et al.  Partial Multi-View Clustering , 2014, AAAI.

[52]  Qi Zhao,et al.  Co-Tracking Using Semi-Supervised Support Vector Machines , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[53]  Xuelong Li,et al.  Parameter-Free Auto-Weighted Multiple Graph Learning: A Framework for Multiview Clustering and Semi-Supervised Classification , 2016, IJCAI.

[54]  Lin Wu,et al.  Iterative Views Agreement: An Iterative Low-Rank Based Structured Optimization Method to Multi-View Spectral Clustering , 2016, IJCAI.

[55]  Pablo A. Parrilo,et al.  Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization , 2007, SIAM Rev..