Multiview Hybrid Embedding: A Divide-and-Conquer Approach

We present a novel cross-view classification algorithm where the gallery and probe data come from different views. A popular approach to tackle this problem is the multiview subspace learning (MvSL) that aims to learn a latent subspace shared by multiview data. Despite promising results obtained on some applications, the performance of existing methods deteriorates dramatically when the multiview data is sampled from nonlinear manifolds or suffers from heavy outliers. To circumvent this drawback, motivated by the Divide-and-Conquer strategy, we propose multiview hybrid embedding (MvHE), a unique method of dividing the problem of cross-view classification into three subproblems and building one model for each subproblem. Specifically, the first model is designed to remove view discrepancy, whereas the second and third models attempt to discover the intrinsic nonlinear structure and to increase the discriminability in intraview and interview samples, respectively. The kernel extension is conducted to further boost the representation power of MvHE. Extensive experiments are conducted on four benchmark datasets. Our methods demonstrate the overwhelming advantages against the state-of-the-art MvSL-based cross-view classification approaches in terms of classification accuracy and robustness.

[1]  H. Hotelling Relations Between Two Sets of Variates , 1936 .

[2]  Ioannis A. Kakadiaris,et al.  Semi-coupled basis and distance metric learning for cross-domain matching: Application to low-resolution face recognition , 2014, IEEE International Joint Conference on Biometrics.

[3]  Qiuping Xu Canonical correlation Analysis , 2014 .

[4]  Yehuda Koren,et al.  Ieee Transactions on Visualization and Computer Graphics Robust Linear Dimensionality Reduction , 2022 .

[5]  Yu Qiao,et al.  Multi-feature canonical correlation analysis for face photo-sketch image retrieval , 2013, ACM Multimedia.

[6]  Jeff A. Bilmes,et al.  Unsupervised learning of acoustic features via deep canonical correlation analysis , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[7]  Ke Lu,et al.  Low-Rank Discriminant Embedding for Multiview Learning , 2017, IEEE Transactions on Cybernetics.

[8]  J. Shawe-Taylor,et al.  Multi-View Canonical Correlation Analysis , 2010 .

[9]  Xuelong Li,et al.  Patch Alignment for Dimensionality Reduction , 2009, IEEE Transactions on Knowledge and Data Engineering.

[10]  Tom Diethe,et al.  Constructing Nonlinear Discriminants from Multiple Data Views , 2010, ECML/PKDD.

[11]  Allan Aasbjerg Nielsen,et al.  Multiset canonical correlations analysis and multispectral, truly multitemporal remote sensing data , 2002, IEEE Trans. Image Process..

[12]  Yun Fu,et al.  Robust Subspace Discovery through Supervised Low-Rank Constraints , 2014, SDM.

[13]  Xiaofei He,et al.  Locality Preserving Projections , 2003, NIPS.

[14]  Shiguang Shan,et al.  Multi-view Deep Network for Cross-View Classification , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Chunheng Wang,et al.  Regularized Latent Least Square Regression for Cross Pose Face Recognition , 2013, IJCAI.

[16]  Juhan Nam,et al.  Multimodal Deep Learning , 2011, ICML.

[17]  Wei Yuan,et al.  Multi-view manifold learning with locality alignment , 2018, Pattern Recognit..

[18]  Ning Chen,et al.  Predictive Subspace Learning for Multi-view Data: a Large Margin Approach , 2010, NIPS.

[19]  Stefanos Zafeiriou,et al.  Subspace Learning from Image Gradient Orientations , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Joshua B. Tenenbaum,et al.  Global Versus Local Methods in Nonlinear Dimensionality Reduction , 2002, NIPS.

[21]  Wei Wang,et al.  Learning Coupled Feature Spaces for Cross-Modal Matching , 2013, 2013 IEEE International Conference on Computer Vision.

[22]  Saeed Shiry Ghidary,et al.  Dimensionality reduction based on distance preservation to local mean for symmetric positive definite matrices and its application in brain–computer interfaces , 2016, Journal of neural engineering.

[23]  Stan Z. Li,et al.  The HFB Face Database for Heterogeneous Face Biometrics research , 2009, 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[24]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[25]  Xiao-Yuan Jing,et al.  Super-resolution Person re-identification with semi-coupled low-rank discriminant dictionary learning , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Shiliang Sun,et al.  Multiview Uncorrelated Discriminant Analysis , 2016, IEEE Transactions on Cybernetics.

[27]  D. Jacobs,et al.  Bypassing synthesis: PLS for face recognition with pose, low-resolution and sketch , 2011, CVPR 2011.

[28]  Ming Shao,et al.  Deep Low-Rank Coding for Transfer Learning , 2015, IJCAI.

[29]  Weihua Ou,et al.  Multi-view non-negative matrix factorization by patch alignment framework with view consistency , 2016, Neurocomputing.

[30]  Ioannis A. Kakadiaris,et al.  An Overview and Empirical Comparison of Distance Metric Learning Methods , 2017, IEEE Transactions on Cybernetics.

[31]  Jing-Yu Yang,et al.  Face recognition based on the uncorrelated discriminant transformation , 2001, Pattern Recognit..

[32]  David W. Jacobs,et al.  Generalized Multiview Analysis: A discriminative latent space , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[33]  Bernhard Schölkopf,et al.  A Generalized Representer Theorem , 2001, COLT/EuroCOLT.

[34]  Wei Wang,et al.  Continuum regression for cross-modal multimedia retrieval , 2012, 2012 19th IEEE International Conference on Image Processing.

[35]  Eric O. Postma,et al.  Dimensionality Reduction: A Comparative Review , 2008 .

[36]  Guillermo Sapiro,et al.  Connecting the Out-of-Sample and Pre-Image Problems in Kernel Methods , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[37]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[38]  Tom Diethe,et al.  Multiview Fisher Discriminant Analysis , 2008 .

[39]  Stan Z. Li,et al.  Coupled Spectral Regression for matching heterogeneous faces , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[40]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[41]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[42]  Horst Bischof,et al.  Nonlinear Feature Extraction Using Generalized Canonical Correlation Analysis , 2001, ICANN.

[43]  Alexandros Iosifidis,et al.  Multi-View Nonparametric Discriminant Analysis for Image Retrieval and Recognition , 2017, IEEE Signal Processing Letters.

[44]  Yan Chen,et al.  Simultaneous and orthogonal decomposition of data using Multimodal Discriminant Analysis , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[45]  Ronald L. Rivest,et al.  Introduction to Algorithms, third edition , 2009 .

[46]  Jeff A. Bilmes,et al.  Deep Canonical Correlation Analysis , 2013, ICML.

[47]  Dacheng Tao,et al.  A Survey on Multi-view Learning , 2013, ArXiv.

[48]  Shiguang Shan,et al.  Multi-View Discriminant Analysis , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[49]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[50]  Terence Sim,et al.  The CMU Pose, Illumination, and Expression (PIE) database , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[51]  Yun Fu,et al.  Low-Rank Common Subspace for Multi-view Learning , 2014, 2014 IEEE International Conference on Data Mining.

[52]  Dahua Lin,et al.  Inter-modality Face Recognition , 2006, ECCV.

[53]  Dong Yi,et al.  Face Matching Between Near Infrared and Visible Light Images , 2007, ICB.

[54]  Xuelong Li,et al.  Learning Discriminative Binary Codes for Large-scale Cross-modal Retrieval , 2017, IEEE Transactions on Image Processing.

[55]  Xiaohua Zhai,et al.  Heterogeneous Metric Learning with Joint Graph Regularization for Cross-Media Retrieval , 2013, AAAI.

[56]  Shotaro Akaho,et al.  A kernel method for canonical correlation analysis , 2006, ArXiv.

[57]  Xiaogang Wang,et al.  Locally Aligned Feature Transforms across Views , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[58]  Ming Yang,et al.  Multi-View Representation Learning: A Survey from Shallow Methods to Deep Methods , 2016, ArXiv.

[59]  V. Kshirsagar,et al.  Face recognition using Eigenfaces , 2011, 2011 3rd International Conference on Computer Research and Development.

[60]  Ronald L. Rivest,et al.  Introduction to Algorithms , 1990 .

[61]  Roman Rosipal,et al.  Overview and Recent Advances in Partial Least Squares , 2005, SLSFS.