Multi-view manifold learning with locality alignment

Abstract Manifold learning aims to discover the low dimensional space where the input high dimensional data are embedded by preserving the geometric structure. Unfortunately, almost all the existing manifold learning methods were proposed under single view scenario, and they cannot be straightforwardly applied to multiple feature sets. Although concatenating multiple views into a single feature provides a plausible solution, it remains a question on how to better explore the independence and interdependence of different views while conducting manifold learning. In this paper, we propose a multi-view manifold learning with locality alignment (MVML-LA) framework to learn a common yet discriminative low-dimensional latent space that contain sufficient information of original inputs. Both supervised algorithm (S-MVML-LA) and unsupervised algorithm (U-MVML-LA) are developed. Experiments on benchmark real-world datasets demonstrate the superiority of our proposed S-MVML-LA and U-MVML-LA over existing state-of-the-art methods.

[1]  Dianfu Ma,et al.  Multiview Locally Linear Embedding for Effective Medical Image Retrieval , 2013, PloS one.

[2]  David G. Stork,et al.  Pattern classification, 2nd Edition , 2000 .

[3]  Arie Yeredor,et al.  MultiView Diffusion Maps , 2015, Inf. Fusion.

[4]  Dieter Fox,et al.  Unsupervised feature learning for 3D scene labeling , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[5]  Ali Farhadi,et al.  Toward a Taxonomy and Computational Models of Abnormalities in Images , 2015, AAAI.

[6]  Yu Liu,et al.  Multi-focus image fusion with dense SIFT , 2015, Inf. Fusion.

[7]  Ameet Talwalkar,et al.  Large-scale manifold learning , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Ethem Alpaydin,et al.  Multiple Kernel Learning Algorithms , 2011, J. Mach. Learn. Res..

[9]  José Carlos Príncipe,et al.  The C-loss function for pattern classification , 2014, Pattern Recognit..

[10]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[11]  Jieping Ye,et al.  Feature Reduction via Generalized Uncorrelated Linear Discriminant Analysis , 2006, IEEE Transactions on Knowledge and Data Engineering.

[12]  Ning Chen,et al.  Predictive Subspace Learning for Multi-view Data: a Large Margin Approach , 2010, NIPS.

[13]  Bin Shen,et al.  Learning dictionary on manifolds for image classification , 2013, Pattern Recognit..

[14]  H. Zha,et al.  Principal manifolds and nonlinear dimensionality reduction via tangent space alignment , 2004, SIAM J. Sci. Comput..

[15]  Shiguang Shan,et al.  Multi-View Discriminant Analysis , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Xianglei Xing,et al.  Complete canonical correlation analysis with application to multi-view gait recognition , 2016, Pattern Recognit..

[17]  Qingming Huang,et al.  Bilevel Multiview Latent Space Learning , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[18]  Shuicheng Yan,et al.  Neighborhood preserving embedding , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[19]  Ulrich Eckhardt,et al.  Linear convergence of generalized Weiszfeld's method , 1980, Computing.

[20]  Xiaohong Chen,et al.  A unified dimensionality reduction framework for semi-paired and semi-supervised multi-view data , 2012, Pattern Recognit..

[21]  Gene H. Golub,et al.  Matrix computations , 1983 .

[22]  Dong Yue,et al.  Multi-view low-rank dictionary learning for image classification , 2016, Pattern Recognit..

[23]  Pietro Perona,et al.  Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[24]  B. Thompson.  Canonical Correlation Analysis: Uses and Interpretation , 1984 .

[25]  Xiaofei He,et al.  Locality Preserving Projections , 2003, NIPS.

[26]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[27]  Ke Lu,et al.  Low-Rank Discriminant Embedding for Multiview Learning , 2017, IEEE Transactions on Cybernetics.

[28]  Ling Chen,et al.  Multi-layer multi-view topic model for classifying advertising video , 2017, Pattern Recognit..

[29]  Liang Wang,et al.  Unified subspace learning for incomplete and unlabeled multi-view data , 2017, Pattern Recognit..

[30]  R. Samworth.  Optimal weighted nearest neighbour classifiers , 2011, 1101.5783.

[31]  Maria-Florina Balcan,et al.  Co-Training and Expansion: Towards Bridging Theory and Practice , 2004, NIPS.

[32]  Zi Huang,et al.  Dimensionality reduction by Mixed Kernel Canonical Correlation Analysis , 2012, Pattern Recognition.

[33]  Zhenyu He,et al.  Joint sparse principal component analysis , 2017, Pattern Recognit..

[34]  Dacheng Tao,et al.  A Survey on Multi-view Learning , 2013, ArXiv.

[35]  Yifeng He,et al.  Multiview learning via deep discriminative canonical correlation analysis , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[36]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[37]  Ahmed M. Elgammal,et al.  Learning representations from multiple manifolds , 2016, Pattern Recognit..

[38]  Antonio Torralba,et al.  Recognizing indoor scenes , 2009, CVPR.

[39]  Yuan Shi,et al.  Information-Theoretical Learning of Discriminative Clusters for Unsupervised Domain Adaptation , 2012, ICML.

[40]  Ming Shao,et al.  Transfer learning for image classification with incomplete multiple sources , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[41]  D. Donoho,et al.  Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[42]  Zheng Cao,et al.  Robust linear discriminant analysis with a Laplacian assumption on projection distribution , 2017, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[43]  Deli Zhao,et al.  Linear local tangent space alignment and application to face recognition , 2007, Neurocomputing.

[44]  Zhenyu He,et al.  A multi-view model for visual tracking via correlation filters , 2016, Knowl. Based Syst..

[45]  Weihua Ou,et al.  Multi-view non-negative matrix factorization by patch alignment framework with view consistency , 2016, Neurocomputing.

[46]  Pengfei Shi,et al.  A Novel Method of Combined Feature Extraction for Recognition , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[47]  Dieter Fox,et al.  A large-scale hierarchical multi-view RGB-D object dataset , 2011, 2011 IEEE International Conference on Robotics and Automation.

[48]  Dacheng Tao,et al.  Multi-View Intact Space Learning , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[49]  Hongyuan Zha,et al.  Principal Manifolds and Nonlinear Dimension Reduction via Local Tangent Space Alignment , 2002, ArXiv.

[50]  Xuelong Li,et al.  Patch Alignment for Dimensionality Reduction , 2009, IEEE Transactions on Knowledge and Data Engineering.

[51]  Yun Fu,et al.  Robust Multi-View Subspace Learning through Dual Low-Rank Decompositions , 2016, AAAI.

[52]  Nicolas Le Roux,et al.  Out-of-Sample Extensions for LLE, Isomap, MDS, Eigenmaps, and Spectral Clustering , 2003, NIPS.

[53]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[54]  Daewon Lee,et al.  Inductive manifold learning using structured support vector machine , 2014, Pattern Recognit..

[55]  Sameer A. Nene,et al.  Columbia Object Image Library (COIL100) , 1996 .

[56]  Shiliang Sun,et al.  Multiview Uncorrelated Discriminant Analysis , 2016, IEEE Transactions on Cybernetics.

[57]  Yousef Saad,et al.  Orthogonal Neighborhood Preserving Projections: A Projection-Based Dimensionality Reduction Technique , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[58]  Dieter Fox,et al.  Hierarchical Matching Pursuit for Image Classification: Architecture and Fast Algorithms , 2011, NIPS.

[59]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[60]  Fuchun Sun,et al.  Large-Margin Predictive Latent Subspace Learning for Multiview Data Analysis , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[61]  Tong Lu,et al.  Learning discriminated and correlated patches for multi-view object detection using sparse coding , 2017, Pattern Recognit..