Universum Discriminant Canonical Correlation Analysis

Over the past decades, extensive studies on multi-view learning and Universum learning have been witnessed in pattern recognition and machine learning. Incorporating multi-view learning and Universum learning together, we propose a novel supervised dimensionality reduction method for multi-view data accompanied by Universum data, termed Universum discriminant canonical correlation analysis (UDCCA). UDCCA exploits inter-view information by means of the within-class correlation as well as the within-Universum correlation between different views, and at the same time, utilizes intra-view discriminant information captured from the target samples and Universum data of each view. In the low-dimensional discriminant space, the within-class correlation of the target samples is maximized, and the correlation of the Universum data is minimized, and simultaneously the scatter among target samples and Universum data is also maximized. Experimental results on real-world multi-view datasets show its effectiveness compared to other related state-of-art dimensionality reduction methods.

[1]  Mohak Shah,et al.  Universum Learning for Multiclass SVM , 2016, ArXiv.

[2]  Shiguang Shan,et al.  Multi-View Discriminant Analysis , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Shiliang Sun,et al.  Multiview Uncorrelated Discriminant Analysis , 2016, IEEE Transactions on Cybernetics.

[4]  V. Vapnik.  Estimation of Dependences Based on Empirical Data , 2006 .

[5]  H. Hotelling.  Relations Between Two Sets of Variates , 1936 .

[6]  Robert P. W. Duin,et al.  Handwritten digit recognition by combined classifiers , 1998, Kybernetika.

[7]  G. Dunteman.  Principal Components Analysis , 1989 .

[8]  Ming Yang,et al.  Multi-View Representation Learning: A Survey from Shallow Methods to Deep Methods , 2016, ArXiv.

[9]  Bernhard Schölkopf,et al.  An Analysis of Inference with the Universum , 2007, NIPS.

[10]  Pengfei Shi,et al.  A Novel Method of Combined Feature Extraction for Recognition , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[11]  Shiliang Sun,et al.  Hierarchical Multi-view Fisher Discriminant Analysis , 2009, ICONIP.

[12]  Bart De Moor,et al.  Meta-clustering of gene expression data and literature-based information , 2003, SKDD.

[13]  Dacheng Tao,et al.  A Survey on Multi-view Learning , 2013, ArXiv.

[14]  Hui Xue,et al.  Universum linear discriminant analysis , 2012 .

[15]  Xin Li,et al.  A novel semi-supervised approach for feature extraction , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[16]  Nicholas Kushmerick,et al.  Learning to remove Internet advertisements , 1999, AGENTS '99.

[17]  Chengqi Zhang,et al.  Boosting for graph classification with universum , 2016, Knowledge and Information Systems.

[18]  Fei Wang,et al.  Semi-Supervised Classification with Universum , 2008, SDM.

[19]  Dongdong Li,et al.  Regularized Matrix-Pattern-Oriented Classification Machine with Universum , 2017, Neural Processing Letters.

[20]  Jason Weston,et al.  Inference with the Universum , 2006, ICML.

[21]  John Shawe-Taylor,et al.  Canonical Correlation Analysis: An Overview with Application to Learning Methods , 2004, Neural Computation.

[22]  R. Fisher.  THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[23]  Changshui Zhang,et al.  Selecting Informative Universum Sample for Semi-Supervised Learning , 2009, IJCAI.

[24]  Christoph H. Lampert,et al.  Learning to detect unseen object classes by between-class attribute transfer , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Shiliang Sun,et al.  A survey of multi-view machine learning , 2013, Neural Computing and Applications.

[26]  Wenwen Liu,et al.  Multi-view learning with Universum , 2014, Knowl. Based Syst..

[27]  A. Maćkiewicz,et al.  Principal Components Analysis (PCA) , 1993 .