A novel ensemble construction method for multi-view data using random cross-view correlation between within-class examples

Correlated information between multiple views can provide useful information for building robust classifiers. One way to extract correlated features from different views is using canonical correlation analysis (CCA). However, CCA is an unsupervised method and can not preserve discriminant information in feature extraction. In this paper, we first incorporate discriminant information into CCA by using random cross-view correlations between within-class examples. Because of the random property, we can construct a lot of feature extractors based on CCA and random correlation. So furthermore, we fuse those feature extractors and propose a novel method called random correlation ensemble (RCE) for multi-view ensemble learning. We compare RCE with existing multi-view feature extraction methods including CCA and discriminant CCA (DCCA) which use all cross-view correlations between within-class examples, as well as the trivial ensembles of CCA and DCCA which adopt standard bagging and boosting strategies for ensemble learning. Experimental results on several multi-view data sets validate the effectiveness of the proposed method.

[1]  Yoav Freund,et al.  Experiments with a New Boosting Algorithm , 1996, ICML.

[2]  Deng Cai,et al.  Laplacian Score for Feature Selection , 2005, NIPS.

[3]  Zhi-HuaZhou,et al.  Adapt Bagging to Nearest Neighbor Classifiers , 2005 .

[4]  David G. Stork,et al.  Pattern Classification , 1973 .

[5]  Hongyu Guo,et al.  Mining relational databases with multi-view learning , 2005, MRDM '05.

[6]  Shotaro Akaho,et al.  A kernel method for canonical correlation analysis , 2006, ArXiv.

[7]  Sham M. Kakade,et al.  Multi-view Regression Via Canonical Correlation Analysis , 2007, COLT.

[8]  Tom Diethe,et al.  Multiview Fisher Discriminant Analysis , 2008 .

[9]  Pengfei Shi,et al.  A Novel Method of Combined Feature Extraction for Recognition , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[10]  Yuxiao Hu,et al.  Learning a Spatially Smooth Subspace for Face Recognition , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Juan José Rodríguez Diez,et al.  Rotation Forest: A New Classifier Ensemble Method , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Tin Kam Ho,et al.  The Random Subspace Method for Constructing Decision Forests , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Qiang Yang,et al.  Semi-Supervised Learning with Very Few Labeled Training Examples , 2007, AAAI.

[14]  Sham M. Kakade,et al.  Multi-view clustering via canonical correlation analysis , 2009, ICML '09.

[15]  H. Hotelling Relations Between Two Sets of Variates , 1936 .

[16]  Yan Liu,et al.  A new method of feature fusion and its application in image recognition , 2005, Pattern Recognit..

[17]  HoTin Kam The Random Subspace Method for Constructing Decision Forests , 1998 .

[18]  David Yarowsky,et al.  Unsupervised Word Sense Disambiguation Rivaling Supervised Methods , 1995, ACL.

[19]  Oleg Okun,et al.  Multiple Views in Ensembles of Nearest Neighbor Classifiers , 2005 .

[20]  David G. Stork,et al.  Pattern classification, 2nd Edition , 2000 .

[21]  Rayid Ghani,et al.  Analyzing the effectiveness and applicability of co-training , 2000, CIKM '00.

[22]  Avrim Blum,et al.  The Bottleneck , 2021, Monopsony Capitalism.

[23]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[24]  Ammad Ali,et al.  Face Recognition with Local Binary Patterns , 2012 .

[25]  Ian Witten,et al.  Data Mining , 2000 .

[26]  Steffen Bickel,et al.  Multi-view clustering , 2004, Fourth IEEE International Conference on Data Mining (ICDM'04).

[27]  Thomas G. Dietterich Multiple Classifier Systems , 2000, Lecture Notes in Computer Science.