Ensemble learning from multiple information sources via label propagation and consensus

Many applications are facing the problem of learning from multiple information sources, where sources may be labeled or unlabeled, and information from multiple information sources may be beneficial but cannot be integrated into a single information source for learning. In this paper, we propose an ensemble learning method for different labeled and unlabeled sources. We first present two label propagation methods to infer the labels of training objects from unlabeled sources by making a full use of class label information from labeled sources and internal structure information from unlabeled sources, which are processes referred to as global consensus and local consensus, respectively. We then predict the labels of testing objects using the ensemble learning model of multiple information sources. Experimental results show that our method outperforms two baseline methods. Meanwhile, our method is more scalable for large information sources and is more robust for labeled sources with noisy data.

[1]  Michael H. Böhlen,et al.  The address connector: noninvasive synchronization of hierarchical data sources , 2012, Knowledge and Information Systems.

[2]  Franco Turini,et al.  Stream mining: a novel architecture for ensemble-based classification , 2011, Knowledge and Information Systems.

[3]  Tin Kam Ho,et al.  The Random Subspace Method for Constructing Decision Forests , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Brijesh Verma,et al.  Hybrid ensemble approach for classification , 2011, Applied Intelligence.

[5]  Philip S. Yu,et al.  Efficient classification across multiple database relations: a CrossMine approach , 2006, IEEE Transactions on Knowledge and Data Engineering.

[6]  Faïez Gargouri,et al.  A new semi-supervised hierarchical active clustering based on ranking constraints for analysts groupization , 2012, Applied Intelligence.

[7]  Ruoming Jin,et al.  Multiple Information Sources Cooperative Learning , 2009, IJCAI.

[8]  Jorma Laaksonen,et al.  Using diversity of errors for selecting members of a committee classifier , 2006, Pattern Recognit..

[9]  Xindong Wu,et al.  Mining globally interesting patterns from multiple databases using kernel estimation , 2009, Expert Syst. Appl..

[10]  Jiri Matas,et al.  On Combining Classifiers , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Luc Vandendorpe,et al.  Multiple classifier combination for face-based identity verification , 2004, Pattern Recognit..

[12]  Xindong Wu,et al.  CLAP: Collaborative pattern mining for distributed information systems , 2011, Decis. Support Syst..

[13]  Michael I. Jordan,et al.  The Handbook of Brain Theory and Neural Networks , 2002 .

[14]  Hoang Huu Viet,et al.  BA*: an online complete coverage algorithm for cleaning robots , 2012, Applied Intelligence.

[15]  Tao Li,et al.  Semisupervised learning from different information sources , 2005, Knowledge and Information Systems.

[16]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[17]  Yoram Singer,et al.  Improved Boosting Algorithms Using Confidence-rated Predictions , 1998, COLT' 98.

[18]  Hervé Glotin,et al.  Cooperative Sparse Representation in Two Opposite Directions for Semi-Supervised Image Annotation , 2012, IEEE Transactions on Image Processing.

[19]  Juan José Rodríguez Diez,et al.  A weighted voting framework for classifiers ensembles , 2012, Knowledge and Information Systems.

[20]  Aixia Guo,et al.  Gene Selection for Cancer Classification using Support Vector Machines , 2014 .

[21]  Michael A. Arbib,et al.  The handbook of brain theory and neural networks , 1995, A Bradford book.

[22]  Xindong Wu,et al.  Multi-level rough set reduction for decision rule mining , 2013, Applied Intelligence.

[23]  R. Schapire.  The Strength of Weak Learnability , 1990, Machine Learning.

[24]  José M. Molina López,et al.  Multi-agent plan based information gathering , 2006, Applied Intelligence.

[25]  Witold Pedrycz,et al.  A new selective neural network ensemble with negative correlation , 2012, Applied Intelligence.

[26]  Xindong Wu,et al.  Synthesizing High-Frequency Rules from Different Data Sources , 2003, IEEE Trans. Knowl. Data Eng..

[27]  Hui Xiong,et al.  Cross-Domain Learning from Multiple Sources: A Consensus Regularization Perspective , 2010, IEEE Transactions on Knowledge and Data Engineering.

[28]  Yizhou Sun,et al.  Graph-based Consensus Maximization among Multiple Supervised and Unsupervised Models , 2009, NIPS.

[29]  Parag Kulkarni,et al.  A Survey of Semi-Supervised Learning Methods , 2008, 2008 International Conference on Computational Intelligence and Security.

[30]  Fabio Roli,et al.  A theoretical and experimental analysis of linear combiners for multiple classifier systems , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Xiaojin Zhu,et al.  --1 CONTENTS , 2006 .

[32]  Ching-Wei Wang,et al.  Boosting-SVM: effective learning with reduced data dimension , 2013, Applied Intelligence.

[33]  Animesh Adhikari,et al.  Synthesizing heavy association rules from different real data sources , 2008, Pattern Recognit. Lett..

[34]  Oleksandr Makeyev,et al.  Neural network with ensembles , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[35]  Yizhou Sun,et al.  Heterogeneous source consensus learning via decision propagation and negotiation , 2009, KDD.

[36]  Paul M. Thompson,et al.  Multi-source feature learning for joint analysis of incomplete multiple heterogeneous neuroimaging data , 2012, NeuroImage.

[37]  Bernhard Schölkopf,et al.  Learning with Local and Global Consistency , 2003, NIPS.

[38]  Alun D. Preece,et al.  Designing for scalability in a knowledge fusion system , 2001, Knowl. Based Syst..

[39]  Naonori Ueda,et al.  Adaptive semi-supervised learning on labeled and unlabeled data with different distributions , 2012, Knowledge and Information Systems.

[40]  Yoav Freund,et al.  Boosting a weak learning algorithm by majority , 1995, COLT '90.

[41]  Witold Pedrycz,et al.  Study of select items in different data sources by grouping , 2010, Knowledge and Information Systems.

[42]  Li Guo,et al.  Classifier and Cluster Ensembles for Mining Concept Drifting Data Streams , 2010, 2010 IEEE International Conference on Data Mining.

[43]  Min Han,et al.  Semi-supervised Bayesian ARTMAP , 2010, Applied Intelligence.