A Classifier Ensemble Framework for Multimedia Big Data Classification

Numerous classification algorithms have been developed for a variety of data types. However, it is nearly impossible for one classifier to perform the best in all kinds of datasets. Therefore, ensemble learning models which aim to take advantages of different classifiers have received a lot of attentions recently. In this paper, a scalable classifier ensemble framework assisted by a set of judgers is proposed to integrate the outputs from multiple classifiers for multimedia big data classification. Specifically, based on the confusion matrices of different classifiers, a set of "judgers" are organized into a hierarchically structured decision model. A testing instance is first input to different classifiers, and then the classification results are passed to the proposed hierarchical structured decision model to derive the final result. The ensemble system can be run on Spark, which is designed for big data processing. Experimental results on multimedia data containing different actions demonstrate that the proposed classifier ensemble framework outperforms several state-of-the-art model fusion approaches.

[1]  Rangasami L. Kashyap,et al.  Augmented transition networks as video browsing models for multimedia databases and multimedia information systems , 1999, Proceedings 11th International Conference on Tools with Artificial Intelligence.

[2]  Xiuqi Li,et al.  An effective content-based visual image retrieval system , 2002, Proceedings 26th Annual International Computer Software and Applications.

[3]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[4]  Mei-Ling Shyu,et al.  Effective Feature Space Reduction with Imbalanced Data for Semantic Concept Detection , 2008, 2008 IEEE International Conference on Sensor Networks, Ubiquitous, and Trustworthy Computing (sutc 2008).

[5]  Xin Huang,et al.  User Concept Pattern Discovery Using Relevance Feedback And Multiple Instance Learning For Content-Based Image Retrieval , 2002, MDM/KDD.

[6]  Mei-Ling Shyu,et al.  Negative Correlation Discovery for Big Multimedia Data Semantic Concept Mining and Retrieval , 2016, 2016 IEEE Tenth International Conference on Semantic Computing (ICSC).

[7]  Choochart Haruechaiyasak,et al.  Collaborative Filtering by Mining Association Rules from User Access Sequences , 2005, International Workshop on Challenges in Web Information Retrieval and Integration.

[8]  Mei-Ling Shyu,et al.  Leveraging Concept Association Network for Multimedia Rare Concept Mining and Retrieval , 2012, 2012 IEEE International Conference on Multimedia and Expo.

[9]  Barbara Caputo,et al.  Recognizing human actions: a local SVM approach , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[10]  Subhash C. Bagui,et al.  Combining Pattern Classifiers: Methods and Algorithms , 2005, Technometrics.

[11]  Shu-Ching Chen,et al.  Network intrusion detection through Adaptive Sub-Eigenspace Modeling in multiagent systems , 2007, ACM Trans. Auton. Adapt. Syst..

[12]  Mei-Ling Shyu,et al.  Effective Moving Object Detection and Retrieval via Integrating Spatial-Temporal Multimedia Information , 2012, 2012 IEEE International Symposium on Multimedia.

[13]  Mei-Ling Shyu,et al.  Sparse Linear Integration of Content and Context Modalities for Semantic Concept Retrieval , 2015, IEEE Transactions on Emerging Topics in Computing.

[14]  Marcelo Bernardes Vieira,et al.  Combining gradient histograms using orientation tensors for human action recognition , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[15]  Serge J. Belongie,et al.  Behavior recognition via sparse spatio-temporal features , 2005, 2005 IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance.

[16]  Robert P. W. Duin,et al.  The combining classifier: to train or not to train? , 2002, Object recognition supported by user interaction for service robots.

[17]  Gang Hua,et al.  Semantic Model Vectors for Complex Video Event Recognition , 2012, IEEE Transactions on Multimedia.

[18]  Juan Carlos Niebles,et al.  Modeling Temporal Structure of Decomposable Motion Segments for Activity Classification , 2010, ECCV.

[19]  Min Chen,et al.  Video Semantic Event/Concept Detection Using a Subspace-Based Multimedia Data Mining Framework , 2008, IEEE Transactions on Multimedia.

[20]  Mohamed A. Deriche,et al.  A New Technique for Combining Multiple Classifiers using The Dempster-Shafer Theory of Evidence , 2002, J. Artif. Intell. Res..

[21]  Chengcui Zhang,et al.  An intelligent framework for spatio-temporal vehicle tracking , 2001, ITSC 2001. 2001 IEEE Intelligent Transportation Systems. Proceedings (Cat. No.01TH8585).

[22]  Chengcui Zhang,et al.  A Dynamic User Concept Pattern Learning Framework for Content-Based Image Retrieval , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[23]  Hayder Radha,et al.  An Information-Theoretic Combining Method for Multi-Classifier Anomaly Detection Systems , 2010, 2010 IEEE International Conference on Communications.

[24]  Mei-Ling Shyu,et al.  Weighted Association Rule Mining for Video Semantic Detection , 2010, Int. J. Multim. Data Eng. Manag..

[25]  Rangasami L. Kashyap,et al.  Identifying Overlapped Objects for Video Indexing and Modeling in Multimedia Database Systems , 2001, Int. J. Artif. Intell. Tools.

[26]  Fang Chen,et al.  Investigating speech features and automatic measurement of cognitive load , 2008, 2008 IEEE 10th Workshop on Multimedia Signal Processing.

[27]  Jun-Wei Hsieh,et al.  PLSA-Based Sparse Representation for Object Classification , 2014, 2014 22nd International Conference on Pattern Recognition.

[28]  Rangasami L. Kashyap,et al.  Generalized Affinity-Based Association Rule Mining for Multimedia Database Queries , 2001, Knowledge and Information Systems.

[29]  Robert P. W. Duin,et al.  Experiments with Classifier Combining Rules , 2000, Multiple Classifier Systems.

[30]  Emmanuel Dellandréa,et al.  Multimodal recognition of visual concepts using histograms of textual concepts and selective weighted late fusion scheme , 2013, Comput. Vis. Image Underst..

[31]  Jiri Matas,et al.  On Combining Classifiers , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[32]  Min Chen,et al.  Utilizing concept correlations for effective imbalanced data classification , 2014, Proceedings of the 2014 IEEE 15th International Conference on Information Reuse and Integration (IEEE IRI 2014).

[33]  Mei-Ling Shyu,et al.  Multimodal Information Integration and Fusion for Histology Image Classification , 2011, Int. J. Multim. Data Eng. Manag..

[34]  Choochart Haruechaiyasak,et al.  Category cluster discovery from distributed WWW directories , 2003, Inf. Sci..

[35]  Adam Krzyżak,et al.  Methods of combining multiple classifiers and their applications to handwriting recognition , 1992, IEEE Trans. Syst. Man Cybern..

[36]  Ivan Laptev,et al.  On Space-Time Interest Points , 2005, International Journal of Computer Vision.

[37]  Jun-Wei Hsieh,et al.  Vehicle make and model recognition using sparse representation and symmetrical SURFs , 2013, 16th International IEEE Conference on Intelligent Transportation Systems (ITSC 2013).

[38]  Yan Meng,et al.  Human activity recognition in video using a hierarchical probabilistic latent model , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[39]  Min Chen,et al.  Spatio-Temporal Analysis for Human Action Detection and Recognition in Uncontrolled Environments , 2015, Int. J. Multim. Data Eng. Manag..

[40]  Ludmila I. Kuncheva,et al.  Combining Pattern Classifiers: Methods and Algorithms , 2004 .

[41]  Shu-Ching Chen,et al.  Feature Selection Using Correlation and Reliability Based Scoring Metric for Video Semantic Detection , 2010, 2010 IEEE Fourth International Conference on Semantic Computing.

[42]  Min Chen,et al.  Image database retrieval utilizing affinity relationships , 2003, MMDB '03.

[43]  Rangasami L. Kashyap,et al.  Temporal And Spatial Semantic Models For Multimedia Presentations , 1997 .

[44]  Min Chen,et al.  A unified framework for image database clustering and content-based retrieval , 2004, MMDB '04.

[45]  Chengcui Zhang,et al.  Innovative Shot Boundary Detection for Video Indexing , 2005 .

[46]  Rangasami L. Kashyap,et al.  Augmented Transition Network as a Semantic Model for Video Data , 2001 .

[47]  Xiuqi Li,et al.  Image Retrieval By Color , Texture , And Spatial Information , 2002 .

[48]  Shu-Ching Chen,et al.  Video Semantic Concept Discovery using Multimodal-Based Association Classification , 2007, 2007 IEEE International Conference on Multimedia and Expo.

[49]  Min Chen,et al.  Deep Learning for Imbalanced Multimedia Data Classification , 2015, 2015 IEEE International Symposium on Multimedia (ISM).

[50]  Guillermo Sapiro,et al.  Online Learning for Matrix Factorization and Sparse Coding , 2009, J. Mach. Learn. Res..