Classifier fusion by judgers on spark clusters for multimedia big data classification

The exponential growth of multimedia data including images and videos has been witnessed on social media websites like Instagram and YouTube. With the rapid growth of multimedia data size, efficient processing of these big data becomes more and more important. Meanwhile, lots of classifiers have been proposed for a number of data types. However, how to assemble these classifiers efficiently remains a challenging research issue. In this paper, a novel scalable framework is proposed for classifier ensemble using a set of judgers generated based on the training and validation results. These judgers are ranked and put together as a hierarchically structured decision model. The proposed ensemble framework is deployed on an Apache Spark cluster for efficient data processing. Our experimental results on multimedia datasets containing different actions show that our ensemble work performs better than several state-of-the-art model fusion approaches.

[1]  Arnaldo de Albuquerque Araújo,et al.  Combining Orientation Tensors for Human Action Recognition , 2013, 2013 XXVI Conference on Graphics, Patterns and Images.

[2]  Shu-Ching Chen,et al.  Network intrusion detection through Adaptive Sub-Eigenspace Modeling in multiagent systems , 2007, ACM Trans. Auton. Adapt. Syst..

[3]  Mei-Ling Shyu,et al.  Effective Moving Object Detection and Retrieval via Integrating Spatial-Temporal Multimedia Information , 2012, 2012 IEEE International Symposium on Multimedia.

[4]  Choochart Haruechaiyasak,et al.  Category cluster discovery from distributed WWW directories , 2003, Inf. Sci..

[5]  Adam Krzyżak,et al.  Methods of combining multiple classifiers and their applications to handwriting recognition , 1992, IEEE Trans. Syst. Man Cybern..

[6]  Jun-Wei Hsieh,et al.  Vehicle make and model recognition using sparse representation and symmetrical SURFs , 2013, 16th International IEEE Conference on Intelligent Transportation Systems (ITSC 2013).

[7]  Gang Hua,et al.  Semantic Model Vectors for Complex Video Event Recognition , 2012, IEEE Transactions on Multimedia.

[8]  Juan Carlos Niebles,et al.  Modeling Temporal Structure of Decomposable Motion Segments for Activity Classification , 2010, ECCV.

[9]  Min Chen,et al.  Utilizing concept correlations for effective imbalanced data classification , 2014, Proceedings of the 2014 IEEE 15th International Conference on Information Reuse and Integration (IEEE IRI 2014).

[10]  Mei-Ling Shyu,et al.  Multimodal Information Integration and Fusion for Histology Image Classification , 2011, Int. J. Multim. Data Eng. Manag..

[11]  Mohamed A. Deriche,et al.  A New Technique for Combining Multiple Classifiers using The Dempster-Shafer Theory of Evidence , 2002, J. Artif. Intell. Res..

[12]  Min Chen,et al.  Video Semantic Event/Concept Detection Using a Subspace-Based Multimedia Data Mining Framework , 2008, IEEE Transactions on Multimedia.

[13]  Min Chen,et al.  Spatio-Temporal Analysis for Human Action Detection and Recognition in Uncontrolled Environments , 2015, Int. J. Multim. Data Eng. Manag..

[14]  Jiri Matas,et al.  On Combining Classifiers , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Hayder Radha,et al.  An Information-Theoretic Combining Method for Multi-Classifier Anomaly Detection Systems , 2010, 2010 IEEE International Conference on Communications.

[16]  Yang Yang,et al.  Learning semantic visual vocabularies using diffusion distance , 2009, CVPR.

[17]  Shu-Ching Chen,et al.  Feature Selection Using Correlation and Reliability Based Scoring Metric for Video Semantic Detection , 2010, 2010 IEEE Fourth International Conference on Semantic Computing.

[18]  Min Chen,et al.  Image database retrieval utilizing affinity relationships , 2003, MMDB '03.

[19]  Jiebo Luo,et al.  Recognizing realistic actions from videos , 2009, CVPR.

[20]  Mei-Ling Shyu,et al.  Negative Correlation Discovery for Big Multimedia Data Semantic Concept Mining and Retrieval , 2016, 2016 IEEE Tenth International Conference on Semantic Computing (ICSC).

[21]  Min Chen,et al.  A unified framework for image database clustering and content-based retrieval , 2004, MMDB '04.

[22]  Mei-Ling Shyu,et al.  Enhancing Rare Class Mining in Multimedia Big Data by Concept Correlation , 2016, 2016 IEEE International Symposium on Multimedia (ISM).

[23]  Chengcui Zhang,et al.  An intelligent framework for spatio-temporal vehicle tracking , 2001, ITSC 2001. 2001 IEEE Intelligent Transportation Systems. Proceedings (Cat. No.01TH8585).

[24]  Chengcui Zhang,et al.  A Dynamic User Concept Pattern Learning Framework for Content-Based Image Retrieval , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[25]  Ludmila I. Kuncheva,et al.  Combining Pattern Classifiers: Methods and Algorithms , 2004 .

[26]  Fang Chen,et al.  Investigating speech features and automatic measurement of cognitive load , 2008, 2008 IEEE 10th Workshop on Multimedia Signal Processing.

[27]  Jun-Wei Hsieh,et al.  PLSA-Based Sparse Representation for Object Classification , 2014, 2014 22nd International Conference on Pattern Recognition.

[28]  Shu-Ching Chen,et al.  A Classifier Ensemble Framework for Multimedia Big Data Classification , 2016, 2016 IEEE 17th International Conference on Information Reuse and Integration (IRI).

[29]  Min Chen,et al.  Deep Learning for Imbalanced Multimedia Data Classification , 2015, 2015 IEEE International Symposium on Multimedia (ISM).

[30]  Guillermo Sapiro,et al.  Online Learning for Matrix Factorization and Sparse Coding , 2009, J. Mach. Learn. Res..

[31]  Choochart Haruechaiyasak,et al.  Collaborative Filtering by Mining Association Rules from User Access Sequences , 2005, International Workshop on Challenges in Web Information Retrieval and Integration.

[32]  Mei-Ling Shyu,et al.  Leveraging Concept Association Network for Multimedia Rare Concept Mining and Retrieval , 2012, 2012 IEEE International Conference on Multimedia and Expo.

[33]  Xin Huang,et al.  User Concept Pattern Discovery Using Relevance Feedback And Multiple Instance Learning For Content-Based Image Retrieval , 2002, MDM/KDD.

[34]  Mei-Ling Shyu,et al.  Sparse Linear Integration of Content and Context Modalities for Semantic Concept Retrieval , 2015, IEEE Transactions on Emerging Topics in Computing.

[35]  Marcelo Bernardes Vieira,et al.  Combining gradient histograms using orientation tensors for human action recognition , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[36]  Ivan Laptev,et al.  On Space-Time Interest Points , 2005, International Journal of Computer Vision.

[37]  Mei-Ling Shyu,et al.  Effective Feature Space Reduction with Imbalanced Data for Semantic Concept Detection , 2008, 2008 IEEE International Conference on Sensor Networks, Ubiquitous, and Trustworthy Computing (sutc 2008).

[38]  Chengcui Zhang,et al.  Innovative Shot Boundary Detection for Video Indexing , 2005 .

[39]  Serge J. Belongie,et al.  Behavior recognition via sparse spatio-temporal features , 2005, 2005 IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance.

[40]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[41]  Rangasami L. Kashyap,et al.  Generalized Affinity-Based Association Rule Mining for Multimedia Database Queries , 2001, Knowledge and Information Systems.

[42]  Mei-Ling Shyu,et al.  Weighted Association Rule Mining for Video Semantic Detection , 2010, Int. J. Multim. Data Eng. Manag..

[43]  Sheng Guan,et al.  Domain Knowledge Assisted Data Processing for Florida Public Hurricane Loss Model (Invited Paper) , 2016, 2016 IEEE 17th International Conference on Information Reuse and Integration (IRI).

[44]  Rangasami L. Kashyap,et al.  Identifying Overlapped Objects for Video Indexing and Modeling in Multimedia Database Systems , 2001, Int. J. Artif. Intell. Tools.

[45]  Shu-Ching Chen,et al.  Video Semantic Concept Discovery using Multimodal-Based Association Classification , 2007, 2007 IEEE International Conference on Multimedia and Expo.

[46]  Juan Liu,et al.  Will scene information help realistic action recognition? , 2012, Proceedings of the 10th World Congress on Intelligent Control and Automation.

[47]  Robert P. W. Duin,et al.  The combining classifier: to train or not to train? , 2002, Object recognition supported by user interaction for service robots.

[48]  Mei-Ling Shyu,et al.  Supporting Semantic Concept Retrieval with Negative Correlations in a Multimedia Big Data Mining System , 2016, Int. J. Semantic Comput..

[49]  Yan Meng,et al.  Human activity recognition in video using a hierarchical probabilistic latent model , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[50]  Barbara Caputo,et al.  Recognizing human actions: a local SVM approach , 2004, ICPR 2004.

[51]  Robert P. W. Duin,et al.  Experiments with Classifier Combining Rules , 2000, Multiple Classifier Systems.

[52]  Emmanuel Dellandréa,et al.  Multimodal recognition of visual concepts using histograms of textual concepts and selective weighted late fusion scheme , 2013, Comput. Vis. Image Underst..