Enhancing Multimedia Imbalanced Concept Detection Using VIMP in Random Forests

Recent developments in social media and cloud storage lead to an exponential growth in the amount of multimedia data, which increases the complexity of managing, storing, indexing, and retrieving information from such big data. Many current content-based concept detection approaches lag from successfully bridging the semantic gap. To solve this problem, a multi-stage random forest framework is proposed to generate predictor variables based on multivariate regressions using variable importance (VIMP). By fine tuning the forests and significantly reducing the predictor variables, the concept detection scores are evaluated when the concept of interest is rare and imbalanced, i.e., having little collaboration with other high level concepts. Using classical multivariate statistics, estimating the value of one coordinate using other coordinates standardizes the covariates and it depends upon the variance of the correlations instead of the mean. Thus, conditional dependence on the data being normally distributed is eliminated. Experimental results demonstrate that the proposed framework outperforms those approaches in the comparison in terms of the Mean Average Precision (MAP) values.

[1]  Rangasami L. Kashyap,et al.  Temporal And Spatial Semantic Models For Multimedia Presentations , 1997 .

[2]  Rangasami L. Kashyap,et al.  Augmented transition networks as video browsing models for multimedia databases and multimedia information systems , 1999, Proceedings 11th International Conference on Tools with Artificial Intelligence.

[3]  Rangasami L. Kashyap,et al.  Generalized Affinity-Based Association Rule Mining for Multimedia Database Queries , 2001, Knowledge and Information Systems.

[4]  Choochart Haruechaiyasak,et al.  Mining user access behavior on the WWW , 2001, 2001 IEEE International Conference on Systems, Man and Cybernetics. e-Systems and e-Man for Cybernetics in Cyberspace (Cat.No.01CH37236).

[5]  Rangasami L. Kashyap,et al.  Augmented Transition Network as a Semantic Model for Video Data , 2001 .

[6]  Rangasami L. Kashyap,et al.  Identifying Overlapped Objects for Video Indexing and Modeling in Multimedia Database Systems , 2001, Int. J. Artif. Intell. Tools.

[7]  Chengcui Zhang,et al.  An intelligent framework for spatio-temporal vehicle tracking , 2001, ITSC 2001. 2001 IEEE Intelligent Transportation Systems. Proceedings (Cat. No.01TH8585).

[8]  Rangasami L. Kashyap,et al.  Semantic Models for Multimedia Database Searching and Browsing , 2000, Advances in Database Systems.

[9]  Xiuqi Li,et al.  An effective content-based visual image retrieval system , 2002, Proceedings 26th Annual International Computer Software and Applications.

[10]  Xin Huang,et al.  User Concept Pattern Discovery Using Relevance Feedback And Multiple Instance Learning For Content-Based Image Retrieval , 2002, MDM/KDD.

[11]  Xiuqi Li,et al.  Image Retrieval By Color , Texture , And Spatial Information , 2002 .

[12]  Min Chen,et al.  Image database retrieval utilizing affinity relationships , 2003, MMDB '03.

[13]  Choochart Haruechaiyasak,et al.  Category cluster discovery from distributed WWW directories , 2003, Inf. Sci..

[14]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[15]  Jianping Fan,et al.  Concept-oriented indexing of video databases: toward semantic sensitive retrieval and browsing , 2004, IEEE Transactions on Image Processing.

[16]  Min Chen,et al.  A unified framework for image database clustering and content-based retrieval , 2004, MMDB '04.

[17]  Tat-Seng Chua,et al.  A bootstrapping framework for annotating and retrieving WWW images , 2004, MULTIMEDIA '04.

[18]  Mark R. Segal,et al.  Machine Learning Benchmarks and Random Forest Regression , 2004 .

[19]  Chengcui Zhang,et al.  Innovative Shot Boundary Detection for Video Indexing , 2005 .

[20]  Mei-Ling Shyu,et al.  Handling nominal features in anomaly intrusion detection problems , 2005, 15th International Workshop on Research Issues in Data Engineering: Stream Data Mining and Applications (RIDE-SDMA'05).

[21]  Min Chen,et al.  A latent semantic indexing based method for solving multiple instance learning problem in region-based image retrieval , 2005, Seventh IEEE International Symposium on Multimedia (ISM'05).

[22]  Choochart Haruechaiyasak,et al.  Collaborative Filtering by Mining Association Rules from User Access Sequences , 2005, International Workshop on Challenges in Web Information Retrieval and Integration.

[23]  Paul Over,et al.  Evaluation campaigns and TRECVid , 2006, MIR '06.

[24]  Chengcui Zhang,et al.  A Dynamic User Concept Pattern Learning Framework for Content-Based Image Retrieval , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[25]  Frédéric Jurie,et al.  Fast Discriminative Visual Codebooks using Randomized Clustering Forests , 2006, NIPS.

[26]  Cordelia Schmid,et al.  Local Features and Kernels for Classification of Texture and Object Categories: A Comprehensive Study , 2006, 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'06).

[27]  Shu-Ching Chen,et al.  Network intrusion detection through Adaptive Sub-Eigenspace Modeling in multiagent systems , 2007, ACM Trans. Auton. Adapt. Syst..

[28]  Mubarak Shah,et al.  Improving Semantic Concept Detection and Retrieval using Contextual Estimates , 2007, 2007 IEEE International Conference on Multimedia and Expo.

[29]  Andrea Vedaldi,et al.  Objects in Context , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[30]  Cordelia Schmid,et al.  Learning Object Representations for Visual Object Class Recognition , 2007, ICCV 2007.

[31]  Shu-Ching Chen,et al.  Video Semantic Concept Discovery using Multimodal-Based Association Classification , 2007, 2007 IEEE International Conference on Multimedia and Expo.

[32]  Min Chen,et al.  Video Semantic Event/Concept Detection Using a Subspace-Based Multimedia Data Mining Framework , 2008, IEEE Transactions on Multimedia.

[33]  Achim Zeileis,et al.  BMC Bioinformatics BioMed Central Methodology article Conditional variable importance for random forests , 2008 .

[34]  Koen E. A. van de Sande,et al.  A comparison of color features for visual concept classification , 2008, CIVR '08.

[35]  Mei-Ling Shyu,et al.  Effective Feature Space Reduction with Imbalanced Data for Semantic Concept Detection , 2008, 2008 IEEE International Conference on Sensor Networks, Ubiquitous, and Trustworthy Computing (sutc 2008).

[36]  Michael Isard,et al.  Lost in quantization: Improving particular object retrieval in large scale image databases , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[37]  Roberto Cipolla,et al.  Semantic texton forests for image categorization and segmentation , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[38]  Shu-Ching Chen,et al.  Video semantic concept detection via associative classification , 2009, 2009 IEEE International Conference on Multimedia and Expo.

[39]  Chong-Wah Ngo,et al.  Domain adaptive semantic diffusion for large scale context-based video annotation , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[40]  Stuart Harvey Rubin,et al.  A Human-Centered Multiple Instance Learning Framework for Semantic Video Retrieval , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[41]  Arnold W. M. Smeulders,et al.  Real-Time Visual Concept Classification , 2010, IEEE Transactions on Multimedia.

[42]  Shu-Ching Chen,et al.  Feature Selection Using Correlation and Reliability Based Scoring Metric for Video Semantic Detection , 2010, 2010 IEEE Fourth International Conference on Semantic Computing.

[43]  Alberto Del Bimbo,et al.  Video Annotation and Retrieval Using Ontologies and Rule Learning , 2010, IEEE MultiMedia.

[44]  Cor J. Veenman,et al.  Visual Word Ambiguity , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  Mei-Ling Shyu,et al.  Weighted Association Rule Mining for Video Semantic Detection , 2010, Int. J. Multim. Data Eng. Manag..

[46]  Xi Chen,et al.  Random survival forests for high‐dimensional data , 2011, Stat. Anal. Data Min..

[47]  Mei-Ling Shyu,et al.  Utilizing Context Information to Enhance Content-Based Image Classification , 2011, Int. J. Multim. Data Eng. Manag..

[48]  Benoit Huet,et al.  An ontology-based evidential framework for video indexing using high-level multimodal fusion , 2011, Multimedia Tools and Applications.

[49]  Adel M. Alimi,et al.  A fuzzy ontology: based framework for reasoning in visual video content analysis and indexing , 2011, MDMKDD '11.

[50]  Songyang Lao,et al.  Video semantic concept detection using ontology , 2011, ICIMCS '11.

[51]  Chao Chen,et al.  Weighted Subspace Filtering and Ranking Algorithms for Video Concept Retrieval , 2011, IEEE MultiMedia.

[52]  W. Marsden.  I and J , 2012 .

[53]  Gang Hua,et al.  Semantic Model Vectors for Complex Video Event Recognition , 2012, IEEE Transactions on Multimedia.

[54]  Antonio Torralba,et al.  A Tree-Based Context Model for Object Recognition , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[55]  Jun-Wei Hsieh,et al.  Vehicle make and model recognition using sparse representation and symmetrical SURFs , 2013, 16th International IEEE Conference on Intelligent Transportation Systems (ITSC 2013).

[56]  Min Chen,et al.  Utilizing concept correlations for effective imbalanced data classification , 2014, Proceedings of the 2014 IEEE 15th International Conference on Information Reuse and Integration (IEEE IRI 2014).

[57]  Min Chen,et al.  Deep Learning for Imbalanced Multimedia Data Classification , 2015, 2015 IEEE International Symposium on Multimedia (ISM).

[58]  Mei-Ling Shyu,et al.  Sparse Linear Integration of Content and Context Modalities for Semantic Concept Retrieval , 2015, IEEE Transactions on Emerging Topics in Computing.

[59]  Min Chen,et al.  Spatio-Temporal Analysis for Human Action Detection and Recognition in Uncontrolled Environments , 2015, Int. J. Multim. Data Eng. Manag..