Semantics and feature discovery via confidence-based ensemble

Providing accurate and scalable solutions to map low-level perceptual features to high-level semantics is essential for multimedia information organization and retrieval. In this paper, we propose a confidence-based dynamic ensemble (CDE) to overcome the shortcomings of the traditional static classifiers. In contrast to the traditional models, CDE can make dynamic adjustments to accommodate new semantics, to assist the discovery of useful low-level features, and to improve class-prediction accuracy. We depict two key components of CDE: a multi-level function that asserts class-prediction confidence, and the dynamic ensemble method based upon the confidence function. Through theoretical analysis and empirical study, we demonstrate that CDE is effective in annotating large-scale, real-world image datasets.

[1]  Mary Czerwinski,et al.  Semi-Automatic Image Annotation , 2001, INTERACT.

[2]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[3]  Shih-Fu Chang,et al.  Semantic knowledge construction from annotated image collections , 2002, Proceedings. IEEE International Conference on Multimedia and Expo.

[4]  V. Vapnik Estimation of Dependences Based on Empirical Data , 2006 .

[5]  Edward Y. Chang,et al.  Support vector machine active learning for image retrieval , 2001, MULTIMEDIA '01.

[6]  Sargur N. Srihari,et al.  Decision Combination in Multiple Classifier Systems , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Beng Chin Ooi,et al.  Giving meanings to WWW images , 2000, MM 2000.

[8]  Vladimir Vapnik Estimations of dependences based on statistical data , 1982 .

[9]  Robert Tibshirani,et al.  Classification by Pairwise Coupling , 1997, NIPS.

[10]  James Ze Wang,et al.  SIMPLIcity: Semantics-Sensitive Integrated Matching for Picture LIbraries , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Wei-Ying Ma,et al.  Learning and inferring a semantic space from user's relevance feedback for image retrieval , 2002, MULTIMEDIA '02.

[12]  Beng Chin Ooi,et al.  Giving meanings to WWW images , 2000, ACM Multimedia.

[13]  Samy Bengio,et al.  SVMTorch: Support Vector Machines for Large-Scale Regression Problems , 2001, J. Mach. Learn. Res..

[14]  Rohini K. Srihari,et al.  Intelligent Indexing and Semantic Retrieval of Multimodal Documents , 2004, Information Retrieval.

[15]  Edward Y. Chang,et al.  SVM binary classifier ensembles for image classification , 2001, CIKM '01.

[16]  Nello Cristianini,et al.  Large Margin DAGs for Multiclass Classification , 1999, NIPS.

[17]  Wei-Ying Ma,et al.  Improving Image Retrieval with Semantic Classification Using Relevance Feedback , 2002, VDB.

[18]  Newton Lee,et al.  ACM Transactions on Multimedia Computing, Communications and Applications (ACM TOMCCAP) , 2007, CIE.

[19]  Djamel Bouchaffra,et al.  A Methodology for Mapping Scores to Probabilities , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Edward Y. Chang,et al.  CBSA: content-based soft annotation for multimodal image retrieval using Bayes point machines , 2003, IEEE Trans. Circuits Syst. Video Technol..

[21]  Edward Y. Chang,et al.  Confidence-based dynamic ensemble for image annotation and semantics discovery , 2003, MULTIMEDIA '03.

[22]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[23]  P. Poddar,et al.  Hierarchical ensemble of neural networks , 1993, IEEE International Conference on Neural Networks.

[24]  Shih-Fu Chang,et al.  Semantic visual templates: linking visual features to semantics , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[25]  C. K. Chow,et al.  On optimum recognition error and reject tradeoff , 1970, IEEE Trans. Inf. Theory.

[26]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[27]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[28]  Javier Muguerza,et al.  A two-stage classifier for broken and blurred digits in forms , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[29]  James Ze Wang,et al.  Automatic Linguistic Indexing of Pictures by a Statistical Modeling Approach , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[30]  Edward Y. Chang,et al.  Using one-class and two-class SVMs for multiclass image annotation , 2005, IEEE Transactions on Knowledge and Data Engineering.

[31]  Edward Y. Chang,et al.  Adaptive Feature-Space Conformal Transformation for Imbalanced-Data Learning , 2003, ICML.

[32]  Nikos Karampatziakis,et al.  Probabilistic Outputs for SVMs and Comparisons to Regularized Likelihood Methods , 2007 .

[33]  Yoram Singer,et al.  Improved Boosting Algorithms Using Confidence-rated Predictions , 1998, COLT' 98.

[34]  John R. Smith,et al.  Learning to annotate video databases , 2001, IS&T/SPIE Electronic Imaging.

[35]  Jianping Fan,et al.  Multi-level annotation of natural scenes using dominant image components and semantic concepts , 2004, MULTIMEDIA '04.