A novel framework for semantic image classification and benchmark

Interpreting semantic image concepts via their dominant compounds is a promising approach to achieve effective image retrieval via key words. In this chapter, a novel framework is proposed by using the salient objects as the semantic building blocks for image concept interpretation. This novel framework includes (a) using Support Vector Machine to achieve automatic detection of the salient objects as our basic visual vocabulary; (b) using Gaussian Mixture Model for semantic image concept interpretation by exploring the quantitative relationship between the semantic image concepts and their dominant compounds, i.e., salient objects. Our broad experiments on natural images have obtained significant improvements on semantic image classification.

[1]  Jianping Fan,et al.  Concept-oriented indexing of video databases: toward semantic sensitive retrieval and browsing , 2004, IEEE Transactions on Image Processing.

[2]  Thomas S. Huang,et al.  Unifying Keywords and Visual Contents in Image Retrieval , 2002, IEEE Multim..

[3]  Michael I. Jordan,et al.  Modeling annotated data , 2003, SIGIR.

[4]  Edward Y. Chang,et al.  Support vector machine active learning for image retrieval , 2001, MULTIMEDIA '01.

[5]  John R. Smith,et al.  Image Classification and Querying Using Composite Region Templates , 1999, Comput. Vis. Image Underst..

[6]  Jianying Hu,et al.  Matching and retrieval based on the vocabulary and grammar of color patterns , 2000, IEEE Trans. Image Process..

[7]  Jitendra Malik,et al.  Blobworld: Image Segmentation Using Expectation-Maximization and Its Application to Image Querying , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Pietro Perona,et al.  Towards automatic discovery of object categories , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[9]  Jiebo Luo,et al.  A physical model-based approach to detecting sky in photographic images , 2002, IEEE Trans. Image Process..

[10]  Qi Tian,et al.  Discriminant-EM algorithm with application to image retrieval , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[11]  Marcel Worring,et al.  Content-Based Image Retrieval at the End of the Early Years , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  David A. Forsyth,et al.  Matching Words and Pictures , 2003, J. Mach. Learn. Res..

[13]  Shih-Fu Chang,et al.  Semantic knowledge construction from annotated image collections , 2002, Proceedings. IEEE International Conference on Multimedia and Expo.

[14]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Antonio Torralba,et al.  Semantic organization of scenes using discriminant structural templates , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[16]  Thorsten Joachims,et al.  Transductive Inference for Text Classification using Support Vector Machines , 1999, ICML.

[17]  Anil K. Jain,et al.  Image classification for content-based indexing , 2001, IEEE Trans. Image Process..

[18]  Aidong Zhang,et al.  Semantics-Based Image Retrieval by Region Saliency , 2002, CIVR.

[19]  W. Eric L. Grimson,et al.  Configuration based scene classification and image indexing , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[20]  Edward Y. Chang,et al.  CBSA: content-based soft annotation for multimodal image retrieval using Bayes point machines , 2003, IEEE Trans. Circuits Syst. Video Technol..

[21]  G. McLachlan,et al.  The EM algorithm and extensions , 1996 .

[22]  Stan Z. Li,et al.  View-based clustering of object appearances based on independent subspace analysis , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[23]  Shih-Fu Chang,et al.  Semantic visual templates: linking visual features to semantics , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[24]  James Ze Wang,et al.  SIMPLIcity: Semantics-Sensitive Integrated Matching for Picture LIbraries , 2001, IEEE Trans. Pattern Anal. Mach. Intell..