Salient Objects: Semantic Building Blocks for Image Concept Interpretation

Interpreting semantic image concepts via their dominant compounds is a promising approach to achieve effective image retrieval via keywords. In this paper, a novel framework is proposed by using the salient objects as the semantic building blocks for image concept interpretation. This novel framework includes: (a) using machine learning technique to achieve automatic detection of the salient objects; (b) using Gaussian mixture model for semantic image concept interpretation by exploring the quantitative relationship between the semantic image concepts and their dominant compounds, i.e., salient objects. Our broad experiments on natural images have obtained significant improvements on semantic image classification.

[1]  Stan Z. Li,et al.  View-based clustering of object appearances based on independent subspace analysis , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[2]  Antonio Torralba,et al.  Semantic organization of scenes using discriminant structural templates , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[3]  Shih-Fu Chang,et al.  Semantic knowledge construction from annotated image collections , 2002, Proceedings. IEEE International Conference on Multimedia and Expo.

[4]  James Ze Wang,et al.  SIMPLIcity: Semantics-Sensitive Integrated Matching for Picture LIbraries , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Edward Y. Chang,et al.  CBSA: content-based soft annotation for multimodal image retrieval using Bayes point machines , 2003, IEEE Trans. Circuits Syst. Video Technol..

[6]  David A. Forsyth,et al.  Matching Words and Pictures , 2003, J. Mach. Learn. Res..

[7]  Aidong Zhang,et al.  Semantics-Based Image Retrieval by Region Saliency , 2002, CIVR.

[8]  Anil K. Jain,et al.  Image classification for content-based indexing , 2001, IEEE Trans. Image Process..

[9]  Pietro Perona,et al.  Towards automatic discovery of object categories , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[10]  Michael I. Jordan,et al.  Modeling annotated data , 2003, SIGIR.

[11]  Jitendra Malik,et al.  Blobworld: Image Segmentation Using Expectation-Maximization and Its Application to Image Querying , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Shih-Fu Chang,et al.  Semantic visual templates: linking visual features to semantics , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[13]  Thomas S. Huang,et al.  Unifying Keywords and Visual Contents in Image Retrieval , 2002, IEEE Multim..

[14]  Marcel Worring,et al.  Content-Based Image Retrieval at the End of the Early Years , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Jianying Hu,et al.  Matching and retrieval based on the vocabulary and grammar of color patterns , 2000, IEEE Trans. Image Process..

[16]  Jianping Fan,et al.  Concept-oriented indexing of video databases: toward semantic sensitive retrieval and browsing , 2004, IEEE Transactions on Image Processing.

[17]  John R. Smith,et al.  Image Classification and Querying Using Composite Region Templates , 1999, Comput. Vis. Image Underst..

[18]  W. Eric L. Grimson,et al.  Configuration based scene classification and image indexing , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[19]  Wei-Ying Ma,et al.  Image and Video Retrieval , 2003, Lecture Notes in Computer Science.

[20]  Jiebo Luo,et al.  A physical model-based approach to detecting sky in photographic images , 2002, IEEE Trans. Image Process..

[21]  Qi Tian,et al.  Discriminant-EM algorithm with application to image retrieval , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).