Modeling, Classifying and Annotating Weakly Annotated Images Using Bayesian Network

We propose a probabilistic graphical model to represent weakly annotated images. This model is used to classify images and automatically extend existing annotations to new images by taking into account semantic relations between keywords. The proposed method has been evaluated in classification and automatic annotation of images. The experimental results, obtained from a database of more than 30000 images, by combining visual and textual information, show an improvement by 50.5% in terms of recognition rate against only visual information classication. Taking into account semantic relations between keywords improves the recognition rate by 10.5% and the mean rate of good annotations by 6.9%. The proposed method is experimentally competitive with the state-of-art classifiers.

[1]  R. Manmatha,et al.  Multiple Bernoulli relevance models for image and video annotation , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[2]  Wei-Ying Ma,et al.  A Probabilistic Semantic Model for Image Annotation and Multi-Modal Image Retrieva , 2005, ICCV.

[3]  Feller William,et al.  An Introduction To Probability Theory And Its Applications , 1950 .

[4]  Latifur Khan,et al.  Image annotations by combining multiple evidence & wordNet , 2005, ACM Multimedia.

[5]  Hironobu Takahashi,et al.  Automatic word assignment to images based on image division and vector quantization , 2000 .

[6]  Changhu Wang,et al.  Image annotation refinement using random walk with restarts , 2006, MM '06.

[7]  Shih-Fu Chang,et al.  Perceptual knowledge construction from annotated image collections , 2002, Proceedings. IEEE International Conference on Multimedia and Expo.

[8]  Gustavo Carneiro,et al.  Supervised Learning of Semantic Classes for Image Annotation and Retrieval , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Jin-Woo Jeong,et al.  Automatic Extraction of Semantic Relationships from Images Using Ontologies and SVM Classifiers , 2007, MCAM.

[10]  Chuan-Yu Chang,et al.  Semantic analysis of real-world images using support vector machine , 2009, Expert Syst. Appl..

[11]  Hervé Glotin,et al.  A Comparative Study of Diversity Methods for Hybrid Text and Image Retrieval Approaches , 2008, CLEF.

[12]  R. Manmatha,et al.  An Inference Network Approach to Image Retrieval , 2004, CIVR.

[13]  Clement T. Yu,et al.  An effective approach to document retrieval via utilizing WordNet and recognizing phrases , 2004, SIGIR '04.

[14]  Bipin C. Desai,et al.  A unified image retrieval framework on local visual and semantic concept-based feature spaces , 2009, J. Vis. Commun. Image Represent..

[15]  Jing Hua,et al.  Region-based Image Annotation using Asymmetrical Support Vector Machine-based Multiple-Instance Learning , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[16]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[17]  Hironobu Takahashi,et al.  Automatic word assignment to images based on dividing image and vector quantization , 2000, RIAO Conference.

[18]  L. Dworsky.  An Introduction to Probability , 2008 .

[19]  Michael J. Swain,et al.  Color indexing , 1991, International Journal of Computer Vision.

[20]  David A. Forsyth,et al.  Object Recognition as Machine Translation: Learning a Lexicon for a Fixed Image Vocabulary , 2002, ECCV.

[21]  Michael I. Jordan,et al.  Modeling annotated data , 2003, SIGIR.

[22]  Edward Y. Chang,et al.  Using one-class and two-class SVMs for multiclass image annotation , 2005, IEEE Transactions on Knowledge and Data Engineering.

[23]  Rik Van de Walle,et al.  Personal content management system: A semantic approach , 2009, J. Vis. Commun. Image Represent..

[24]  David A. Forsyth,et al.  Matching Words and Pictures , 2003, J. Mach. Learn. Res..

[25]  Jianping Fan,et al.  Automatic image annotation by incorporating feature hierarchy and boosting to scale up SVM classifiers , 2006, MM '06.

[26]  Wei-Ying Ma,et al.  Bipartite graph reinforcement model for web image annotation , 2007, ACM Multimedia.

[27]  Christiane Fellbaum,et al.  Book Reviews: WordNet: An Electronic Lexical Database , 1999, CL.

[28]  Antonio Torralba,et al.  Statistics of natural image categories , 2003, Network.

[29]  R. Manmatha,et al.  A Model for Learning the Semantics of Pictures , 2003, NIPS.

[30]  Luo Si,et al.  Effective automatic image annotation via a coherent language model and active learning , 2004, MULTIMEDIA '04.

[31]  William Feller,et al.  An Introduction to Probability Theory and Its Applications , 1967 .

[32]  A. Murat Tekalp,et al.  Region-Based Shape Matching for Automatic Image Annotation and Query-by-Example , 1997, J. Vis. Commun. Image Represent..

[33]  Laurent Wendling,et al.  Technical symbols recognition using the two-dimensional Radon transform , 2002, Object recognition supported by user interaction for service robots.

[34]  Clement T. Yu,et al.  Using semantic contents and WordNet in image retrieval , 1997, SIGIR '97.

[35]  Stefan M. Rüger,et al.  Information-theoretic semantic multimedia indexing , 2007, CIVR '07.

[36]  R. Manmatha,et al.  Automatic image annotation and retrieval using cross-media relevance models , 2003, SIGIR.

[37]  William I. Grosky,et al.  Negotiating the semantic gap: from feature maps to semantic landscapes , 2001, Pattern Recognit..

[38]  Nizar Bouguila,et al.  Unsupervised learning of a finite discrete mixture: Applications to texture modeling and image databases summarization , 2007, J. Vis. Commun. Image Represent..

[39]  Charles M. Grinstead,et al.  Introduction to probability , 1999, Statistics for the Behavioural Sciences.

[40]  Shih-Fu Chang,et al.  Image Retrieval: Current Techniques, Promising Directions, and Open Issues , 1999, J. Vis. Commun. Image Represent..