Enhancement of Textual Images Classification using their Global and Local Visual Contents

This paper deals with the existence of a dependance between the textual indexation of an image (a set of keywords) and its visual indexation (color and shape attributes). This experience has been realized on a corpus of news photos manually indexed by keywords extracted from a hierarchically structured thesaurus. First, a reference classification of these photos has been constructed from their textual indexation (regarded as relevant), then textual and visual features characterizing these classes have been constructed. Finally, they have been used to evaluate performances of a content-based image retrieval combining textual and visual descriptions. Results of the visuo-textual classification show an improvement of 54% against classification using only textual information.

[1]  C.-C. Jay Kuo,et al.  Introduction to Content‐Based Image Retrieval—Overview of Key Techniques , 2002 .

[2]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Michael E. Lesk,et al.  Computer Evaluation of Indexing and Text Processing , 1968, JACM.

[4]  Thomas S. Huang,et al.  Unifying Keywords and Visual Contents in Image Retrieval , 2002, IEEE Multim..

[5]  Editors , 2003 .

[6]  Jacques Le Maitre,et al.  Indexation et interrogation de photos de presse décrites en MPEG-7 et stockées dans une base de données XML , 2002, Ingénierie des Systèmes d Inf..

[7]  Hervé Glotin.  Elaboration et comparaison de systèmes adaptatifs multi-flux de reconnaissance robuste de la parole : incorporation des indices de voisement et de localisation , 2001 .

[8]  B. S. Manjunath,et al.  Introduction to mpeg-7 , 2002 .

[9]  Gerard Salton,et al.  The SMART Retrieval System—Experiments in Automatic Document Processing , 1971 .

[10]  G. N. Lance,et al.  A General Theory of Classificatory Sorting Strategies: 1. Hierarchical Systems , 1967, Comput. J..

[11]  Christos Faloutsos,et al.  QBIC project: querying images by content, using color, texture, and shape , 1993, Electronic Imaging.

[12]  Philippe Mulhem,et al.  Un modèle vectoriel étendu de recherche d'information adapté aux images , 2002, INFORSID.

[13]  G. N. Lance,et al.  A general theory of classificatory sorting strategies: II. Clustering systems , 1967, Comput. J..

[14]  S. Sclaroff,et al.  Combining textual and visual cues for content-based image retrieval on the World Wide Web , 1998, Proceedings. IEEE Workshop on Content-Based Access of Image and Video Libraries (Cat. No.98EX173).