Narrowing the semantic gap in natural images

Over two decades of intensive research, researchers employed many different approaches to solve the what so called “semantic gap” problem in image indexing and retrieval. Yet, the gap is still recognized as a barrier to progress, and further work is needed to bridge (or at least) narrow the gap. This suggests that more emphasis should be placed on understanding how humans perceive images. This study measures the effectiveness of two image indexing techniques for estimating similarities between images; the semantic basis functions and the affective basis functions. These indexing techniques aim at providing a measure of similarity between outdoor natural images as humans see it. The results presented in this study suggest that the semantic basis functions outperform the affective basis functions, and are able to index the content of outdoor natural images in a manner that allows retrieval of images that have been judged to be subjectively similar.

[1]  Toshikazu Kato,et al.  Query by Visual Example - Content based Image Retrieval , 1992, EDBT.

[2]  Myron Flickner,et al.  Query by Image and Video Content , 1995 .

[3]  Alex Pentland,et al.  Modal Matching for Correspondence and Recognition , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Vijay V. Raghavan,et al.  Design and evaluation of algorithms for image retrieval by spatial similarity , 1995, TOIS.

[5]  Dragutin Petkovic,et al.  Query by Image and Video Content: The QBIC System , 1995, Computer.

[6]  Shih-Fu Chang,et al.  VisualSEEk: a fully automated content-based image query system , 1997, MULTIMEDIA '96.

[7]  Charles A. Bouman,et al.  Perceptual image similarity experiments , 1998, Electronic Imaging.

[8]  S. Panchanathan,et al.  The use of lexical basis functions to characterize faces, and to measure their perceived similarity , 2002, Proceedings of the 9th International Conference on Neural Information Processing, 2002. ICONIP '02..

[9]  John Tait,et al.  Image classification using hybrid neural networks , 2003, SIGIR.

[10]  Akifumi Makinouchi,et al.  Semantic Approach to Image Database Classification and Retrieval (「夏のデータベースワークショップ(DBWS2003)」一般) , 2003 .

[11]  David A. Forsyth,et al.  Matching Words and Pictures , 2003, J. Mach. Learn. Res..

[12]  Sethuraman Panchanathan,et al.  Characterizing the high-level content of natural images using lexical basis functions , 2003, IS&T/SPIE Electronic Imaging.

[13]  Sethuraman Panchanathan,et al.  Indexing natural images for retrieval based on Kansei factors , 2004, IS&T/SPIE Electronic Imaging.

[14]  Sethuraman Panchanathan,et al.  Using perceptually based face indexing to facilitate human-computer collaborative retrieval , 2005, IS&T/SPIE Electronic Imaging.

[15]  Tom Minka,et al.  Vision texture for annotation , 1995, Multimedia Systems.

[16]  Stéphane Ayache,et al.  Context-Based Conceptual Image Indexing , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[17]  Gustavo Carneiro,et al.  Supervised Learning of Semantic Classes for Image Annotation and Retrieval , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Wei Liu,et al.  Semi-supervised distance metric learning for Collaborative Image Retrieval , 2008, CVPR.

[19]  Dacheng Tao,et al.  Biased Discriminant Euclidean Embedding for Content-Based Image Retrieval , 2010, IEEE Transactions on Image Processing.

[20]  Md. Monirul Islam,et al.  A review on automatic image annotation techniques , 2012, Pattern Recognit..

[21]  Yi Yang,et al.  A Multimedia Retrieval Framework Based on Semi-Supervised Ranking and Relevance Feedback , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Harpreet Kaur,et al.  Survey of Techniques of High Level Semantic Based Image Retrieval , 2013 .

[23]  J. Shanbehzadeh,et al.  Relevance Feedback Optimization in Content Based Image Retrieval Via Enhanced Radial Basis Function Network , 2022 .