Towards intelligent image retrieval

Research into techniques for the retrieval of images by semantic content is still in its infancy. This paper reviews recent trends in the field, distinguishing four separate lines of activity: automatic scene analysis, model-based and statistical approaches to object classification, and adaptive learning from user feedback. It compares the strengths and weaknesses of model-based and adaptive techniques, and argues that further advances in the field are likely to involve the increasing use of techniques from the field of artificial intelligence.

[1]  Luigi Cinque,et al.  Indexing pictorial documents by their content: a survey of current techniques , 1997, Image Vis. Comput..

[2]  Thierry Pun,et al.  A Comparison of Human and Machine Assessments of Image Similarity for the Organization of Image Databases , 1997 .

[3]  Rohini K. Srihari.  Automatic indexing and content-based retrieval of captioned photographs , 1995, Proceedings of 3rd International Conference on Document Analysis and Recognition.

[4]  Bernt Schiele,et al.  The Robustness of Object Recognition to View Point Changes Using Multidimensional Receptive Field Histograms , 1996 .

[5]  Tom M. Mitchell,et al.  Explanation-Based Generalization: A Unifying View , 1986, Machine Learning.

[6]  J. Zhang,et al.  Image retrieval for information systems , 1995, Electronic Imaging.

[7]  Bruce A. Draper,et al.  The schema system , 1988, International Journal of Computer Vision.

[8]  Neill W. Campbell,et al.  Iterative refinement by relevance feedback in content-based digital image retrieval , 1998, MULTIMEDIA '98.

[9]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, CACM.

[10]  Martin Szummer,et al.  Indoor-outdoor image classification , 1998, Proceedings 1998 IEEE International Workshop on Content-Based Access of Image and Video Database.

[11]  Aude Oliva,et al.  Global semantic classification of scenes using power spectrum templates , 1999 .

[12]  Bernt Schiele,et al.  The Concept of Visual Classes for Object Classification , 1997 .

[13]  Peter G. B. Enser,et al.  Analysis of user need in image archives , 1997, J. Inf. Sci..

[14]  Hayit Greenspan,et al.  Finding Pictures of Objects in Large Collections of Images , 1996, Object Representation in Computer Vision.

[15]  Shih-Fu Chang,et al.  Model-based classification of visual information for content-based retrieval , 1998, Electronic Imaging.

[16]  Simone Santini,et al.  Do images mean anything? , 1997, Proceedings of International Conference on Image Processing.

[17]  Toshikazu Kato,et al.  Database architecture for content-based image retrieval , 1992, Electronic Imaging.

[18]  Max Wertheimer,et al.  Untersuchungen zur Lehre von der Gestalt , .

[19]  W. Eric L. Grimson,et al.  Configuration based scene classification and image indexing , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[20]  Janet L. Kolodner,et al.  Case-Based Reasoning , 1988, IJCAI 1989.

[21]  Anil K. Jain,et al.  On image classification: city images vs. landscapes , 1998, Pattern Recognit..

[22]  B. S. Manjunath,et al.  Texture Features for Browsing and Retrieval of Image Data , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Takeo Kanade,et al.  Neural Network-Based Face Detection , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Eric J. Pauwels,et al.  Automatic Interpretation Based on Robust Segmentation and Shape-Extraction , 1999, VISUAL.

[25]  Gerard Salton,et al.  The SMART Retrieval System—Experiments in Automatic Document Processing , 1971 .

[26]  Shih-Fu Chang,et al.  Semantic visual templates: linking visual features to semantics , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[27]  Michael J. Swain,et al.  Color indexing , 1991, International Journal of Computer Vision.

[28]  Neill W. Campbell,et al.  Interpreting image databases by region classification , 1997, Pattern Recognit..

[29]  Rodney A. Brooks,et al.  Model-Based Three-Dimensional Interpretations of Two-Dimensional Images , 1981, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Chahab Nastar,et al.  Relevance feedback in Surfimage , 1998, Proceedings Fourth IEEE Workshop on Applications of Computer Vision. WACV'98 (Cat. No.98EX201).

[31]  Takashi Matsuyama,et al.  SIGMA: A Knowledge-Based Aerial Image Understanding System , 1990 .

[32]  Shih-Fu Chang,et al.  Integration of Visual and Text-Based Approaches for the Content Labeling and Classification of Photographs , 1999, SIGIR 1999.

[33]  Alex Pentland,et al.  Photobook: tools for content-based manipulation of image databases , 1994, Other Conferences.

[34]  Jitendra Malik,et al.  Recognizing surfaces using three-dimensional textons , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[35]  Hugh C. Davis,et al.  Towards Multimedia Thesaurus Support for Media-based Navigation , 1998, Image Databases and Multi-Media Search.

[36]  Anil K. Jain,et al.  Incremental learning for Bayesian classification of images , 1999, Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348).

[37]  Tom Michael Mitchell,et al.  Explanation-based generalization: A unifying view , 1986 .

[38]  Michael S. Lew,et al.  Visual Learning of Simple Semantics in ImageScape , 1999, VISUAL.

[39]  John P. Oakley,et al.  Storage and Retrieval for Image and Video Databases , 1993 .

[40]  Vijay V. Raghavan,et al.  Design and evaluation of algorithms for image retrieval by spatial similarity , 1995, TOIS.

[41]  R. Manmatha,et al.  Retrieving images by appearance , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[42]  Christos Faloutsos,et al.  QBIC project: querying images by content, using color, texture, and shape , 1993, Electronic Imaging.

[43]  Brian Scassellati,et al.  Alternative Essences of Intelligence , 1998, AAAI/IAAI.

[44]  Allen Newell,et al.  Computer science as empirical inquiry: symbols and search , 1976, CACM.

[45]  Wayne D. Gray,et al.  Basic objects in natural categories , 1976, Cognitive Psychology.

[46]  Rogers P. Hall,et al.  Computational Approaches to Analogical Reasoning: A Comparative Analysis , 1989, Artif. Intell..

[47]  Anil K. Jain,et al.  Image retrieval using color and shape , 1996, Pattern Recognit..

[48]  J. Eakins.  Techniques for image retrieval , 1998 .

[49]  Anil K. Jain,et al.  Detecting sky and vegetation in outdoor images , 1999, Electronic Imaging.

[50]  W PicardRosalind,et al.  Periodicity, Directionality, and Randomness , 1996 .

[51]  Takeo Kanade,et al.  Probabilistic modeling of local appearance and spatial relationships for object recognition , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[52]  Rajiv Mehrotra,et al.  Similar-Shape Retrieval in Shape Data Management , 1995, Computer.

[53]  M. Wertheimer.  Untersuchungen zur Lehre von der Gestalt. II , 1923 .

[54]  Thomas P. Minka,et al.  An image database browser that learns from user interaction , 1996 .

[55]  Peter Stanchev,et al.  Content-Based Image Retrieval Systems , 2001 .

[56]  Shih-Fu Chang,et al.  Image Retrieval: Current Techniques, Promising Directions, and Open Issues , 1999, J. Vis. Commun. Image Represent..

[57]  Thomas S. Huang,et al.  Relevance feedback techniques in interactive content-based image retrieval , 1997, Electronic Imaging.

[58]  M MitchellTom,et al.  Explanation-Based Generalization , 1986 .

[59]  Jitendra Malik,et al.  Color- and texture-based image segmentation using EM and its application to content-based image retrieval , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[60]  Sethuraman Panchanathan,et al.  Review of Image and Video Indexing Techniques , 1997, J. Vis. Commun. Image Represent..

[61]  Ramesh Jain,et al.  Storage and Retrieval for Image and Video Databases III , 1995 .

[62]  Jitendra Malik,et al.  Learning Appearance Based Models: Mixtures of Second Moment Experts , 1996, NIPS.

[63]  Fang Liu,et al.  Periodicity, Directionality, and Randomness: Wold Features for Image Modeling and Retrieval , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[64]  Aleix M. Martínez,et al.  Semantic access to a database of images: an approach to object-related image retrieval , 1999, Proceedings IEEE International Conference on Multimedia Computing and Systems.

[65]  Cedric Thienot,et al.  Extraction of composite visual objects from audiovisual materials , 1999, Optics East.

[66]  Bernt Schiele,et al.  Object Recognition Using Multidimensional Receptive Field Histograms , 1996, ECCV.

[67]  Janet L. Kolodner,et al.  Case-Based Reasoning , 1989, IJCAI 1989.

[68]  C.-C. Jay Kuo,et al.  Implementation and performance evaluation of a progressive image retrieval system , 1997, Electronic Imaging.

[69]  Ryszard S. Michalski,et al.  A theory and methodology of inductive learning , 1993 .

[70]  Arnold W. M. Smeulders,et al.  Image Databases and Multi-Media Search , 1998, Image Databases and Multi-Media Search.

[71]  Shih-Fu Chang,et al.  Querying by color regions using VisualSEEk content-based visual query system , 1997 .

[72]  Hans-Peter Kriegel,et al.  The R*-tree: an efficient and robust access method for points and rectangles , 1990, SIGMOD '90.

[73]  Wei-Ying Ma,et al.  Information embedding based on user's relevance feedback for image retrieval , 1999, Optics East.