New frontiers for intelligent content-based retrieval

In this paper, we examine emerging frontiers in the evolution of content-based retrieval systems that rely on an intelligent infrastructure. Here, we refer to intelligence as the capabilities of the systems to build and maintain situational or world models, utilize dynamic knowledge representation, exploit context, and leverage advanced reasoning and learning capabilities. We argue that these elements are essential to producing effective systems for retrieving audio-visual content at semantic levels matching those of human perception and cognition. In this paper, we review relevant research on the understanding of human intelligence and construction of intelligent system in the fields of cognitive psychology, artificial intelligence, semiotics, and computer vision. We also discus how some of the principal ideas form these fields lead to new opportunities and capabilities for content-based retrieval systems. Finally, we describe some of our efforts in these directions. In particular, we present MediaNet, a multimedia knowledge presentation framework, and some MPEG-7 description tools that facilitate and enable intelligent content-based retrieval.

[1]  Hugh C. Davis,et al.  Towards Multimedia Thesaurus Support for Media-based Navigation , 1998, Image Databases and Multi-Media Search.

[2]  Stephen W. Smoliar,et al.  Multi-Media Search: An Authoring Perspective , 1998, Image Databases and Multi-Media Search.

[3]  Marvin Minsky,et al.  A framework for representing knowledge" in the psychology of computer vision , 1975 .

[4]  H. Chertkow,et al.  Semantic memory , 2002, Current neurology and neuroscience reports.

[5]  Avron Barr,et al.  Representation of Knowledge , 1980 .

[6]  Azriel Rosenfeld,et al.  Computer Vision , 1988, Adv. Comput..

[7]  Marvin Minsky,et al.  A framework for representing knowledge , 1974 .

[8]  Wolfgang Bibel,et al.  The Representation of Knowledge , 1989 .

[9]  Alexander M. Meystel,et al.  Semiotic Modeling and Situation Analysis : An Introduction , 1995 .

[10]  Anil K. Jain,et al.  On image classification: city vs. landscape , 1998, Proceedings. IEEE Workshop on Content-Based Access of Image and Video Libraries (Cat. No.98EX173).

[11]  Shih-Fu Chang,et al.  VisualSEEk: a fully automated content-based image query system , 1997, MULTIMEDIA '96.

[12]  John R. Smith,et al.  Quantitative assessment of image retrieval effectiveness , 2001, J. Assoc. Inf. Sci. Technol..

[13]  Thomas S. Huang,et al.  Relevance feedback techniques in interactive content-based image retrieval , 1997, Electronic Imaging.

[14]  S. Kosslyn.  Image and mind , 1982 .

[15]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[16]  Dragutin Petkovic,et al.  Query by Image and Video Content: The QBIC System , 1995, Computer.

[17]  Martin Szummer,et al.  Indoor-outdoor image classification , 1998, Proceedings 1998 IEEE International Workshop on Content-Based Access of Image and Video Database.

[18]  Paul H. Lewis,et al.  Semiotics and agents for integrating and navigating through multimedia representations of concepts , 1999, Electronic Imaging.

[19]  Alberto Del Bimbo,et al.  Expressive Semantics for Automatic Annotation and Retrieval of Video Streams , 2000, IEEE International Conference on Multimedia and Expo.

[20]  John R. Smith,et al.  Conceptual modeling of audio-visual content , 2000, 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No.00TH8532).

[21]  Shih-Fu Chang,et al.  MediaNet: a multimedia information network for knowledge representation , 2000, SPIE Optics East.

[22]  B. S. Manjunath,et al.  A Texture Thesaurus for Browsing Large Aerial Photographs , 1998, J. Am. Soc. Inf. Sci..

[23]  George A. Miller,et al.  WordNet: A Lexical Database for English , 1995, HLT.

[24]  Clement T. Yu,et al.  Using semantic contents and WordNet in image retrieval , 1997, SIGIR '97.

[25]  Shih-Fu Chang,et al.  SaFe: a general framework for integrated spatial and feature image search , 1997, Proceedings of First Signal Processing Society Workshop on Multimedia Signal Processing.

[26]  D. Lenat.  The Dimensions of Context-Space , 1998 .

[27]  Amarnath Gupta,et al.  Virage image search engine: an open framework for image management , 1996, Electronic Imaging.

[28]  Marvin Minsky,et al.  Semantic Information Processing , 1968 .

[29]  Rodney A. Brooks,et al.  Building brains for bodies , 1995, Auton. Robots.

[30]  Brian Scassellati,et al.  Alternative Essences of Intelligence , 1998, AAAI/IAAI.

[31]  Rodney A. Brooks,et al.  Intelligence Without Reason , 1991, IJCAI.

[32]  Robert Tansley.  The multimedia thesaurus : adding a semantic layer to multimedia information , 2000 .

[33]  Shih-Fu Chang,et al.  Integration of Visual and Text-Based Approaches for the Content Labeling and Classification of Photographs , 1999, SIGIR 1999.