Multimedia Knowledge Integration, Summarization And Evaluation

This paper presents new methods for automatically integrating, summarizing and evaluating multimedia knowledge. These are essential for multimedia applications to efficiently and coherently deal with multimedia knowledge at different abstraction levels such as perceptual and semantic knowledge (e.g., image clusters and word senses, respectively). The proposed methods include automatic techniques (1) for interrelating the concepts in the multimedia knowledge using probabilistic Bayesian learning, (2) for reducing the size of multimedia knowledge by clustering the concepts and collapsing the relationships among the clusters, and (3) for evaluating the quality of multimedia knowledge using notions from information and graph theory. Experiments show the potential of knowledge integration techniques for improving the knowledge quality, the importance of good concept distance measures for clustering and summarizing knowledge, and the usefulness of automatic measures for comparing the effects of different processing techniques on multimedia knowledge.

[1]  Shih-Fu Chang,et al.  Semantic knowledge construction from annotated image collections , 2002, Proceedings. IEEE International Conference on Multimedia and Expo.

[2]  Christiane Fellbaum,et al.  Combining Local Context and Wordnet Similarity for Word Sense Identification , 1998 .

[3]  Ray A. Jarvis,et al.  Clustering Using a Similarity Measure Based on Shared Near Neighbors , 1973, IEEE Transactions on Computers.

[4]  Michael J. Muller,et al.  VISAR: a system for inference and navigation of hypertext , 1989, Hypertext.

[5]  Martin Chodorow,et al.  Combining local context and wordnet similarity for word sense identification , 1998 .

[6]  Shih-Fu Chang,et al.  A knowledge engineering approach for image classification based on probabilistic reasoning systems , 2000, 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No.00TH8532).

[7]  Rada Mihalcea,et al.  Automatic generation of a coarse grained WordNet , 2001, HTL 2001.

[8]  Alicia Perez,et al.  Evaluation of Taxonomic Knowledge in Ontologies and Knowledge Bases , 1999 .

[9]  A. Shapiro.  Monte Carlo Sampling Methods , 2003 .

[10]  Robert Tansley.  The multimedia thesaurus : adding a semantic layer to multimedia information , 2000 .

[11]  Shih-Fu Chang,et al.  IMKA: a multimedia organization system combining perceptual and semantic knowledge , 2001, MULTIMEDIA '01.

[12]  Alan F. Smeaton,et al.  Using WordNet in a Knowledge-Based Approach to Information Retrieval , 1995 .

[13]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[14]  Shih-Fu Chang,et al.  MediaNet: a multimedia information network for knowledge representation , 2000, SPIE Optics East.

[15]  Graeme Hirst,et al.  Semantic distance in WordNet: An experimental, application-oriented evaluation of five measures , 2004 .

[16]  David A. Forsyth,et al.  Matching Words and Pictures , 2003, J. Mach. Learn. Res..

[17]  Martin Szummer,et al.  Indoor-outdoor image classification , 1998, Proceedings 1998 IEEE International Workshop on Content-Based Access of Image and Video Database.

[18]  Michael Sussna,et al.  Word sense disambiguation for free-text indexing using a massive semantic network , 1993, CIKM '93.

[19]  Anil K. Jain,et al.  On image classification: city vs. landscape , 1998, Proceedings. IEEE Workshop on Content-Based Access of Image and Video Libraries (Cat. No.98EX173).

[20]  W. K. Hastings,et al.  Monte Carlo Sampling Methods Using Markov Chains and Their Applications , 1970 .

[21]  David G. Stork,et al.  Pattern Classification , 1973 .

[22]  George A. Miller,et al.  WordNet: A Lexical Database for English , 1995, HLT.

[23]  David W. Conrath,et al.  Semantic Similarity Based on Corpus Statistics and Lexical Taxonomy , 1997, ROCLING/IJCLCLP.

[24]  Shih-Fu Chang,et al.  Perceptual knowledge construction from annotated image collections , 2002, Proceedings. IEEE International Conference on Multimedia and Expo.