Collages as Dynamic Summaries of Mined Video Content for Intelligent Multimedia Knowledge Management

The video collage is a novel and effective interface for dynamically summarizing and presenting mined multimedia information from video collections. It provides a flexible mechanism for inferring knowledge and trailing analytic thoughts, which are the essential tasks in multimedia knowledge management. We will describe multimedia knowledge management, discuss how collages are automatically produced, illustrate their use for multimedia knowledge management, and evaluate their effectiveness as summaries across news stories. Collages are presentations of text and images extracted from multiple video sources. They provide an interactive visualization for a set of analyzed video documents, summarizing their contents and offering a navigation aid for further exploration. The dynamic creation of collages is based on user context, e.g., an originating query, coupled with automatic processing to refine the candidate imagery. Named entity identification and common phrase extraction provides associative textual description. The dynamic manipulation of collages allows user-directed filtering as well as revealing additional details and previously undiscovered content. The utility of collages as summaries is examined with respect to other published news summaries.

[1]  Richard M. Schwartz,et al.  Nymble: a High-Performance Learning Name-finder , 1997, ANLP.

[2]  Edward R. Tufte,et al.  Envisioning Information , 1990 .

[3]  Ronald Rosenfeld,et al.  Statistical language modeling using the CMU-cambridge toolkit , 1997, EUROSPEECH.

[4]  Debbie Richards,et al.  Visual Mapping of Articulable Tacit Knowledge , 2001, InVis.au.

[5]  Marcia J. Bates,et al.  The design of browsing and berrypicking techniques for the online search interface , 1989 .

[6]  Gary Marchionini,et al.  Multimodal surrogates for video browsing , 1999, DL '99.

[7]  John D. Lowrance,et al.  Capturing analytic thought , 2001, K-CAP '01.

[8]  M. G. Rodd,et al.  Knowledge management in a multimedia environment: some key research issues , 2002, Student Conference on Research and Development.

[9]  Shih-Fu Chang,et al.  Multimedia access and retrieval: the state of the art and future directions (panel session). , 1999, ACM Multimedia.

[10]  Henry Schneiderman,et al.  A histogram-based method for detection of faces and cars , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[11]  Yihong Gong,et al.  Automatic parsing of news video , 1994, 1994 Proceedings of IEEE International Conference on Multimedia Computing and Systems.

[12]  Yihong Gong,et al.  Lessons Learned from Building a Terabyte Digital Video Library , 1999, Computer.

[13]  Shingo Uchihashi,et al.  An interactive comic book presentation for exploring video , 2000, CHI.

[14]  Eric Brill,et al.  A Simple Rule-Based Part of Speech Tagger , 1992, HLT.

[15]  M. L. Mitchell,et al.  Visual Explanations: Images and Quantities, Evidence and Narrative , 1997 .

[16]  Ben Shneiderman,et al.  Visual information seeking: tight coupling of dynamic query filters with starfield displays , 1994, CHI '94.

[17]  Alain Breuleux,et al.  Multimedia and Comprehension: The Relationship among Text, Animation and Captions , 1995, J. Am. Soc. Inf. Sci..

[18]  Michael G. Christel,et al.  The effect of text in storyboards for video navigation , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).

[19]  Ricky Houghton.  Named Faces: Putting Names to Faces , 1999, IEEE Intell. Syst..

[20]  Scott McCoy,et al.  The most important issues in knowledge management , 2002, CACM.

[21]  Michael G. Christel.  Visual digests for news video libraries , 1999, MULTIMEDIA '99.

[22]  Dorothy E. Leidner,et al.  Knowledge Management Systems: Issues, Challenges, and Benefits , 1999, Commun. Assoc. Inf. Syst..

[23]  Ralph Weischedel,et al.  Named Entity Extraction from Broadcast News , 1999 .

[24]  Mark T. Maybury,et al.  Broadcast news navigation using story segmentation , 1997, MULTIMEDIA '97.

[25]  Hsinchun Chen,et al.  Knowledge Management Systems: A Text Mining Perspective , 2001 .

[26]  Ben Shneiderman,et al.  The eyes have it: a task by data type taxonomy for information visualizations , 1996, Proceedings 1996 IEEE Symposium on Visual Languages.

[27]  Anoop Gupta,et al.  Browsing digital video , 2000, CHI.

[28]  Alexander Dekhtyar,et al.  Information Retrieval , 2018, Lecture Notes in Computer Science.