Scene Determination Based on Video and Audio Features

Determining automatically what constitutes a scene in a video is a challenging task, particularly since there is no precise definition of the term "scene". It is left to the individual to set attributes shared by consecutive shots which group them into scenes. Certain basic attributes such as dialogs, like settings and continuing sounds are consistent indicators. We have therefore developed a scheme for identifying scenes by clustering shots according to detected dialogs, like settings and similar audio. Results from experiments show automatic identification of these types of scenes to be reliable.

[1]  David Bordwell,et al.  Film Art: An Introduction , 1979 .

[2]  Frank Eugene Beaver,et al.  Dictionary of film terms , 1983 .

[3]  Alan A. Armer.  Directing television and film , 1986 .

[4]  Bernd Jähne,et al.  Digital Image Processing: Concepts, Algorithms, and Scientific Applications , 1991 .

[5]  Remi Depommier,et al.  Content-based browsing of video sequences , 1994, MULTIMEDIA '94.

[6]  Alex Pentland,et al.  View-based and modular eigenspaces for face recognition , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Rosalind W. Picard,et al.  Texture orientation for sorting photos "at a glance" , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[8]  Boon-Lock Yeo,et al.  Video browsing using clustering and scene transitions on compressed sequences , 1995, Electronic Imaging.

[9]  Ramin Zabih,et al.  A feature-based algorithm for detecting and classifying scene breaks , 1995, MULTIMEDIA '95.

[10]  Boon-Lock Yeo,et al.  Time-constrained clustering for segmentation of video into story units , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[11]  John M. Gauch,et al.  Vision: a digital video library , 1996, DL '96.

[12]  Shih-Fu Chang,et al.  Clustering methods for video browsing and annotation , 1996, Electronic Imaging.

[13]  Jitendra Malik,et al.  Recognition of Images in Large Databases Using a Learning Framework , 1997 .

[14]  Hiroshi Hamada,et al.  Enhanced video handling based on audio analysis , 1997, Proceedings of IEEE International Conference on Multimedia Computing and Systems.

[15]  Qi Tian,et al.  An automatic news video parsing, indexing and browsing system , 1997, MULTIMEDIA '96.

[16]  Yukinobu Taniguchi,et al.  PanoramaExcerpts: extracting and packing panoramas for video browsing , 1997, MULTIMEDIA '97.

[17]  Boon-Lock Yeo,et al.  Video content characterization and compaction for digital library applications , 1997, Electronic Imaging.

[18]  Ah Chung Tsoi,et al.  Face recognition: a convolutional neural-network approach , 1997, IEEE Trans. Neural Networks.

[19]  Jeho Nam,et al.  Speaker identification and video analysis for hierarchical video shot classification , 1997, Proceedings of International Conference on Image Processing.

[20]  Jing Huang,et al.  Image indexing using color correlograms , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[21]  Wolfgang Effelsberg,et al.  Video abstracting , 1997, CACM.

[22]  Mark T. Maybury,et al.  Broadcast news navigation using story segmentation , 1997, MULTIMEDIA '97.

[23]  Riccardo Leonardi,et al.  Audio as a support to scene change detection and characterization of video sequences , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[24]  Ramin Zabih,et al.  Comparing images using color coherence vectors , 1997, MULTIMEDIA '96.

[25]  Osamu Hori,et al.  A shot classification method of selecting effective key-frames for video browsing , 1997, MULTIMEDIA '96.

[26]  John M. Gauch,et al.  The vision digital video library , 1997, Inf. Process. Manag..

[27]  Wolfgang Effelsberg,et al.  Towards a Visual Grep: A systematic analysis of various methods to compare video sequences , 1998 .

[28]  Tsuhan Chen,et al.  Audio Feature Extraction and Analysis for Scene Segmentation and Classification , 1998, J. VLSI Signal Process..

[29]  Boon-Lock Yeo,et al.  Video query: Research directions , 1998, IBM J. Res. Dev..