Integration of audio and visual information for content-based video segmentation

A video sequence usually consists of separate scenes, and each scene includes many shots. For video understanding purposes, it is most important to detect scene breaks. To analyze the content of each scene, detection of shot breaks is also required. Usually, a scene break is associated with a simultaneous change of image, motion, and audio characteristics, while a shot break is only accompanied with changes in image or motion or both. We propose to use audio information along with image and motion information to accomplish segmentation at different levels. Promising results have been obtained with videos digitized from TV programs.

[1]  Tsuhan Chen,et al.  Multimedia content classification using motion and audio information , 1997, Proceedings of 1997 IEEE International Symposium on Circuits and Systems. Circuits and Systems in the Information Age ISCAS '97.

[2]  John S. Boreczky,et al.  A hidden Markov model framework for video segmentation using audio and image features , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[3]  Tsuhan Chen,et al.  Audio feature extraction and analysis for scene classification , 1997, Proceedings of First Signal Processing Society Workshop on Multimedia Signal Processing.

[4]  B. S. Manjunath,et al.  Content-based search of video using color, texture, and motion , 1997, Proceedings of International Conference on Image Processing.

[5]  Riccardo Leonardi,et al.  Audio as a support to scene change detection and characterization of video sequences , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[6]  Jeho Nam,et al.  Combined audio and visual streams analysis for video sequence segmentation , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[7]  Tsuhan Chen,et al.  Audio Feature Extraction and Analysis for Scene Segmentation and Classification , 1998, J. VLSI Signal Process..