A method for extraction of audio-visual leitmotif in movies by cross media analysis

The joint analysis of video and audio components in multimedia documents has been widely used since the beginning of activities related to the new multimedia Standard MPEG7. In this context the paper is focused on a method for extraction of an emotional and structuring cue from artistic content we call “audio-visual leitmotif”, which is a first step in characterization of an author style in producing the content. The method is based on joint motion -based video partitioning and on model-based music recognition.

[1]  Patrick Bouthemy,et al.  A unified approach to shot change detection and camera motion characterization , 1999, IEEE Trans. Circuits Syst. Video Technol..

[2]  Riccardo Leonardi,et al.  Audio as a support to scene change detection and characterization of video sequences , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[3]  Antti Eronen,et al.  Automatic musical instrument recognition , 2001 .

[4]  Xavier Rodet,et al.  Automatic Characterisation of Musical Signals: Feature Extraction and Temporal Segmentation , 1999 .

[5]  John R. Smith,et al.  MPEG-7 multimedia description schemes , 2001, IEEE Trans. Circuits Syst. Video Technol..

[6]  Philippe Joly,et al.  Efficient automatic analysis of camera work and microsegmentation of video using spatiotemporal images , 1996, Signal Process. Image Commun..

[7]  Chung-Lin Huang,et al.  A robust scene-change detection method for video segmentation , 2001, IEEE Trans. Circuits Syst. Video Technol..

[8]  Riccardo Leonardi,et al.  Identification of story units in audio-visual sequences by joint audio and video processing , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).