Interactive Logical Story Unit Segmenta- tion

Logical Story Unit (LSU) segmentation in general domains, such as movies and television series, requires interaction between human experts and automatic tools. We present an interaction model that allows users, who are non-experts in the field of video processing, to segment videos into logical story units by tuning models rather than manual correction of results of automatic methods. Suitable features are determined interactively by visualizing their values in the context of the original video data. The VidDex system developed uses backprojection on various video visualization modes to meet presented requirements for good visualization. Evaluation on two Hollywood movies shows that systematic adaptation of LSU-segmentation parameters is more effective than adaptation of the segmentation results. More importantly it allows for storage of human judgement on computed similarity functions.

[1]  Marcel Worring,et al.  Multimodal Video Indexing : A Review of the State-ofthe-art , 2001 .

[2]  Arnold W. M. Smeulders,et al.  Grouping lines. Finding curvilinear structures in images , 2001 .

[3]  A. Murat Tekalp,et al.  Efficient Filtering and Clustering Methods for Temporal Video Segmentation and Visual Summarization , 1998, J. Vis. Commun. Image Represent..

[4]  Shih-Fu Chang,et al.  Integrated spatial and feature image query , 1999, Multimedia Systems.

[5]  Christos Faloutsos,et al.  VideoTrails: representing and visualizing structure in video sequences , 1997, MULTIMEDIA '97.

[6]  Alan Hanjalic,et al.  Automatically Segmenting Movies into Logical Story Units , 1999, VISUAL.

[7]  Joseph M. Boggs.  The Art of Watching Films , 1978 .

[8]  Arnold W. M. Smeulders,et al.  Statistical strategy for object class recognition using part detectors , 2001 .

[9]  Rosalind W. Picard.  A Society of Models for Video and Image Libraries , 1996, IBM Syst. J..

[10]  Marcel Worring,et al.  Feature Driven Visualization of Video Content for Interactive Indexing , 2000, VISUAL.

[11]  Simone Santini,et al.  Beyond query by example , 1998, MULTIMEDIA '98.

[12]  Marcel Worring,et al.  The UvA color document dataset , 2004, International Journal of Document Analysis and Recognition (IJDAR).

[13]  Arnold W. M. Smeulders,et al.  A line tracker , 1997 .

[14]  Thomas S. Huang,et al.  Constructing table-of-content for videos , 1999, Multimedia Systems.

[15]  Boon-Lock Yeo,et al.  Retrieving and visualizing video , 1997, CACM.

[16]  Silvia Delgado Olabarriaga,et al.  Human-Computer Interaction for the Segmentation of Medical Images , 1999 .

[17]  Marcel Worring,et al.  Filter Image Browsing: Interactive Image Retrieval by Using Database Overviews , 2001, Multimedia Tools and Applications.

[18]  Marcel Worring,et al.  Systematic evaluation of logical story unit segmentation , 2002, IEEE Trans. Multim..

[19]  Jan Biemond,et al.  Image and Video Databases: Restoration, Watermarking and Retrieval , 2000 .

[20]  Boon-Lock Yeo,et al.  Time-constrained clustering for segmentation of video into story units , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[21]  Philippe Aigrain,et al.  Medium knowledge-based macro-segmentation of video into sequences , 1997 .

[22]  Arnold W. M. Smeulders,et al.  Combining strings and necklaces for interactive three-dimensional segmentation of spinal images using an Integral deformable spine model , 2004, IEEE Transactions on Biomedical Engineering.

[23]  R. Brunelli,et al.  A Survey on the Automatic Indexing of Video Data, , 1999, J. Vis. Commun. Image Represent..