Compact Visualisation of Video Summaries

This paper presents a system for compact and intuitive video summarisation aimed at both high-end professional production environments and small-screen portable devices. To represent large amounts of information in the form of a video key-frame summary, this paper studies the narrative grammar of comics, and using its universal and intuitive rules, lays out visual summaries in an efficient and user-centered way. In addition, the system exploits visual attention modelling and rapid serial visual presentation to generate highly compact summaries on mobile devices. A robust real-time algorithm for key-frame extraction is presented. The system ranks importance of key-frame sizes in the final layout by balancing the dominant visual representability and discovery of unanticipated content utilising a specific cost function and an unsupervised robust spectral clustering technique. A final layout is created using an optimisation algorithm based on dynamic programming. Algorithm efficiency and robustness are demonstrated by comparing the results with a manually labelled ground truth and with optimal panelling solutions.

[1]  Janko Calic,et al.  Spatial analysis in key-frame extraction using video segmentation , 2004 .

[2]  Dirk Walther,et al.  Interactions of visual attention and object recognition : computational modeling, algorithms, and psychophysics. , 2006 .

[3]  Paul E. Sweeney,et al.  Cutting and Packing Problems: A Categorized, Application-Orientated Research Bibliography , 1992 .

[4]  C. Koch,et al.  Computational modelling of visual attention , 2001, Nature Reviews Neuroscience.

[5]  Andreas Girgensohn,et al.  A fast layout algorithm for visual video summaries , 2003, 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698).

[6]  Anil K. Jain,et al.  Data clustering: a review , 1999, CSUR.

[7]  Albert Nijenhuis,et al.  Combinatorial Algorithms for Computers and Calculators , 1978 .

[8]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[9]  Oscar de Bruijn,et al.  Rapid serial visual presentation: a space-time trade-off in information presentation , 2000, AVI '00.

[10]  Marcel Worring,et al.  Content-Based Image Retrieval at the End of the Early Years , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Andrea Lodi,et al.  Two-dimensional packing problems: A survey , 2002, Eur. J. Oper. Res..

[12]  T. W. Ridler,et al.  Picture thresholding using an iterative selection method. , 1978 .

[13]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[14]  P. B. Coaker,et al.  Applied Dynamic Programming , 1964 .

[15]  Gary Marchionini,et al.  Dynamic key frame presentation techniques for augmenting video browsing , 1998, AVI '98.

[16]  Pietro Perona,et al.  Grouping and dimensionality reduction by locally linear embedding , 2001, NIPS.

[17]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[18]  Janko Calic,et al.  Efficient Layout of Comic-Like Video Summaries , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[19]  Shingo Uchihashi,et al.  Video Manga: generating semantically meaningful video summaries , 1999, MULTIMEDIA '99.

[20]  Majid Mirmehdi,et al.  Temporal video segmentation and classification of edit effects , 2003, Image Vis. Comput..

[21]  Boon-Lock Yeo,et al.  Rapid scene analysis on compressed video , 1995, IEEE Trans. Circuits Syst. Video Technol..

[22]  John P. Collomosse,et al.  Video Analysis for Cartoon-like Special Effects , 2003, BMVC.

[23]  G. Andrews.  The Theory of Partitions: Frontmatter , 1976 .

[24]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[25]  Pietro Perona,et al.  Self-Tuning Spectral Clustering , 2004, NIPS.

[26]  Robert Spence,et al.  Rapid, Serial and Visual: A Presentation Technique with Potential , 2002, Inf. Vis..

[27]  Lev Kuleshov,et al.  Kuleshov on Film: Writings by Lev Kuleshov , 1975 .

[28]  W. Eisner.  Comics and Sequential Art , 1985 .

[29]  John A. Robinson,et al.  Techniques for automated reverse storyboarding , 2005 .

[30]  Majid Mirmehdi,et al.  ICBR - Multimedia Management System for Intelligent Content Based Retrieval , 2004, CIVR.

[31]  David Bull,et al.  Towards Intelligent Content Based Retrieval of Wildlife Videos , 2005 .