Temporal semantic compression for video browsing

We present a video browsing approach, termed Temporal Semantic Compression (TSC), that uses automated measures of interest to support today's foraging behaviours. Conventional browsers 'compress' a video stream using simple 2x or 8x fast-forward. TSC browsers dynamically filter video based on a single user gesture to leave out more or less of the boring bits. We demonstrate a browser with an example interest measure, derived from an automated estimate of movie tempo, to forage in terms of narrative structures such as crises, climaxes, and action sequence book-ends. Media understanding algorithms facilitate browsing, and interactivity enables the human-in-the-loop to cope when those algorithms fail to cross the semantic gap.

[1]  Marco Ceccarelli,et al.  The color browser: a content driven linear video browsing tool , 2001, IEEE International Conference on Multimedia and Expo, 2001. ICME 2001..

[2]  Svetha Venkatesh,et al.  A Probabilistic Framework for Extracting Narrative Act Boundaries and Semantics in Motion Pictures , 2005, Multimedia Tools and Applications.

[3]  Svetha Venkatesh,et al.  Multi-modal emotive computing in a smart house environment , 2007, Pervasive Mob. Comput..

[4]  Chong-Wah Ngo,et al.  Structuring home video by snippet detection and pattern parsing , 2004, MIR '04.

[5]  Svetha Venkatesh,et al.  Toward automatic extraction of expressive elements from motion pictures: tempo , 2002, IEEE Trans. Multim..

[6]  Steven M. Drucker,et al.  SmartSkip: consumer level browsing and skipping of digital video content , 2002, CHI.

[7]  Svetha Venkatesh,et al.  Towards automatic extraction of expressive elements from motion pictures: tempo , 2000, 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No.00TH8532).

[8]  John R. Kender,et al.  Time-constrained dynamic semantic compression for video indexing and interactive searching , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[9]  Chia-Hung Yeh,et al.  Techniques for movie content analysis and skimming: tutorial and overview on video abstraction techniques , 2006, IEEE Signal Processing Magazine.

[10]  Anoop Gupta,et al.  Browsing digital video , 2000, CHI.

[11]  Alan F. Smeaton,et al.  Designing the User Interface for the Físchlár Digital Video Library , 2006, J. Digit. Inf..

[12]  S. Shipman,et al.  Augmenting fast-forward and rewind for personal digital video recorders , 2005, 2005 Digest of Technical Papers. International Conference on Consumer Electronics, 2005. ICCE..

[13]  Alan Hanjalic,et al.  Adaptive extraction of highlights from a sport video based on excitement modeling , 2005, IEEE Transactions on Multimedia.

[14]  Wolfgang Hürst,et al.  Interactive, dynamic video browsing with the zoomslider interface , 2005, 2005 IEEE International Conference on Multimedia and Expo.