Multi-video browsing and summarization

We propose a method for browsing multiple videos with a common theme, such as the result of a search query on a video sharing website, or videos of an event covered by multiple cameras. Given the collection of videos we first align each video with all others. This pairwise video alignment forms the basis of a novel browsing interface, termed the Browsing Companion. It is used to play a primary video and, in addition as thumbnails, other video clips that are temporally synchronized with it. The user can, at any time, click on one of the thumbnails to make it the primary. We also show that video alignment can be used for other applications such as automatic highlight detection and multi-video summarization.

[1]  M. Irani,et al.  Spatio-Temporal Alignment of Sequences , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Yael Pritch,et al.  This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2008 1 Non-Chronological Video , 2022 .

[3]  Tinne Tuytelaars,et al.  Synchronizing video sequences , 2004, CVPR 2004.

[4]  Ariel Shamir,et al.  Mode-detection via median-shift , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[5]  Alexander M. Bronstein,et al.  The Video Genome , 2010, ArXiv.

[6]  Bing-Yu Chen,et al.  SmartPlayer: user-centric video fast-forwarding , 2009, CHI.

[7]  Chng Eng Siong,et al.  Automatic replay generation for soccer video broadcasting , 2004, MULTIMEDIA '04.

[8]  Cordelia Schmid,et al.  Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[9]  Remi Depommier,et al.  Content-based browsing of video sequences , 1994, MULTIMEDIA '94.

[10]  John R. Kender,et al.  Video Summaries through Mosaic-Based Shot and Scene Clustering , 2002, ECCV.

[11]  Nebojsa Jojic,et al.  Adaptive Video Fast Forward , 2005, Multimedia Tools and Applications.

[12]  Paul Over,et al.  Evaluation campaigns and TRECVid , 2006, MIR '06.

[13]  Biing-Hwang Juang,et al.  Fundamentals of speech recognition , 1993, Prentice Hall signal processing series.

[14]  Xiangming Mu.  A content-based video browsing system based on visual neighbor similarity , 2006, Proceedings of the 6th ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL '06).

[15]  David Burshtein,et al.  Support Vector Machine Training for Improved Hidden Markov Modeling , 2008, IEEE Transactions on Signal Processing.

[16]  Noel E. O'Connor,et al.  An interactive and multi-level framework for summarising user generated videos , 2009, ACM Multimedia.

[17]  David S. Doermann,et al.  Video summarization by curve simplification , 1998, MULTIMEDIA '98.

[18]  Samy Bengio,et al.  Semi-supervised adapted HMMs for unusual event detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[19]  Seth Teller,et al.  Video matching , 2004, SIGGRAPH 2004.

[20]  Michael A. Smith,et al.  Video skimming and characterization through the combination of image and language understanding techniques , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[21]  Yi-Ping Phoebe Chen,et al.  Content-based video indexing for sports applications using integrated multi-modal approach , 2005, MULTIMEDIA '05.

[22]  Patrick Pérez,et al.  Retrieving actions in movies , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[23]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[24]  Michal Irani,et al.  Aligning Sequences and Actions by Maximizing Space-Time Correlations , 2006, ECCV.

[25]  F. Dirfaux.  Key frame selection to represent a video , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[26]  Michal Irani,et al.  Detecting Irregularities in Images and in Video , 2005, ICCV.

[27]  Andrew Zisserman,et al.  Video Google: a text retrieval approach to object matching in videos , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.