Automatic Home Video Editing

In this chapter, we present an optimization-based system for editing home video in an automatic manner. This system automatically selects highlight segments from a set of raw home videos and aligns them with a given piece of incidental music to create an edited video segment with a desired length, which is based on the content of the video and the incidental music. We developeded an approach for extracting temporal structure and determining the importance of a video segment in order to facilitate the selection of highlight segments. And we also extract temporal structure, beats and tempos from the incidental music. In order to create more professional-looking results, the selected highlight segments satisfy a set of editing rules and are matched to the content of the incidental music. This task is formulated as a nonlinear 0-1 programming problem with adjustable and increasable constraints. The output video is rendered by connecting the selected highlight video segments with transition effects and the incidental music. We will also introduce the real product with this technology, as well as a set of extensions based on the automatic home video editing system.

[1]  Xian-Sheng Hua,et al.  Automatic location of text in video frames , 2001, MULTIMEDIA '01.

[2]  Rainer Lienhart.  Dynamic video summarization of home video , 1999, Electronic Imaging.

[3]  Anoop Gupta,et al.  Time-compression: systems concerns, usage, and benefits , 1999, CHI '99.

[4]  Michael A. Smith,et al.  Video skimming and characterization through the combination of image and language understanding techniques , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[5]  HongJiang Zhang,et al.  Video scene extraction by force competition , 2001, IEEE International Conference on Multimedia and Expo, 2001. ICME 2001..

[6]  Shipeng Li,et al.  Interactive video authoring and sharing based on two-layer templates , 2006, HCM '06.

[7]  John R. Kender,et al.  Video scene segmentation via continuous video coherence , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[8]  Alan Hanjalic,et al.  Automated high-level movie segmentation for advanced video-retrieval systems , 1999, IEEE Trans. Circuits Syst. Video Technol..

[9]  Atreyi Kankanhalli,et al.  Automatic partitioning of full-motion video , 1993, Multimedia Systems.

[10]  Andreas Girgensohn,et al.  Creating music videos using automatic media analysis , 2002, MULTIMEDIA '02.

[11]  Laurent Itti,et al.  Real-time high-performance attention focusing in outdoors color video streams , 2002, IS&T/SPIE Electronic Imaging.

[12]  Rainer Lienhart,et al.  On the segmentation of text in videos , 2000, 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No.00TH8532).

[13]  Mohan S. Kankanhalli,et al.  Detection and removal of lighting & shaking artifacts in home videos , 2002, MULTIMEDIA '02.

[14]  André Gagalowicz,et al.  Image-based rendering of diffuse, specular and glossy surfaces from a single image , 2001, SIGGRAPH.

[15]  Lie Lu,et al.  Optimization-based automated home video editing system , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[16]  Darrell Whitley,et al.  A genetic algorithm tutorial , 1994, Statistics and Computing.

[17]  Rainer Lienhart,et al.  Abstracting home video automatically , 1999, MULTIMEDIA '99.

[18]  David S. Doermann,et al.  Video summarization by curve simplification , 1998, MULTIMEDIA '98.

[19]  Anthony Stefanidis,et al.  Summarizing video datasets in the spatiotemporal domain , 2000, Proceedings 11th International Workshop on Database and Expert Systems Applications.

[20]  Lie Lu,et al.  Speech segmentation without speech recognition , 2003, 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698).

[21]  Eric D. Scheirer,et al.  Tempo and beat analysis of acoustic musical signals. , 1998, The Journal of the Acoustical Society of America.

[22]  Shingo Uchihashi,et al.  A semi-automatic approach to home video editing , 2000, UIST '00.

[23]  Lie Lu,et al.  Automatic music video generation based on temporal pattern analysis , 2004, MULTIMEDIA '04.

[24]  Ken-ichi Anjyo,et al.  Tour into the picture: using a spidery mesh interface to make animation from a single image , 1997, SIGGRAPH.

[25]  Xavier Binefa,et al.  An EM algorithm for video summarization, generative model approach , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[26]  Xin Liu,et al.  Video summarization using singular value decomposition , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[27]  Shih-Fu Chang,et al.  A utility framework for the automatic generation of audio-visual skims , 2002, MULTIMEDIA '02.

[28]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1997, EuroCOLT.

[29]  Harry Shum,et al.  Statistical Learning of Multi-view Face Detection , 2002, ECCV.

[30]  Jeho Nam,et al.  Dynamic video summarization and visualization , 1999, MULTIMEDIA '99.

[31]  Lie Lu,et al.  Automatically converting photographic series into video , 2004, MULTIMEDIA '04.

[32]  Kuniaki Uehara,et al.  Mining video editing rules in video streams , 2002, MULTIMEDIA '02.

[33]  Lie Lu,et al.  A robust audio classification and segmentation method , 2001, MULTIMEDIA '01.

[34]  Jan Ozer.  Microsoft Windows Movie Maker 2: Visual QuickStart Guide , 2004 .

[35]  Shipeng Li,et al.  SMART: An Efficient, Scalable, and Robust Streaming Video System , 2004, EURASIP J. Adv. Signal Process..

[36]  Frédo Durand,et al.  A gentle introduction to bilateral filtering and its applications , 2007, SIGGRAPH Courses.

[37]  Xian-Sheng Hua,et al.  Personal media sharing and authoring on the web , 2005, MULTIMEDIA '05.

[38]  Frank M. Shipman,et al.  Home Video Editing Made Easy - Balancing Automation and User Control , 2001, INTERACT.