A Computational Framework for Vertical Video Editing

Vertical video editing is the process of digitally editing the image within the frame as opposed to horizontal video editing, which arranges the shots along a timeline. Vertical editing can be a time-consuming and error-prone process when using manual key-framing and simple interpolation. In this paper, we present a general framework for automatically computing a variety of cinematically plausible shots from a single input video suitable to the special case of live performances. Drawing on working practices in traditional cinematography, the system acts as a virtual camera assistant to the film editor, who can call novel shots in the edit room with a combination of high-level instructions and manually selected keyframes.

[1]  Rene Kaiser,et al.  The FascinatE Production Scripting Engine , 2012, MMM.

[2]  Christel Chamaret,et al.  Attention-based video reframing: Validation using eye-tracking , 2008, 2008 19th International Conference on Pattern Recognition.

[3]  Hermann Ney,et al.  Pan, zoom, scan — Time-coherent, trained automatic video cropping , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Rémi Ronfard,et al.  Multi-clip video editing from a single viewpoint , 2014, CVMP.

[5]  Andrew R. Parker,et al.  In The Blink Of An Eye , 2003 .

[6]  Michael Gleicher,et al.  Subspace video stabilization , 2011, TOGS.

[7]  RuiYong,et al.  An automated end-to-end lecture capture and broadcasting system , 2008 .

[8]  Michael Bianchi.  Automatic video production of lectures using an intelligent and aware environment , 2004, MUM '04.

[9]  Cordelia Schmid,et al.  Human Detection Using Oriented Histograms of Flow and Appearance , 2006, ECCV.

[10]  Christian Weissig,et al.  Ultrahigh-Resolution Panoramic Imaging for Format-Agnostic Video Production , 2013, Proceedings of the IEEE.

[11]  Eli Peli,et al.  Where people look when watching movies: Do all viewers look at the same place? , 2007, Comput. Biol. Medicine.

[12]  Lie Lu,et al.  Optimization-based automated home video editing system , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[13]  Roy Thompson,et al.  Grammar of the Shot , 1998 .

[14]  Yaser Sheikh,et al.  Gaze-Driven Video Re-Editing , 2015, TOGS.

[15]  Rémi Ronfard,et al.  Detecting and Naming Actors in Movies Using Generative Appearance Models , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Irfan A. Essa,et al.  Auto-directed video stabilization with robust L1 optimal camera paths , 2011, CVPR 2011.

[17]  Yasuo Ariki,et al.  Video editing support system based on video grammar and content analysis , 2002, Object recognition supported by user interaction for service robots.

[18]  Wilmot Li,et al.  Tools for placing cuts and transitions in interview video , 2012, ACM Trans. Graph..

[19]  David A. McAllester,et al.  Object Detection with Discriminatively Trained Part Based Models , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Vineet Gandhi.  Automatic Rush Generation with Application to Theatre Performances. (Généation Automatique de Prises de Vues Cinématographiques avec Applications aux Captations de Théâtre) , 2014 .

[21]  Michael Gleicher,et al.  Virtual videography , 2006, MM '06.

[22]  Andrew Zisserman,et al.  Video Google: a text retrieval approach to object matching in videos , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[23]  Rémi Ronfard,et al.  The Prose Storyboard Language: A Tool for Annotating and Directing Movies , 2015, FDG 2015.