Selectively de-animating video

We present a semi-automated technique for selectively deanimating video to remove the large-scale motions of one or more objects so that other motions are easier to see. The user draws strokes to indicate the regions of the video that should be immobilized, and our algorithm warps the video to remove the large-scale motion of these regions while leaving finer-scale, relative motions intact. However, such warps may introduce unnatural motions in previously motionless areas, such as background regions. We therefore use a graph-cut-based optimization to composite the warped video regions with still frames from the input video; we also optionally loop the output in a seamless manner. Our technique enables a number of applications such as clearer motion visualization, simpler creation of artistic cinemagraphs (photos that include looping motions in some regions), and new ways to edit appearance and complicated motion paths in video by manipulating a de-animated representation. We demonstrate the success of our technique with a number of motion visualizations, cinemagraphs and video editing examples created from a variety of short input videos, as well as visual and numerical comparison to previous techniques.

[1]  Kwan-Liu Ma,et al.  Dynamic video narratives , 2010, SIGGRAPH 2010.

[2]  Adam Finkelstein,et al.  Video tapestries with continuous temporal zoom , 2010, SIGGRAPH 2010.

[3]  Michael Gleicher,et al.  Content-preserving warps for 3D video stabilization , 2009, ACM Trans. Graph..

[4]  David Salesin,et al.  Panoramic video textures , 2005, ACM Trans. Graph..

[5]  Richard Szeliski,et al.  Video textures , 2000, SIGGRAPH.

[6]  Richard Szeliski,et al.  The Moment Camera , 2006, Computer.

[7]  M. Burrows,et al.  Biomechanics of jumping in the flea , 2011, Journal of Experimental Biology.

[8]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Frédo Durand,et al.  Motion denoising with application to time-lapse photography , 2011, CVPR 2011.

[10]  David Salesin,et al.  Interactive digital photomontage , 2004, SIGGRAPH 2004.

[11]  David Salesin,et al.  Animating pictures with stochastic motion textures , 2005, SIGGRAPH 2005.

[12]  Yasuyuki Matsushita,et al.  Dynamic stills and clip trailers , 2006, The Visual Computer.

[13]  Ba Tu Truong,et al.  Video abstraction: A systematic review and classification , 2007, TOMCCAP.

[14]  Irfan A. Essa,et al.  Video-based nonphotorealistic and expressive illustration of motion , 2005, International 2005 Computer Graphics.

[15]  Pushmeet Kohli,et al.  Unwrap mosaics: a new representation for video editing , 2008, SIGGRAPH 2008.

[16]  Leonard McMillan,et al.  Computational time-lapse video , 2007, SIGGRAPH 2007.

[17]  Guillermo Sapiro,et al.  Video SnapCut: robust video object cutout using localized classifiers , 2009, ACM Trans. Graph..

[18]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[19]  A. Torralba,et al.  Motion magnification , 2005, SIGGRAPH 2005.

[20]  David Salesin,et al.  Schematic storyboarding for video visualization and editing , 2006, SIGGRAPH 2006.

[21]  Irfan A. Essa,et al.  Graphcut textures: image and video synthesis using graph cuts , 2003, ACM Trans. Graph..

[22]  Olga Veksler,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Michael Gleicher,et al.  Subspace video stabilization , 2011, TOGS.

[24]  Jan Kautz,et al.  Towards Moment Imagery: Automatic Cinemagraphs , 2011, 2011 Conference for Visual Media Production.

[25]  Daniel Cohen-Or,et al.  Action synopsis: pose selection and illustration , 2005, ACM Trans. Graph..

[26]  Yael Pritch,et al.  This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2008 1 Non-Chronological Video , 2022 .

[27]  David Salesin,et al.  Video matting of complex scenes , 2002, SIGGRAPH.