Video Acceleration Magnification

The ability to amplify or reduce subtle image changes over time is useful in contexts such as video editing, medical video analysis, product quality control and sports. In these contexts there is often large motion present which severely distorts current video amplification methods that magnify change linearly. In this work we propose a method to cope with large motions while still magnifying small changes. We make the following two observations: i) large motions are linear on the temporal scale of the small changes, ii) small changes deviate from this linearity. We ignore linear motion and propose to magnify acceleration. Our method is pure Eulerian and does not require any optical flow, temporal alignment or region annotations. We link temporal second-order derivative filtering to spatial acceleration magnification. We apply our method to moving objects where we show motion magnification and color magnification. We provide quantitative as well as qualitative evidence for our method while comparing to the state-of-the-art.

[1]  Edward H. Adelson,et al.  The Design and Use of Steerable Filters , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Andrea J. van Doorn,et al.  Generic Neighborhood Operators , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  William T. Freeman,et al.  Presented at: 2nd Annual IEEE International Conference on Image , 1995 .

[4]  Tony Lindeberg,et al.  Scale-Space Theory in Computer Vision , 1993, Lecture Notes in Computer Science.

[5]  C. Schmid,et al.  Indexing based on scale invariant interest points , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[6]  David J. Fleet,et al.  Computation of component image velocity from local phase information , 1990, International Journal of Computer Vision.

[7]  Eero P. Simoncelli,et al.  A Parametric Texture Model Based on Joint Statistics of Complex Wavelet Coefficients , 2000, International Journal of Computer Vision.

[8]  J. Koenderink The structure of images , 2004, Biological Cybernetics.

[9]  A. Torralba,et al.  Motion magnification , 2005, SIGGRAPH 2005.

[10]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[11]  Frédo Durand,et al.  Eulerian video magnification for revealing subtle changes in the world , 2012, ACM Trans. Graph..

[12]  Maneesh Agrawala,et al.  Selectively de-animating video , 2012, ACM Trans. Graph..

[13]  Frédo Durand,et al.  Detecting Pulse from Head Motions in Video , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Frédo Durand,et al.  Phase-based video motion processing , 2013, ACM Trans. Graph..

[15]  Frédo Durand,et al.  Revealing Invisible Changes in the World , 2013 .

[16]  Frédo Durand,et al.  The visual microphone , 2014, ACM Trans. Graph..

[17]  Frédo Durand,et al.  Riesz pyramids for fast phase-based video magnification , 2014, 2014 IEEE International Conference on Computational Photography (ICCP).

[18]  Frédo Durand,et al.  Video magnification in presence of large motions , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Isao Yamada,et al.  Algebraic phase unwrapping along the real axis: extensions and stabilizations , 2015, Multidimens. Syst. Signal Process..

[20]  Frédo Durand,et al.  Modal identification of simple structures with high-speed video using motion magnification , 2015 .

[21]  Max Grosse,et al.  Phase-based frame interpolation for video , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Thomas Brox,et al.  FlowNet: Learning Optical Flow with Convolutional Networks , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[23]  Cordelia Schmid,et al.  EpicFlow: Edge-preserving interpolation of correspondences for optical flow , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Julian F. P. Kooij,et al.  Depth-Aware Motion Magnification , 2016, ECCV.

[25]  Markus H. Gross,et al.  Phase-Based Modification Transfer for Video , 2016, ECCV.

[26]  Luc Van Gool,et al.  Fast Optical Flow Using Dense Inverse Search , 2016, ECCV.

[27]  Nicu Sebe,et al.  Self-Adaptive Matrix Completion for Heart Rate Estimation from Face Videos under Realistic Conditions , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Gorjan Alagic,et al.  #p , 2019, Quantum information & computation.

[29]  Frédo Durand,et al.  Visual vibrometry: Estimating material properties from small motions in video , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).