Seeing the Invisible

The latest techniques in video motion magnification and relevant small motion analysis are surveyed. The main motion magnification techniques are discussed in chronological fashion, highlighting the inherent limitations of predecessor techniques in comparison with subsequent variants. The focus is then shifted to the specific stages within the motion magnification framework to discuss advancements that have been proposed in the literature, namely for spatial decomposition and for emphasizing, representing, and distinguishing different motion signals. The survey concludes with a treatment of different problems in varying application contexts that have benefited from motion magnification and small motion analysis.

[1]  Kwong-Sak Leung,et al.  A Modular Plug-And-Play Sensor System for Urban Air Pollution Monitoring: Design, Implementation and Evaluation , 2017, Sensors.

[2]  Frédo Durand,et al.  Riesz pyramids for fast phase-based video magnification , 2014, 2014 IEEE International Conference on Computational Photography (ICCP).

[3]  Lihi Zelnik-Manor,et al.  Modifying Non-local Variations Across Multiple Views , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[4]  Charles R. Farrar,et al.  Blind identification of full-field vibration modes from video measurements with phase-based video motion magnification , 2017 .

[5]  Hubert Konik,et al.  Subtle Motion Analysis and Spotting using the Riesz Pyramid , 2018, VISIGRAPP.

[6]  Kiran B. Raja,et al.  Video Presentation Attack Detection in Visible Spectrum Iris Recognition Using Magnified Phase Information , 2015, IEEE Transactions on Information Forensics and Security.

[7]  Peter W. McOwan,et al.  Robust velocity computation from a biologically motivated model of motion perception , 1999, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[8]  Jean-Bernard Martens,et al.  The Hermite transform-theory , 1990, IEEE Trans. Acoust. Speech Signal Process..

[9]  Ivan Roselli,et al.  Motion Magnification Analysis for structural monitoring of ancient constructions , 2018, Measurement.

[10]  Manisha Verma,et al.  Edge-Aware Spatial Filtering-Based Motion Magnification , 2017, CVIP.

[11]  Edward H. Adelson,et al.  Shiftable multiscale transforms , 1992, IEEE Trans. Inf. Theory.

[12]  Gamal Fahmy,et al.  A new video magnification technique using complex wavelets with Radon transform application , 2018, Signal, Image and Video Processing.

[13]  Edward H. Adelson,et al.  Motion without movement , 1991, SIGGRAPH.

[14]  Daniel McDuff,et al.  DeepMag , 2018, ACM Trans. Graph..

[15]  Danail Stoyanov,et al.  Higher Order of Motion Magnification for Vessel Localisation in Surgical Video , 2018, MICCAI.

[16]  Frédo Durand,et al.  Visual vibrometry: Estimating material properties from small motions in video , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Kjersti Engan,et al.  Motion based detection of respiration rate in infants using video , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[18]  Zeev Zalevsky,et al.  Simultaneous remote extraction of multiple speech sources and heart beats from secondary speckles pattern. , 2009, Optics express.

[19]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[20]  Hiram Ponce,et al.  Video motion magnification for monitoring of vital signals using a perceptual model , 2017, Symposium on Medical Information Processing and Analysis.

[21]  John See,et al.  Sparsity in Dynamics of Spontaneous Subtle Emotions: Analysis and Application , 2016, IEEE Transactions on Affective Computing.

[22]  Y. J. Tejwani,et al.  Robot vision , 1989, IEEE International Symposium on Circuits and Systems,.

[23]  J. Jensen,et al.  Investigation of transverse oscillation method , 2006, IEEE Transactions on Ultrasonics, Ferroelectrics and Frequency Control.

[24]  Bindang Xue,et al.  Video motion magnification based on Laplacian-Riesz pyramid , 2016, Selected Proceedings from CSOE.

[25]  Terry M. Peters,et al.  Stereoscopic Motion Magnification in Minimally-Invasive Robotic Prostatectomy , 2015, CARE@MICCAI.

[26]  Boris Escalante-Ramírez,et al.  Motion magnification using the Hermite transform , 2015, Symposium on Medical Information Processing and Analysis.

[27]  Hideaki Kimata,et al.  Jerk-Aware Video Acceleration Magnification , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[28]  Eduardo Ulises Moya-Sánchez,et al.  Hilbert and Riesz Transforms Using Atomic Function for Quaternionic Phase Computation , 2013, Advances in Applied Clifford Algebras.

[29]  Jean-Bernard Martens The Hermite transform-applications , 1990, IEEE Trans. Acoust. Speech Signal Process..

[30]  Frédo Durand,et al.  Modal identification of simple structures with high-speed video using motion magnification , 2015 .

[31]  William T. Freeman,et al.  Presented at: 2nd Annual IEEE International Conference on Image , 1995 .

[32]  Julian F. P. Kooij,et al.  Depth-Aware Motion Magnification , 2016, ECCV.

[33]  David J. Fleet,et al.  Computation of component image velocity from local phase information , 1990, International Journal of Computer Vision.

[34]  John See,et al.  A Survey of Automatic Facial Micro-Expression Analysis: Databases, Methods, and Challenges , 2018, Front. Psychol..

[35]  John See,et al.  Eulerian emotion magnification for subtle expression recognition , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[36]  Max Grosse,et al.  Phase-based frame interpolation for video , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Manisha Verma,et al.  Interest Region Based Motion Magnification , 2017, ICIAP.

[38]  Frédo Durand,et al.  Phase-based video motion processing , 2013, ACM Trans. Graph..

[39]  Frédo Durand,et al.  Structural Modal Identification Through High Speed Camera Video: Motion Magnification , 2014 .

[40]  Frédo Durand,et al.  Eulerian video magnification for revealing subtle changes in the world , 2012, ACM Trans. Graph..

[41]  Frédo Durand,et al.  Motion magnification , 2005, ACM Trans. Graph..

[42]  Xuezhi Yang,et al.  PCA-based magnification method for revealing small signals in video , 2018, Signal, Image and Video Processing.

[43]  Silvia L. Pintea,et al.  Video Acceleration Magnification , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[44]  D. Robert Iskander,et al.  Assessing the Feasibility of the Use of Video Motion Magnification for Measuring Microdisplacements , 2017, IEEE Transactions on Instrumentation and Measurement.

[45]  Christopher Niezrecki,et al.  Vibration-Based Damage Detection in Wind Turbine Blades using Phase-Based Motion Estimation and Motion Magnification , 2018, ArXiv.

[46]  Sungsoo Park,et al.  Subtle facial expression recognition using motion magnification , 2009, Pattern Recognit. Lett..

[47]  Andrew Owens,et al.  Visually Indicated Sounds , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[48]  Frédo Durand,et al.  Detecting Pulse from Head Motions in Video , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[49]  Edward H. Adelson,et al.  Steerable filters for early vision, image analysis, and wavelet decomposition , 1990, [1990] Proceedings Third International Conference on Computer Vision.

[50]  Rosalind W. Picard,et al.  Eliminating Physiological Information from Facial Videos , 2017, 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017).

[51]  Eduardo Vázquez-Santacruz,et al.  Quaternion Atomic Phase Magnification for 2D and 3D Motion , 2016, Res. Comput. Sci..

[52]  Manisha Verma,et al.  Saliency Driven Video Motion Magnification , 2017, NCVPRIPG.

[53]  Peter Avitabile,et al.  Feasibility of extracting operating shapes using phase-based motion magnification technique and stereo-photogrammetry , 2017 .

[54]  William T. Freeman,et al.  Revealing and modifying non-local variations in a single image , 2015, ACM Trans. Graph..

[55]  Javaan S. Chahl,et al.  An efficient motion magnification system for real-time applications , 2018, Machine Vision and Applications.

[56]  Frédo Durand,et al.  Video magnification in presence of large motions , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[57]  Frédo Durand,et al.  Eulerian video magnification and analysis , 2016, Commun. ACM.

[58]  Sheng Hong,et al.  Classification of the emotional stress and physical stress using signal magnification and canonical correlation analysis , 2018, Pattern Recognit..

[59]  Maneesh Agrawala,et al.  Selectively de-animating video , 2012, ACM Trans. Graph..

[60]  Gamal Fahmy,et al.  Micro-movement magnification in video signals using complex wavelet analysis , 2017, IET Image Process..

[61]  John See,et al.  Micro-Expression Motion Magnification: Global Lagrangian vs. Local Eulerian Approaches , 2018, 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018).

[62]  Frédo Durand,et al.  Learning-based Video Motion Magnification , 2018, ECCV.

[63]  Hubert Konik,et al.  Micro-Expression Spotting Using the Riesz Pyramid , 2018, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).

[64]  Michael Felsberg,et al.  The monogenic signal , 2001, IEEE Trans. Signal Process..

[65]  Hassen Drira,et al.  Magnifying Subtle Facial Motions for Effective 4D Expression Recognition , 2019, IEEE Transactions on Affective Computing.

[66]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[67]  Frédo Durand,et al.  Motion microscopy for visualizing and quantifying small motions , 2017, Proceedings of the National Academy of Sciences.

[68]  Marc M. Van Hulle,et al.  A phase-based approach to the estimation of the optical flow field using spatial filtering , 2002, IEEE Trans. Neural Networks.

[69]  Daniel J. Strauss,et al.  Lagrangian Motion Magnification revisited: Continuous, Magnitude Driven Motion Scaling for Psychophysiological Experiments , 2018, 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[70]  Jorge Brieva,et al.  Phase-based motion magnification video for monitoring of vital signals using the Hermite transform , 2017, Symposium on Medical Information Processing and Analysis.

[71]  Didier Vray,et al.  Video Magnification Applied in Ultrasound , 2019, IEEE Transactions on Biomedical Engineering.

[72]  Frédo Durand,et al.  The visual microphone , 2014, ACM Trans. Graph..

[73]  Zhao Yang,et al.  Amplitude-Based Filtering for Video Magnification in Presence of Large Motion , 2018, Sensors.

[74]  Frédo Durand,et al.  Deviation magnification , 2015, ACM Trans. Graph..

[75]  Steven S. Beauchemin,et al.  The computation of optical flow , 1995, CSUR.