Lagrangian Motion Magnification revisited: Continuous, Magnitude Driven Motion Scaling for Psychophysiological Experiments

Video motion magnification forms a relatively novel family of visualization techniques, that aim to magnify imperceivably small motions in videos. The most prominent techniques are based on Eulerian video processing and local phase shifting, which modify pixel time courses, rather than relying on explicit motion estimation.In this work, we show that under ideal conditions in the context of psychophysiological experiments, a Lagrangian motion magnification approach based on dense optical flow estimation, can be superior to Eulerian motion magnification strategies. We present a novel, continuous and motion magnitude driven forward warping scheme of small motions, which implements motion compensation and magnification into a single motion estimation step. Our approach does not rely on temporal filtering and works in the presence of large motion. It does not require the explicit identification of fast moving objects and more generally no segmentation and or matting in the image domain is necessary. We apply our method to the visualization of blinking related modulations in micro-saccadic eye movements ((i.a.. iridodonesis), pupil dilation (hippus) and micro-expression analysis.

[1]  Dani Lischinski,et al.  Gradient Domain High Dynamic Range Compression , 2023 .

[2]  Daniel J. Strauss,et al.  Quantitative motion analysis for non-contact affective and neurocognitive assessment , 2017, 2017 8th International IEEE/EMBS Conference on Neural Engineering (NER).

[3]  Frédo Durand,et al.  Phase-based video motion processing , 2013, ACM Trans. Graph..

[4]  Daniel Cremers,et al.  Structure- and motion-adaptive regularization for high accuracy optic flow , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[5]  Frédo Durand,et al.  Motion magnification , 2005, ACM Trans. Graph..

[6]  Wen-Jing Yan,et al.  How Fast are the Leaked Facial Expressions: The Duration of Micro-Expressions , 2013 .

[7]  Silvia L. Pintea,et al.  Video Acceleration Magnification , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Thomas Brox,et al.  High Accuracy Optical Flow Estimation Based on a Theory for Warping , 2004, ECCV.

[9]  Joost van de Weijer,et al.  Adaptive Structure Tensors and their Applications , 2006, Visualization and Processing of Tensor Fields.

[10]  Frédo Durand,et al.  Riesz pyramids for fast phase-based video magnification , 2014, 2014 IEEE International Conference on Computational Photography (ICCP).

[11]  Frédo Durand,et al.  Eulerian video magnification for revealing subtle changes in the world , 2012, ACM Trans. Graph..

[12]  Frédo Durand,et al.  Video magnification in presence of large motions , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Thomas Brox,et al.  FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).