When Deep Learning Meets Digital Image Correlation

Abstract Convolutional Neural Networks (CNNs) constitute a class of Deep Learning models which have been used in the recent past to resolve many problems in computer vision, in particular optical flow estimation. Measuring displacement and strain fields can be regarded as a particular case of this problem. However, it seems that CNNs have never been used so far to perform such measurements. This work is aimed at implementing a CNN able to retrieve displacement and strain fields from pairs of reference and deformed images of a flat speckled surface, as Digital Image Correlation (DIC) does. This paper explains how a CNN called ‘StrainNet can be developed to reach this goal, and how specific ground truth datasets are elaborated to train this CNN. The main result is that StrainNet successfully performs such measurements, and that it achieves competing results in terms of metrological performance and computing time. The conclusion is that CNNs like StrainNet offer a viable alternative to DIC, especially for real-time applications.

[1]  Anand Asundi,et al.  Two-dimensional digital image correlation for in-plane displacement and strain measurement: a review , 2009 .

[2]  Thomas Brox,et al.  FlowNet: Learning Optical Flow with Convolutional Networks , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[3]  Thomas Brox,et al.  A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Jia Xu,et al.  Accurate Optical Flow via Direct Cost Volume Processing , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Zhenyu Jiang,et al.  High accuracy digital image correlation powered by GPU-based parallel computing , 2015 .

[6]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[7]  Jan Kautz,et al.  PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[8]  Hubert W. Schreier,et al.  Image Correlation for Shape, Motion and Deformation Measurements: Basic Concepts,Theory and Applications , 2009 .

[9]  Richard Szeliski,et al.  A Database and Evaluation Methodology for Optical Flow , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[10]  Didier Stricker,et al.  Flow Fields: Dense Correspondence Fields for Highly Accurate Large Displacement Optical Flow Estimation , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[11]  D. Seidl,et al.  Spatial DIC Errors due to Pattern-Induced Bias and Grey Level Discretization , 2020, Experimental Mechanics.

[12]  Jacob D. Hochhalter,et al.  Increasing accuracy and precision of digital image correlation through pattern optimization , 2017 .

[13]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[14]  Karen O. Egiazarian,et al.  Practical Poissonian-Gaussian Noise Modeling and Fitting for Single-Image Raw-Data , 2008, IEEE Transactions on Image Processing.

[15]  Frédéric Sur,et al.  A Critical Comparison of Some Metrological Parameters Characterizing Local Digital Image Correlation and Grid Method , 2017 .

[16]  Wei-Shi Zheng,et al.  Unsupervised Learning for Optical Flow Estimation Using Pyramid Convolution LSTM , 2019, 2019 IEEE International Conference on Multimedia and Expo (ICME).

[17]  Xiaoou Tang,et al.  LiteFlowNet: A Lightweight Convolutional Neural Network for Optical Flow Estimation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[18]  M. Grédiac,et al.  Towards Criteria Characterizing the Metrological Performance of Full-field Measurement Techniques , 2020 .

[19]  Michael J. Black,et al.  Optical Flow Estimation Using a Spatial Pyramid Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Frédéric Sur,et al.  Rendering Deformed Speckle Images with a Boolean Model , 2017, Journal of Mathematical Imaging and Vision.

[21]  Dorian Garcia,et al.  A speckle texture image generator , 2006, Speckle: International Conference on Speckle Metrology.

[22]  Cordelia Schmid,et al.  DeepFlow: Large Displacement Optical Flow with Deep Matching , 2013, 2013 IEEE International Conference on Computer Vision.

[23]  Iadicola Jones,et al.  A Good Practices Guide for Digital Image Correlation , 2018 .

[24]  Richard B. Lehoucq,et al.  The Effect of the Ill-posed Problem on Quantitative Error Assessment in Digital Image Correlation , 2017 .

[25]  Chao Xu,et al.  Dense motion estimation of particle images via a convolutional neural network , 2019, Experiments in Fluids.

[26]  Yi Yang,et al.  Occlusion Aware Unsupervised Learning of Optical Flow , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[27]  M. Grédiac,et al.  Extracting Displacement and Strain Fields from Checkerboard Images with the Localized Spectrum Analysis , 2018, Experimental Mechanics.

[28]  Hugh A. Bruck,et al.  DIC Challenge: Developing Images and Guidelines for Evaluating Accuracy and Resolution of 2D Analyses , 2018 .

[29]  Ming-Hsuan Yang,et al.  Semi-Supervised Learning for Optical Flow with Generative Adversarial Networks , 2017, NIPS.

[30]  Stepan Vladimirovitch Lomov,et al.  A Self Adaptive Global Digital Image Correlation Algorithm , 2014, Experimental Mechanics.

[31]  Chao Xu,et al.  Particle Image Velocimetry Based on a Deep Learning Motion Estimator , 2020, IEEE Transactions on Instrumentation and Measurement.

[32]  Frédéric Sur,et al.  On Biases in Displacement Estimation for Image Registration, with a Focus on Photomechanics , 2020, Journal of Mathematical Imaging and Vision.

[33]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[34]  Andreas Geiger,et al.  Are we ready for autonomous driving? The KITTI vision benchmark suite , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[35]  Thomas Brox,et al.  FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Stefano Soatto,et al.  Conditional Prior Networks for Optical Flow , 2018, ECCV.

[37]  Frédéric Sur,et al.  Towards deconvolution to enhance the grid method for in-plane strain measurement , 2014 .

[38]  Andreas Geiger,et al.  Object scene flow for autonomous vehicles , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[40]  Frédéric Sur,et al.  A Robust-to-Noise Deconvolution Algorithm to Enhance Displacement and Strain Maps Obtained with Local DIC and LSA , 2018, Experimental Mechanics.

[41]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[42]  M. Sutton,et al.  Systematic errors in digital image correlation due to undermatched subset shape functions , 2002 .

[43]  Gang Wang,et al.  Recurrent Spatial Pyramid CNN for Optical Flow Estimation , 2018, IEEE Transactions on Multimedia.

[44]  Michael J. Black,et al.  Optical Flow in Mostly Rigid Scenes , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[45]  Pierre Slangen,et al.  Speckle06: Speckles, From Grains to Flowers , 2006 .

[46]  Ioannis Patras,et al.  Unsupervised convolutional neural networks for motion estimation , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[47]  Michael J. Black,et al.  A Naturalistic Open Source Movie for Optical Flow Evaluation , 2012, ECCV.

[48]  Benoît Blaysat,et al.  On the Optimal Pattern for Displacement Field Measurement: Random Speckle and DIC, or Checkerboard and LSA? , 2020 .