Calibration of fringe projection profilometry: A comparative review

Abstract Fringe projection profilometry is widely used for three-dimensional (3D) measurements. System calibration is crucial to fringe projection systems, since it determines how the phase is converted into 3D geometry. Many methods and studies on calibration have been presented in the last few years. However, a comparative overview on the accuracy and the implementation details is still lacking. In this work, we present a review of some of the most used calibration methods. First, the basic principles are reviewed. Then, both quantitative and qualitative comparative experiments are implemented to show the advantages and constraints of each method. Further, a number of factors that may affect the performance of the calibration are discussed. Sample codes are available for better comprehension and a quick start of these methods.

[1]  Zhaoyang Wang,et al.  Hyper-accurate flexible calibration technique for fringe-projection-based three-dimensional imaging , 2012 .

[2]  Shijie Feng,et al.  High-speed three-dimensional shape measurement for dynamic scenes using bi-frequency tripolar pulse-width-modulation fringe projection , 2013 .

[3]  Haitao He,et al.  Gamma correction for digital fringe projection profilometry. , 2004, Applied optics.

[4]  Francisco J. Cuevas,et al.  Multi-layer neural network applied to phase and depth recovery from fringe patterns , 2000 .

[5]  Feng Gao,et al.  A simple, flexible and automatic 3D calibration method for a phase calculation-based fringe projection imaging system. , 2013, Optics express.

[6]  Xu Zhang,et al.  Projector calibration from the camera image point of view , 2009 .

[7]  Hongzhi Jiang,et al.  High dynamic range fringe acquisition: A novel 3-D scanning technique for high-reflective surfaces , 2012 .

[8]  Sebastian Thrun,et al.  3D shape scanning with a time-of-flight camera , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[9]  Yunhui Yan,et al.  Fast 3D shape measurement using Fourier transform profilometry without phase unwrapping , 2016 .

[10]  Ming Tang,et al.  A flexible multi-view calibration and 3D measurement method based on digital fringe projection , 2019, Measurement Science and Technology.

[11]  Phj Piet Schellekens,et al.  Design of a High-Precision 3D-Coordinate Measuring Machine , 1998 .

[12]  Peisen S. Huang,et al.  Novel method for structured light system calibration , 2006 .

[13]  Kehar Singh,et al.  Profilometry for the measurement of three-dimensional object shape using radial basis function, and multi-layer perceptron neural networks , 2002 .

[14]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Zonghua Zhang,et al.  Uneven fringe projection for efficient calibration in high-resolution 3D shape metrology. , 2007, Applied optics.

[16]  Qian Chen,et al.  Robust and efficient multi-frequency temporal phase unwrapping: optimal fringe frequency and pattern sequence selection. , 2017, Optics express.

[17]  Xiaohui Zhou,et al.  Multiview phase shifting: a full-resolution and high-speed 3D measurement framework for arbitrary shape dynamic objects. , 2013, Optics letters.

[18]  Qian Luo,et al.  Fringe projection profilometry based on a novel phase shift method. , 2011, Optics express.

[19]  Qiang Sun,et al.  Projector distortion correction in 3D shape measurement using a structured-light system by deep neural networks , 2020 .

[20]  Li Zhang,et al.  Rapid shape acquisition using color structured light and multi-pass dynamic programming , 2002, Proceedings. First International Symposium on 3D Data Processing Visualization and Transmission.

[21]  Qingjin Peng,et al.  A unified-calibration method in FTP-based 3D data acquisition for reverse engineering , 2007 .

[22]  Song Zhang,et al.  Phase error compensation for three-dimensional shape measurement with projector defocusing. , 2011, Applied optics.

[23]  Zhiwei Xiong,et al.  Computational Depth Sensing : Toward high-performance commodity depth cameras , 2017, IEEE Signal Processing Magazine.

[24]  J. A. Quiroga,et al.  Flexible calibration procedure for fringe projection profilometry , 2007 .

[25]  A. Ozcan,et al.  Deep learning in holography and coherent imaging , 2019, Light: Science & Applications.

[26]  Changku Sun,et al.  An Accurate Projector Calibration Method Based on Polynomial Distortion Representation , 2015, Sensors.

[27]  Victor H. Diaz-Ramirez,et al.  Flexible camera-projector calibration using superposed color checkerboards , 2019 .

[28]  Zhaoyang Wang,et al.  Generic gamma correction for accuracy enhancement in fringe-projection profilometry. , 2010, Optics letters.

[29]  Hongwei Guo,et al.  Iterative calibration method for measurement system having lens distortions in fringe projection profilometry. , 2020, Optics express.

[30]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[31]  Lili Xie,et al.  Accurate projector calibration method by using an optical coaxial camera. , 2015, Applied optics.

[32]  T. Hewett,et al.  Reliability of landing 3D motion analysis: implications for longitudinal analyses. , 2007, Medicine and science in sports and exercise.

[33]  Heming Huang,et al.  Flexible calibration method for telecentric fringe projection profilometry systems. , 2016, Optics express.

[34]  Qian Chen,et al.  Robust dynamic 3-D measurements with motion-compensated phase-shifting profilometry , 2018 .

[35]  Jingang Zhong,et al.  Spatial carrier-fringe pattern analysis by means of wavelet transform: wavelet transform profilometry. , 2004, Applied optics.

[36]  Qian Chen,et al.  Deep-learning-enabled geometric constraints and phase unwrapping for single-shot absolute 3D shape measurement , 2020 .

[37]  Tong Guo,et al.  Simple calibration of a phase-based 3D imaging system based on uneven fringe projection. , 2011, Optics letters.

[38]  Zhaoyang Wang,et al.  Flexible calibration technique for fringe-projection-based three-dimensional imaging. , 2010, Optics letters.

[39]  Qian Chen,et al.  A new microscopic telecentric stereo vision system - Calibration, rectification, and three-dimensional reconstruction , 2019, Optics and Lasers in Engineering.

[40]  Anand Asundi,et al.  Comparison of Fourier transform, windowed Fourier transform, and wavelet transform methods for phase extraction from a single fringe pattern in fringe projection profilometry , 2010 .

[41]  Qi Hao,et al.  Dual-frequency pattern scheme for high-speed 3-D shape measurement. , 2010, Optics express.

[43]  Andreas Tünnermann,et al.  5D hyperspectral imaging: fast and accurate measurement of surface shape and spectral characteristics using structured light. , 2018, Optics express.

[44]  Bin Liu,et al.  Accurate and robust calibration method based on pattern geometric constraints for fringe projection profilometry. , 2017, Applied optics.

[45]  Xiang Peng,et al.  Strategy for automatic and complete three-dimensional optical digitization. , 2012, Optics letters.

[46]  Qian Chen,et al.  Graphics processing unit-assisted real-time three-dimensional measurement using speckle-embedded fringe. , 2015, Applied optics.

[47]  Jinhui Huang,et al.  A new reconstruction method based on fringe projection of three-dimensional measuring system , 2014 .

[48]  Chao Zuo,et al.  Automatic identification and removal of outliers for high-speed fringe projection profilometry , 2013 .

[49]  Jason Geng,et al.  Structured-light 3D surface imaging: a tutorial , 2011 .

[50]  X. Su,et al.  High-speed optical measurement for the drumhead vibration. , 2005, Optics express.

[51]  Cyril Breque,et al.  Calibration of a structured-light projection system: Development to large dimension objects , 2012 .

[52]  J. Ziegert,et al.  In-Situ Metrology System for Laser Powder Bed Fusion Additive Process , 2015 .

[53]  Mumin Song,et al.  Overview of three-dimensional shape measurement using optical methods , 2000 .

[54]  Song Zhang,et al.  High dynamic range scanning technique , 2009 .

[55]  Wei-Hung Su,et al.  Calibration-based two-frequency projected fringe profilometry: a robust, accurate, and single-shot measurement for objects with large depth discontinuities. , 2006, Optics express.

[56]  Song Zhang,et al.  Superfast multifrequency phase-shifting technique with optimal pulse width modulation. , 2011, Optics express.

[57]  Xiang Peng,et al.  Distortion correction for microscopic fringe projection system with Scheimpflug telecentric lens. , 2015, Applied optics.

[58]  K. Reichard,et al.  Calibration-based phase-shifting projected fringe profilometry for accurate absolute 3D surface profile measurement , 2003 .

[59]  Lei Jia,et al.  Flexible calibration method of an FPP system based on a geometrical model and NLSM with fewer parameters. , 2018, Applied optics.

[60]  Shuai Li,et al.  Lensless computational imaging through deep learning , 2017, ArXiv.

[61]  Zonghua Zhang,et al.  Review of single-shot 3D shape measurement by phase calculation-based fringe projection techniques , 2012 .

[62]  Qican Zhang,et al.  Flexible camera calibration using not-measured imperfect target. , 2013, Applied optics.

[63]  Lei Huang,et al.  Temporal phase unwrapping algorithms for fringe projection profilometry: A comparative review , 2016 .

[64]  Liang-Chia Chen,et al.  Calibration of 3D surface profilometry using digital fringe projection , 2005 .

[65]  Xiang Peng,et al.  Ray calibration and phase mapping for structured-light-field 3D reconstruction. , 2018, Optics express.

[66]  Nikolaos Doulamis,et al.  Deep Learning for Computer Vision: A Brief Review , 2018, Comput. Intell. Neurosci..

[67]  Qian Chen,et al.  Phase shifting algorithms for fringe projection profilometry: A review , 2018, Optics and Lasers in Engineering.

[68]  Anand Asundi,et al.  Temporal phase unwrapping using deep learning , 2019, Scientific Reports.

[69]  Roger Y. Tsai,et al.  A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses , 1987, IEEE J. Robotics Autom..

[70]  Miao Liu,et al.  Generic precise augmented reality guiding system and its calibration method based on 3D virtual model. , 2016, Optics express.

[71]  Chao Chen,et al.  Simple calibration method for dual-camera structured light system , 2018 .

[72]  Guohua Gu,et al.  Micro deep learning profilometry for high-speed 3D surface imaging , 2019, Optics and Lasers in Engineering.

[73]  Rihong Zhu,et al.  A fast and accurate gamma correction based on Fourier spectrum analysis for digital fringe projection profilometry , 2012 .

[74]  Zhenkun Lei,et al.  Multi-frequency inverse-phase fringe projection profilometry for nonlinear phase error compensation , 2015 .

[75]  Yueyang Li,et al.  High-speed and high-efficiency three-dimensional shape measurement based on Gray-coded light , 2020, Photonics Research.

[76]  David G Lidzey,et al.  A hybrid organic–inorganic polariton LED , 2019, Light: Science & Applications.

[77]  W Li,et al.  Large-scale three-dimensional object measurement: a practical coordinate mapping and image data-patching method. , 2001, Applied optics.

[78]  Jiangtao Xi,et al.  Accurate projector calibration based on a new point-to-point mapping relationship between the camera and projector images , 2015 .

[79]  Qican Zhang,et al.  Quality-guided phase unwrapping technique: comparison of quality maps and guiding strategies. , 2011, Applied optics.

[80]  Qianbing Zhang,et al.  High-Speed Photography and Digital Optical Measurement Techniques for Geomaterials: Fundamentals and Applications , 2017, Rock Mechanics and Rock Engineering.

[81]  Qian Chen,et al.  High-speed 3D shape measurement using the optimized composite fringe patterns and stereo-assisted structured light system. , 2019, Optics express.

[82]  Xiang Peng,et al.  Phase-3D mapping method developed from back-projection stereovision model for fringe projection profilometry. , 2017, Optics express.

[83]  Qian Chen,et al.  A carrier removal technique for Fourier transform profilometry based on principal component analysis , 2015 .

[84]  M. Takeda,et al.  Fourier transform profilometry for the automatic measurement of 3-D object shapes. , 1983, Applied optics.

[85]  Wei Wang,et al.  Least-squares calibration method for fringe projection profilometry with some practical considerations , 2013 .

[86]  Beiwen Li,et al.  Microscopic structured light 3D profilometry: Binary defocusing technique vs. sinusoidal fringe projection , 2017 .

[87]  Qian Chen,et al.  Real-time 3-D shape measurement with composite phase-shifting fringes and multi-view system. , 2016, Optics express.

[88]  Shijie Feng,et al.  General solution for high dynamic range three-dimensional shape measurement using the fringe projection technique , 2014 .

[89]  Alfredo Paolillo,et al.  A New Calibration Procedure for 3-D Shape Measurement System Based on Phase-Shifting Projected Fringe Profilometry , 2009, IEEE Transactions on Instrumentation and Measurement.

[90]  Ronald Chung,et al.  Use of LCD Panel for Calibrating Structured-Light-Based Range Sensing System , 2008, IEEE Transactions on Instrumentation and Measurement.

[91]  Wenbo Guo,et al.  A fast reconstruction method for three-dimensional shape measurement using dual-frequency grating projection and phase-to-height lookup table , 2019, Optics & Laser Technology.

[92]  Yiannis Kompatsiaris,et al.  Deep Learning Advances in Computer Vision with 3D Data , 2017, ACM Comput. Surv..

[93]  Jiangtao Xi,et al.  New approach to improve the accuracy of 3-D shape measurement of moving object using phase shifting profilometry. , 2013, Optics express.

[94]  Zhang Liang,et al.  High-speed high dynamic range 3D shape measurement based on deep learning , 2020 .

[95]  Song Zhang,et al.  Flexible 3-D shape measurement using projector defocusing. , 2009, Optics letters.

[96]  Emanuele Zappa,et al.  Innovative calibration technique for fringe projection based 3D scanner , 2011 .

[97]  Yingjie Yu,et al.  Least-squares calibration method for fringe projection profilometry , 2005 .

[98]  Yang Xu,et al.  Calibration for stereo vision system based on phase matching and bundle adjustment algorithm , 2015 .

[99]  Qi Hao,et al.  Robust Active Stereo Vision Using Kullback-Leibler Divergence , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[100]  Anand Asundi,et al.  Phase error analysis and compensation for nonsinusoidal waveforms in phase-shifting digital fringe projection profilometry. , 2009, Optics letters.

[101]  Laiq Hasan,et al.  Projector Calibration for Pattern Projection Systems , 2014 .

[102]  Chenggen Quan,et al.  Fringe projection profilometry with nonparallel illumination: a least-squares approach. , 2005, Optics letters.

[103]  Werner Jüptner,et al.  Accurate procedure for the calibration of a structured light system , 2004 .

[104]  J. Kofman,et al.  Comparison of linear and nonlinear calibration methods for phase-measuring profilometry , 2007 .

[105]  Jae-Sang Hyun,et al.  High-speed and high-accuracy 3D surface measurement using a mechanical projector. , 2018, Optics express.

[106]  Qican Zhang,et al.  Dynamic 3-D shape measurement method: A review , 2010 .

[107]  Pedro Arias,et al.  Metrological comparison between Kinect I and Kinect II sensors , 2015 .

[108]  Xiang Peng,et al.  Automatic and rapid whole-body 3D shape measurement based on multinode 3D sensing and speckle projection. , 2017, Applied optics.

[109]  Hua Du,et al.  Out-of-plane shape determination in generalized fringe projection profilometry. , 2006, Optics express.

[110]  Hao Jiang,et al.  Calibration of fringe projection profilometry using an inaccurate 2D reference target , 2017 .

[111]  A Asundi,et al.  Unified calibration technique and its applications in optical triangular profilometry. , 1999, Applied optics.

[112]  Shijie Feng,et al.  High-speed real-time 3-D coordinates measurement based on fringe projection profilometry considering camera lens distortion , 2014 .

[113]  Ken Chen,et al.  A simple calibration method for structured light-based 3D profile measurement , 2013 .

[114]  Janne Heikkilä,et al.  A four-step camera calibration procedure with implicit image correction , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[115]  Song Zhang,et al.  Three-dimensional shape measurement using a structured light system with dual projectors. , 2018, Applied optics.

[116]  D. Scharstein,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, Proceedings IEEE Workshop on Stereo and Multi-Baseline Vision (SMBV 2001).

[117]  Zhongwei Li,et al.  Accurate calibration method for a structured light system , 2008 .

[118]  Xianyu Su,et al.  Reliability-guided phase unwrapping algorithm: a review ☆ , 2004 .

[119]  Qican Zhang,et al.  A carrier removal method in Fourier transform profilometry with Zernike polynomials , 2013 .

[120]  Ye Jin,et al.  Accurate calibration for a camera-projector measurement system based on structured light projection , 2009 .

[121]  Qi Xue,et al.  Calibration of a camera–projector measurement system and error impact analysis , 2012 .

[122]  Liang Zhang,et al.  Fringe pattern analysis using deep learning , 2018, Advanced Photonics.

[123]  Eann A. Patterson,et al.  Location and shape measurement using a portable fringe projection system , 2005 .

[124]  Xiang Peng,et al.  Self-calibration of a fringe projection system using epipolar constraint , 2008 .

[125]  Cengizhan Ozturk,et al.  Structured Light Using Pseudorandom Codes , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[126]  Euripides G. M. Petrakis,et al.  A survey on industrial vision systems, applications, tools , 2003, Image Vis. Comput..

[127]  Xianyu Su,et al.  A direct mapping algorithm for phase-measuring profilometry , 1994 .

[128]  Chia-Yen Chen,et al.  An Incremental Target-Adapted Strategy for Active Geometric Calibration of Projector-Camera Systems , 2013, Sensors.

[129]  Liang-Chia Chen,et al.  Miniaturized 3D surface profilometer using digital fringe projection , 2005 .

[130]  Liandong Yu,et al.  Sub-pixel projector calibration method for fringe projection profilometry. , 2017, Optics express.

[131]  Zonghua Zhang,et al.  Generic exponential fringe model for alleviating phase error in phase measuring profilometry , 2018, Optics and Lasers in Engineering.

[132]  Qian Chen,et al.  High-resolution real-time 360° 3D model reconstruction of a handheld object with fringe projection profilometry. , 2019, Optics letters.

[133]  Xiang Peng,et al.  Structured light field 3D imaging. , 2016, Optics express.

[134]  Malgorzata Kujawinska,et al.  Digital fringe projection system for large-volume 360-deg shape measurement , 2002 .

[135]  Yibo Zhang,et al.  Deep Learning Microscopy , 2017, ArXiv.

[136]  Zhoujie Wu,et al.  High-speed three-dimensional shape measurement based on cyclic complementary Gray-code light. , 2019, Optics express.

[137]  Gunther Notni,et al.  Digital fringe projection in 3D shape measurement: an error analysis , 2003, SPIE Optical Metrology.

[138]  Rong Mo,et al.  Flexible calibration of phase-to-height conversion in fringe projection profilometry. , 2016, Applied optics.

[139]  A. Tünnermann,et al.  High-speed three-dimensional shape measurement using GOBO projection , 2016 .

[140]  Zhaoyang Wang,et al.  Three-dimensional shape measurement with an arbitrarily arranged fringe projection profilometry system. , 2007, Optics letters.

[141]  Sam Van der Jeught,et al.  Real-time structured light profilometry: a review , 2016 .

[142]  Giovanna Sansoni,et al.  Calibration and performance evaluation of a 3-D imaging sensor based on the projection of structured light , 2000, IEEE Trans. Instrum. Meas..

[143]  Wenjing Chen,et al.  Improved Fourier-transform profilometry. , 2007, Applied optics.

[144]  Reinhold Ritter,et al.  3-D shape measurement of complex objects by combining photogrammetry and fringe projection , 2000 .

[145]  Lenny A. Romero,et al.  Hybrid calibration procedure for fringe projection profilometry based on stereo-vision and polynomial fitting , 2020, Applied optics.

[146]  Lei Huang,et al.  Least-squares calibration method for fringe projection profilometry considering camera lens distortion. , 2010, Applied optics.

[147]  Kuang Peng,et al.  A new pixel matching method using the modulation of shadow areas in online 3D measurement , 2013 .

[148]  Gabriel Taubin,et al.  Simple, Accurate, and Robust Projector-Camera Calibration , 2012, 2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission.

[149]  Zhanyi Hu,et al.  Flexible method for structured light system calibration , 2008 .

[150]  Anand Asundi,et al.  Two-dimensional digital image correlation for in-plane displacement and strain measurement: a review , 2009 .

[151]  Anand Asundi,et al.  Fringe pattern denoising based on deep learning , 2019, Optics Communications.

[152]  Zhang Liang,et al.  High dynamic range 3D measurements with fringe projection profilometry: a review , 2018 .

[153]  Ken Chen,et al.  Accurate calibration method for camera and projector in fringe patterns measurement system. , 2016, Applied optics.

[154]  Yingchun Wu,et al.  Improved algorithm for phase-to-height mapping in phase measuring profilometry. , 2012, Applied optics.

[155]  Shijie Feng,et al.  Single-shot absolute 3D shape measurement with deep-learning-based color fringe projection profilometry. , 2020, Optics letters.

[156]  Zhongwei Li,et al.  Complex object 3D measurement based on phase-shifting and a neural network , 2009 .

[157]  Xiaoli Liu,et al.  Calibration strategy of optical measurement network for large-scale and shell-like objects , 2012 .

[158]  Shijie Feng,et al.  Microscopic fringe projection profilometry: A review , 2020 .

[159]  N. Hurtós,et al.  Plane-based calibration of a projector-camera system , 2008 .

[160]  Beiwen Li,et al.  Motion-induced error reduction by combining Fourier transform profilometry with phase-shifting profilometry. , 2016, Optics express.

[161]  Bo Li,et al.  Flexible structured-light-based three-dimensional profile reconstruction method considering lens projection-imaging distortion. , 2012, Applied optics.

[162]  Wolfgang Schreiber,et al.  Theory and arrangements of self-calibrating whole-body 3-D-measurement systems using fringe projection technique , 2000 .

[163]  Dong Li,et al.  Adaptive fringe-pattern projection for image saturation avoidance in 3D surface-shape measurement. , 2014, Optics express.

[164]  Qian Kemao,et al.  Two-dimensional windowed Fourier transform for fringe pattern analysis: Principles, applications and implementations , 2007 .

[165]  Yan Zhao,et al.  3D fingerprint imaging system based on full-field fringe projection profilometry , 2014 .

[166]  Anand Asundi,et al.  Review of phase measuring deflectometry , 2018, Optics and Lasers in Engineering.

[167]  Qican Zhang,et al.  Universal calculation formula and calibration method in Fourier transform profilometry. , 2010, Applied optics.

[168]  S. Sitharama Iyengar,et al.  Data-Driven Techniques in Disaster Information Management , 2017, ACM Comput. Surv..

[169]  Qian Chen,et al.  High-speed three-dimensional profilometry for multiple objects with complex shapes. , 2012, Optics express.

[170]  A. Ozcan,et al.  On the use of deep learning for computational imaging , 2019, NanoScience + Engineering.

[171]  Jesús Villa,et al.  Transformation of phase to (x,y,z)-coordinates for the calibration of a fringe projection profilometer , 2012 .

[172]  Dongliang Zheng,et al.  Phase coding method for absolute phase retrieval with a large number of codewords. , 2012, Optics express.

[173]  Ken Chen,et al.  A self-recalibration method based on scale-invariant registration for structured light measurement systems , 2017 .

[174]  Qian Chen,et al.  Motion-artifact-free dynamic 3D shape measurement with hybrid Fourier-transform phase-shifting profilometry. , 2019, Optics express.

[175]  Anand K. Asundi,et al.  Micro Fourier Transform Profilometry (μFTP): 3D shape measurement at 10, 000 frames per second , 2017, ArXiv.

[176]  Xiubao Sui,et al.  Optimized pulse width modulation pattern strategy for three-dimensional profilometry with projector defocusing. , 2012, Applied optics.