Projector and Camera Shade Detection Based on Stripe Pattern Sequence for High Quality Structured Light Depth Imaging

Structured light depth imaging offers a high precision of 3D measurement for applications to industrial modeling, inspection, reproduction and archiving. Despite recent progress, however, structured light depth imaging has yet to settle the trade-off between the number of captured 3D points and that of unwanted outliers, in particular, when a low reflectance surface is mixed with projector and camera shades. This paper presents an approach to solving such trade-off by accurately mapping projector and camera shades under the presence of a low reflectance surface. Unlike conventional approaches relying mostly on pattern intensity for detecting shades, the proposed approach first identifies stripe boundaries legitimate in terms of the projected pattern sequence. Based on the list of legitimate boundaries identified, then, the gaps in pixel and boundary addresses are analyzed to define the respective projector and camera shades, possibly, in the presence of address juxtaposition. Experimental results show that the proposed approach shows superior performance in shade detection as well as outlier removal compared to the state-of-the-art approaches, maximizing the number of captured 3D points while minimizing that of unwanted outliers.

[1]  Zhaoyang Wang,et al.  Real-time, high-accuracy 3D imaging and shape measurement. , 2015, Applied optics.

[2]  Zhenzhou Wang,et al.  An Imaging and Measurement System for Robust Reconstruction of Weld Pool During Arc Welding , 2015, IEEE Transactions on Industrial Electronics.

[3]  Sabine Süsstrunk,et al.  Automatic and Accurate Shadow Detection from (potentially) a Single Image Using Near-infrared Information , 2022 .

[4]  William H. Richardson,et al.  Bayesian-Based Iterative Method of Image Restoration , 1972 .

[5]  Giovanna Sansoni,et al.  OPL-3D: A novel, portable optical digitizer for fast acquisition of free-form surfaces , 2003 .

[6]  Avinash C. Kak,et al.  Dual-beam structured-light scanning for 3-D object modeling , 2001, Proceedings Third International Conference on 3-D Digital Imaging and Modeling.

[7]  Sukhan Lee,et al.  FPGA Based Adaptive Rate and Manifold Pattern Projection for Structured Light 3D Camera System † , 2018, Sensors.

[8]  Georg Wiora High-resolution measurement of phase-shift amplitude and numeric object phase calculation , 2000, SPIE Optics + Photonics.

[9]  Song Zhang,et al.  High-resolution, real-time 3D absolute coordinate measurement based on a phase-shifting method. , 2006, Optics express.

[10]  Jens Guehring,et al.  Dense 3D surface acquisition by structured light using off-the-shelf components , 2000, IS&T/SPIE Electronic Imaging.

[11]  Hans-Peter Seidel,et al.  Modulated phase-shifting for 3D scanning , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Sukhan Lee,et al.  A real-time 3D IR camera based on hierarchical orthogonal coding , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[13]  Shree K. Nayar,et al.  Diffuse structured light , 2012, 2012 IEEE International Conference on Computational Photography (ICCP).

[14]  Christopher Schwartz,et al.  A Multi-camera, Multi-projector Super-Resolution Framework for Structured Light , 2011, 2011 International Conference on 3D Imaging, Modeling, Processing, Visualization and Transmission.

[15]  Youfu Li,et al.  Performance analysis of 3-D shape measurement algorithm with a short baseline projector-camera system , 2014, Robotics and biomimetics.

[16]  Ronald Chung,et al.  An Accurate and Robust Strip-Edge-Based Structured Light Means for Shiny Surface Micromeasurement in 3-D , 2013, IEEE Transactions on Industrial Electronics.

[17]  Feng Xiao,et al.  Illuminating Illumination , 2001, CIC.

[18]  Mark S. Drew,et al.  Detecting Illumination in Images , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[19]  Lam Quang Bui,et al.  Accurate estimation of the boundaries of a structured light pattern. , 2011, Journal of the Optical Society of America. A, Optics, image science, and vision.

[20]  Song Zhang,et al.  Ultrafast 3-D shape measurement with an off-the-shelf DLP projector. , 2010, Optics express.

[21]  Jeffrey L. Posdamer,et al.  Surface measurement by space-encoded projected beam systems , 1982, Comput. Graph. Image Process..

[22]  Sukhan Lee,et al.  A method of eliminating interreflection in 3D reconstruction using structured light 3D camera , 2014, 2014 International Conference on Computer Vision Theory and Applications (VISAPP).

[23]  Dongliang Zheng,et al.  Self-correction phase unwrapping method based on Gray-code light , 2012 .

[24]  Derek Hoiem,et al.  Single-image shadow detection and removal using paired regions , 2011, CVPR 2011.

[25]  L. Lucy An iterative technique for the rectification of observed distributions , 1974 .

[26]  Hans J. Tiziani,et al.  New highly efficient binary codes for structured light methods , 1996, Other Conferences.

[27]  José M. Bravo,et al.  Application of Structured Light System Technique for Authentication of Wooden Panel Paintings , 2016, Sensors.

[28]  Lam Quang Bui,et al.  Boundary Inheritance Codec for high-accuracy structured light three-dimensional reconstruction with comparative performance evaluation. , 2013, Applied optics.

[29]  Zhongwei Li,et al.  Enhanced phase measurement profilometry for industrial 3D inspection automation , 2015 .

[30]  Sukhan Lee,et al.  FPGA Based Pattern Generation and Synchonization for High Speed Structured Light 3D Camera , 2017 .

[31]  Song Zhang,et al.  High-speed 3D imaging using digital binary defocusing method vs sinusoidal method , 2017, OPTO.

[32]  François Blais Review of 20 years of range sensor development , 2004, J. Electronic Imaging.

[33]  François Berry,et al.  A robotic structured light camera , 2014, 2014 IEEE/ASME International Conference on Advanced Intelligent Mechatronics.

[34]  Laurence G. Hassebrook,et al.  Multicamera phase measuring profilometry for accurate depth measurement , 2007, SPIE Defense + Commercial Sensing.

[35]  Hong-Tzong Yau,et al.  Comparison of 3-D Printing and 5-axis Milling for the Production of Dental e-models from Intra-oral Scanning , 2016 .

[36]  Sukhan Lee,et al.  Outlier removal based on boundary order and shade information in structured light 3D camera , 2015, 2015 IEEE 7th International Conference on Cybernetics and Intelligent Systems (CIS) and IEEE Conference on Robotics, Automation and Mechatronics (RAM).

[37]  Joaquim Salvi,et al.  A state of the art in structured light patterns for surface profilometry , 2010, Pattern Recognit..

[38]  Zhenzhou Wang,et al.  Three-dimensional surface imaging by multi-frequency phase shift profilometry with angle and pattern modeling for system calibration , 2016 .

[39]  Sukhan Lee,et al.  Boundary based shade detection , 2016, 2016 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI).

[40]  Sukhan Lee,et al.  Signal Separation Coding for Robust Depth Imaging Based on Structured Light , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[41]  Jason Geng,et al.  Review of 3-D Endoscopic Surface Imaging Techniques , 2014, IEEE Sensors Journal.