Robust depth sensing with adaptive structured light illumination

Automatic focus and exposure are the key components in digital cameras nowadays, which jointly play an essential role for capturing a high quality image/video. In this paper, we make an attempt to address these two challenging issues for future depth cameras. Relying on a programmable projector, we establish a structured light system for depth sensing with focus and exposure adaptation. The basic idea is to change current illumination pattern and intensity locally according to the prior depth information. Consequently, multiple object surfaces appearing at different depths in the scene can receive proper illumination respectively. In this way, more flexible and robust depth sensing can be achieved in comparison with fixed illumination, especially at near depth.

[1]  Frédo Durand,et al.  Image and depth from a conventional camera with a coded aperture , 2007, SIGGRAPH 2007.

[2]  Joaquim Salvi,et al.  Pattern codification strategies in structured light systems , 2004, Pattern Recognit..

[3]  Stefano Soatto,et al.  A geometric approach to shape from defocus , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Joseph Shamir,et al.  Range Imaging With Adaptive Color Structured Light , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Richard Szeliski,et al.  High-accuracy stereo depth maps using structured light , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[6]  Walter M. Duncan,et al.  Emerging digital micromirror device (DMD) applications , 2003, SPIE MOEMS-MEMS.

[7]  Ashok Veeraraghavan,et al.  Structured light 3D scanning in the presence of global illumination , 2011, CVPR 2011.

[8]  Luc Van Gool,et al.  Real-time range acquisition by adaptive structured light , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Jake K. Aggarwal,et al.  Structure from stereo-a review , 1989, IEEE Trans. Syst. Man Cybern..

[10]  André Oosterlinck,et al.  Range Image Acquisition with a Single Binary-Encoded Light Pattern , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Shree K. Nayar,et al.  Programmable imaging using a digital micromirror array , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[12]  Berthold K. P. Horn.  Height and gradient from shading , 1989, International Journal of Computer Vision.

[13]  Hideshi Yamada,et al.  Rendering for an interactive 360° light field display , 2007, ACM Trans. Graph..

[14]  周炯,et al.  Kind of static method and system for object reconstruction , 2014 .

[15]  S. Inokuchi,et al.  Range-imaging system for 3-D object recognition , 1984 .

[16]  Shree K. Nayar,et al.  Projection defocus analysis for scene capture and image display , 2006, SIGGRAPH 2006.

[17]  Avideh Zakhor,et al.  Selection of temporally dithered codes for increasing virtual depth of field in structured light systems , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[18]  Song Zhang.  Recent progresses on real-time 3D shape measurement using digital fringe projection techniques , 2010 .

[19]  J. Goodman.  Speckle Phenomena in Optics: Theory and Applications , 2020 .

[20]  Yuandong Tian,et al.  (De) focusing on global light transport for active scene recovery , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Pierre Graebling,et al.  Robust Structured Light Coding for 3D Reconstruction , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[22]  Luc Van Gool,et al.  Scene-adapted structured light , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[23]  R. Lange,et al.  Solid-state time-of-flight range camera , 2001 .

[24]  Robert J. Woodham,et al.  Photometric method for determining surface orientation from multiple images , 1980 .

[25]  C.-F. Chen,et al.  Adaptive 3D Reconstruction system with improved recovery of miscoded region to automatically adjust projected light patterns , 2008, 2008 23rd International Conference Image and Vision Computing New Zealand.

[26]  Shuntaro Yamazaki,et al.  Temporal Dithering of Illumination for Fast Active Vision , 2008, ECCV.

[27]  Jeffrey L. Posdamer,et al.  Surface measurement by space-encoded projected beam systems , 1982, Comput. Graph. Image Process..

[28]  Emil M. Petriu,et al.  Visual Object Recognition Using Pseudo-random Grid Encoding , 1992, Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems.

[29]  Jianwei Zhang,et al.  Vision Processing for Realtime 3-D Data Acquisition Based on Coded Structured Light , 2008, IEEE Transactions on Image Processing.