Image-based motion compensation for structured light scanning of dynamic surfaces

Many structured light scanning systems based on temporal pattern codification produce dense and robust results on static scenes but behave very poorly when applied to dynamic scenes in which objects are allowed to move or to deform during the acquisition process. The main reason for this lies in the wrong combination of encoded correspondence information because the same point in the projector pattern sequence can map to different points within the camera images due to depth changes over time. We present a novel approach suitable for measuring and compensating such kind of pattern motion. The described technique can be combined with existing active range scanning systems designed for static surface reconstruction making them applicable for the dynamic case. We demonstrate the benefits of our method by integrating it into a gray code-based structured light scanner, which runs at 30 3D scans per second.

[1]  M. Maruyama,et al.  Range sensing by projecting multi-slits with random cuts , 1989, International Workshop on Industrial Applications of Machine Intelligence and Vision,.

[2]  Luc Van Gool,et al.  Fast 3D Scanning with Automatic Motion Compensation , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Li Zhang,et al.  Rapid shape acquisition using color structured light and multi-pass dynamic programming , 2002, Proceedings. First International Symposium on 3D Data Processing Visualization and Transmission.

[4]  Timo Kohlberger,et al.  Real-Time Optic Flow Computation with Variational Methods , 2003, CAIP.

[5]  David W. Capson,et al.  Surface profile measurement using color fringe projection , 1991, Machine Vision and Applications.

[6]  Paolo Cignoni,et al.  A low cost 3D scanner based on structured light , 2001 .

[7]  Szymon Rusinkiewicz,et al.  Spacetime stereo: a unifying framework for depth from triangulation , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Luc Van Gool,et al.  Real-time range scanning of deformable surfaces by adaptively coded structured light , 2003, Fourth International Conference on 3-D Digital Imaging and Modeling, 2003. 3DIM 2003. Proceedings..

[9]  Richard Szeliski,et al.  High-accuracy stereo depth maps using structured light , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[10]  Roger Y. Tsai,et al.  A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses , 1987, IEEE J. Robotics Autom..

[11]  André Oosterlinck,et al.  Range Image Acquisition with a Single Binary-Encoded Light Pattern , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Szymon Rusinkiewicz,et al.  Stripe boundary codes for real-time structured-light range scanning of moving objects , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[13]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Antonio Adán,et al.  3D feature tracking using a dynamic structured light system , 2005, The 2nd Canadian Conference on Computer and Robot Vision (CRV'05).

[15]  J.-Y. Bouguet,et al.  Pyramidal implementation of the lucas kanade feature tracker , 1999 .

[16]  Song Zhang,et al.  High-Resolution, Real-time 3D Shape Acquisition , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[17]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[18]  Li Zhang,et al.  Spacetime stereo: shape recovery for dynamic scenes , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[19]  Unai Bidarte,et al.  Hardware implementation of optical flow constraint equation using FPGAs , 2005, Comput. Vis. Image Underst..

[20]  Joaquim Salvi,et al.  Pattern codification strategies in structured light systems , 2004, Pattern Recognit..

[21]  Joaquim Salvi,et al.  Recent progress in structured light in order to solve the correspondence problem in stereovision , 1997, Proceedings of International Conference on Robotics and Automation.

[22]  Robert A. Hummel,et al.  Experiments with the intensity ratio depth sensor , 1985, Comput. Vis. Graph. Image Process..

[23]  François Blais.  Review of 20 years of range sensor development , 2004, J. Electronic Imaging.