High-resolution, High-speed 3-D Dynamically Deformable Shape Measurement Using Digital Fringe Projection Techniques

With recent advancements of 3-D geometric shape analysis, acquiring 3-D dynamic scenes in real time becomes increasingly important in enormous fields, such as manufacturing, medical sciences, computer sciences, homeland security, and entertainment, etc. Over the last decades, 3-D shape measurement techniques have been improving dramatically rapidly. Numerous techniques have been developed including time of flight, stereo vision, structured light, and digital fringe projection. Because of the development of digital display technology, the structured-light-based methods have the potential to be the most important technology for 3-D shape measurement in both scientific research and industrial practices in the long run. Real-time 3-D shape measurement has recently emerged as a quickly expanding field with the enhancements of computation power of an ordinary personal computer. For real-time 3-D shapemeasurement, 3-D shapes have to be acquired rapidly, processed quickly, and displayed in real time (Zhang, 2010). In the past few years, a number of technologies have been developed. Among which, the optical method is one of the core techniques. However, due to its fundamentally difficult nature, capturing 3-D dynamic scenes with high quality remains challenging, and even fewer systems are capable of processing 3-D geometry in real time because of the complexity of the problem Our group has devoted a vast amount of effort in real-time 3-D shape measurement field. Over the past few years, we have been developing technologies using a technique called digital fringe projection and phase-shifting technique, a variation of the structured light technique with structured patterns being sinusoidally changing intensity. We have developed a number of algorithms to improve the processing speed, and taken advantage of the most advanced hardware technologies, graphics processing unit (GPU), to increase the speed of 3-D geometry reconstruction and rendering. With these technologies, we have successfully developed a high-resolution, real-time 3-D shape measurement that achieved simultaneous 3-D absolute shape acquisition, reconstruction, and display at a speed of faster than 26 frames / sec with 300K points per frame under an ordinary personal computer (Zhang et al., 2006). Experiments have demonstrated that the system can accurately capture dynamic changing 3-D scenes, such as human facial expressions. The data acquired by such a system have already been applied to 2

[1]  Song Zhang,et al.  High-speed three-dimensional shape measurement system using a modified two-plus-one phase-shifting algorithm , 2007 .

[2]  Song Zhang,et al.  High dynamic range scanning technique , 2009 .

[3]  Thomas J. Flynn,et al.  TWO-DIMENSIONAL PHASE UNWRAPPING WITH MINIMUM WEIGHTED DISCONTINUITY , 1997 .

[4]  Szymon Rusinkiewicz,et al.  Stripe boundary codes for real-time structured-light range scanning of moving objects , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[5]  Francisco J. Cuevas,et al.  Depth object recovery using radial basis functions , 1999 .

[6]  Marc Levoy,et al.  Real-time 3D model acquisition , 2002, ACM Trans. Graph..

[7]  W. Staszewski,et al.  Fatigue crack detection in metallic structures with Lamb waves and 3D laser vibrometry , 2007 .

[8]  Sen Wang,et al.  High Resolution Tracking of Non-Rigid Motion of Densely Sampled 3D Data Using Harmonic Maps , 2008, International Journal of Computer Vision.

[9]  Ioannis A. Kakadiaris,et al.  Three-Dimensional Face Recognition in the Presence of Facial Expressions: An Annotated Deformable Model Approach , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Dimitris N. Metaxas,et al.  A Hierarchical Framework For High Resolution Facial Expression Tracking , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[11]  Antonio Baldi Phase unwrapping by region growing. , 2003, Applied optics.

[12]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Song Zhang,et al.  High-Resolution, Real-time 3D Shape Acquisition , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[14]  Joaquim Salvi,et al.  Pattern codification strategies in structured light systems , 2004, Pattern Recognit..

[15]  Jeffrey J. Gierloff Phase Unwrapping By Regions , 1987, Optics & Photonics.

[16]  Fu-Pen Chiang,et al.  Color-encoded digital fringe projection technique for high-speed three-dimensional surface contouring , 1999 .

[17]  Tomaso A. Poggio,et al.  Reanimating Faces in Images and Video , 2003, Comput. Graph. Forum.

[18]  E Stüssi,et al.  Three-dimensional video analysis of facial movements: a new method to assess the quantity and quality of the smile. , 1999, Plastic and reconstructive surgery.

[19]  Werner Jüptner,et al.  Accurate procedure for the calibration of a structured light system , 2004 .

[20]  Maurício Pamplona Segundo,et al.  3D Face Recognition Using Simulated Annealing and the Surface Interpenetration Measure , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Song Zhang,et al.  High-resolution, real-time three-dimensional shape measurement , 2006 .

[22]  J. N. Bleicher,et al.  A Survey of Facial Paralysis: Etiology and Incidence , 1996, Ear, nose, & throat journal.

[23]  Sukun Kim,et al.  Health Monitoring of Civil Infrastructures Using Wireless Sensor Networks , 2007, 2007 6th International Symposium on Information Processing in Sensor Networks.

[24]  Ahmed M. Elgammal,et al.  High Resolution Acquisition, Learning and Transfer of Dynamic 3‐D Facial Expressions , 2004, Comput. Graph. Forum.

[25]  Song Zhang,et al.  High dynamic range scanning technique , 2008, Optical Engineering + Applications.

[26]  T. Hadlock,et al.  Facial reanimation: an invited review and commentary. , 2008, Archives of facial plastic surgery.

[27]  C. Linstrom Objective Facial Motion Analysis in Patients With Facial Nerve Dysfunction , 2002, The Laryngoscope.

[28]  Song Zhang,et al.  GPU-assisted high-resolution, real-time 3-D shape measurement. , 2006, Optics express.

[29]  Louis A. Romero,et al.  Minimum Lp-norm two-dimensional phase unwrapping , 1996 .

[30]  T. Sejnowski,et al.  Measuring facial expressions by computer image analysis. , 1999, Psychophysiology.

[31]  Luc Van Gool,et al.  Realistic face animation for speech , 2002, Comput. Animat. Virtual Worlds.

[32]  Song Zhang,et al.  Flexible 3-D shape measurement using projector defocusing. , 2009, Optics letters.

[33]  Yazhu Chen,et al.  Flexible and accurate implementation of a binocular structured light system , 2008 .

[34]  R. Erbacher,et al.  A 3 D Shape Measurement System , 2005 .

[35]  Song Zhang,et al.  A fast three-step phase-shifting algorithm , 2005, SPIE Optics East.

[36]  Peisen S. Huang,et al.  Fast three-step phase-shifting algorithm , 2006 .

[37]  Kevin George Harding,et al.  Shiny parts measurement using color separation , 2005, SPIE Optics East.

[38]  Zhongwei Li,et al.  Accurate calibration method for a structured light system , 2008 .

[39]  Joel H. Saltz,et al.  Exploiting Parallelism on Irregular Applications Using the GPU , 2005, PARCO.

[40]  James Demmel,et al.  Health Monitoring of Civil Infrastructures Using Wireless Sensor Networks , 2007, 2007 6th International Symposium on Information Processing in Sensor Networks.

[41]  Song Zhang Recent progresses on real-time 3D shape measurement using digital fringe projection techniques , 2010 .

[42]  Eryi Hu,et al.  Phase deviation analysis and phase retrieval for partial intensity saturation in phase-shifting projected fringe profilometry☆ , 2008 .

[43]  Zhanyi Hu,et al.  Flexible method for structured light system calibration , 2008 .

[44]  R. Manktelow,et al.  Evaluation of a New Measurement Tool for Facial Paralysis Reconstruction , 2005, Plastic and reconstructive surgery.

[45]  William J. Dally,et al.  Exploring the VLSI scalability of stream processors , 2003, The Ninth International Symposium on High-Performance Computer Architecture, 2003. HPCA-9 2003. Proceedings..

[46]  Song Zhang,et al.  COLOR N-ARY GRAY CODE FOR 3-D SHAPE MEASUREMENT , 2004 .

[47]  Sen Wang,et al.  High resolution tracking of non-rigid 3D motion of densely sampled data using harmonic maps , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[48]  M. Takeda,et al.  Fourier transform profilometry for the automatic measurement of 3-D object shapes. , 1983, Applied optics.

[49]  Jonathan Kofman,et al.  Two-step triangular-pattern phase-shifting method for three-dimensional object-shape measurement , 2007 .

[50]  Patrick J. Flynn,et al.  A survey of approaches and challenges in 3D and multi-modal 3D + 2D face recognition , 2006, Comput. Vis. Image Underst..

[51]  Peisen S. Huang,et al.  Absolute phase retrieval for 3D shape measurement by Fourier transform method , 2007, SPIE Optics East.

[52]  J. M. Huntley Noise-immune phase unwrapping algorithm. , 1989, Applied optics.

[53]  Fu-Pen Chiang,et al.  Trapezoidal phase-shifting method for three-dimensional shape measurement , 2005 .

[54]  Peisen S. Huang,et al.  3-D shape measurement by use of a modified Fourier transform method , 2008, Optical Engineering + Applications.

[55]  Peisen S. Huang,et al.  Novel method for structured light system calibration , 2006 .

[56]  Song Zhang Digital multiple wavelength phase shifting algorithm , 2009, Optical Engineering + Applications.

[57]  Qican Zhang,et al.  Dynamic 3-D shape measurement method: A review , 2010 .

[58]  J. Jones,et al.  Optimum frequency selection in multifrequency interferometry. , 2003, Optics letters.

[59]  Ying Xu,et al.  Optimal checkerboard selection for structured light system calibration , 2009, Optical Engineering + Applications.

[60]  Xiaolin Li,et al.  Multilevel quality-guided phase unwrapping algorithm for real-time three-dimensional shape reconstruction. , 2007, Applied optics.

[61]  Ioannis A. Kakadiaris,et al.  Evaluation of 3D Face Recognition in the presence of facial expressions: an Annotated Deformable Model approach , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[62]  T. Hadlock,et al.  Multimodality Approach to Management of the Paralyzed Face , 2006, The Laryngoscope.

[63]  Xu Han,et al.  On improving the accuracy of structured light systems , 2006, SPIE Optics East.

[64]  Jim Austin,et al.  Three-dimensional face recognition using combinations of surface feature map subspace components , 2008, Image Vis. Comput..

[65]  Fu-Pen Chiang,et al.  High-speed 3-D shape measurement based on digital fringe projection , 2003 .

[66]  Jake K. Aggarwal,et al.  Structure from stereo-a review , 1989, IEEE Trans. Syst. Man Cybern..

[67]  Tessa A. Hadlock,et al.  Novel 3-D video for quantification of facial movement , 2008, Otolaryngology--head and neck surgery : official journal of American Academy of Otolaryngology-Head and Neck Surgery.

[68]  Alexander M. Bronstein,et al.  Three-Dimensional Face Recognition , 2005, International Journal of Computer Vision.