Real-time range imaging for human-machine interfaces

Diese Arbeit befasst sich mit der automatischen kontaktlosen Echtzeiterfassung von Entfernungsbildern im Nahbereich. Dazu stellt sie ein entsprechendes neues Verfahren vor, das die klassischen Prinzipien des farbkodierten Lichtansatzes und des Stereosehens kombiniert und erweitert; daruber hinaus entwickelt sie neue Losungen fur verwandte Aufgaben wie der Kalibrierung des Aufnahmesystems. Die Evaluierung einer prototypischen Implementierung des Verfahrens ergibt eine Datenrate von bis zu 25 Entfernungsbildern der Auflosung 780 x 580 Punkten pro Sekunde. Die Standardabweichung der Messunsicherheit betragt etwa 0.2 mm bei einem Arbeitsraum von ca. 500 mm x 500 mm x 500 mm. Schlieslich zeigt diese Arbeit, dass die Erkennungssicherheit eines Gesichtserkennungs-Systems durch den Einsatz von mit dem Prototypen aufgenommenen Entfernungsbildern deutlich gesteigert werden kann.

[1]  Heiko Hirschmüller,et al.  Improvements in real-time correlation-based stereo vision , 2001, CVPR 2001.

[2]  Azriel Rosenfeld,et al.  Face recognition: A literature survey , 2003, CSUR.

[3]  T. P. Monks,et al.  Measuring the shape of time-varying objects , 1995 .

[4]  Florentin Wörgötter,et al.  A VLSI-Compatible Computer Vision Algorithm for Stereoscopic Depth Analysis in Real-Time , 2002, International Journal of Computer Vision.

[5]  Olga Veksler,et al.  A Variable Window Approach to Early Vision , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Bernd Radig,et al.  The HISCORE camera a real time three dimensional and color camera , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[7]  O. Faugeras Three-dimensional computer vision: a geometric viewpoint , 1993 .

[8]  M. Goresky,et al.  Polynomial pseudo-noise sequences based on algebraic feedback shift registers , 2004 .

[9]  Alex Pentland,et al.  Local Shading Analysis , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Rama Chellappa,et al.  Human and machine recognition of faces: a survey , 1995, Proc. IEEE.

[11]  Jake K. Aggarwal,et al.  Stochastic Analysis of Stereo Quantization Error , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Johji Tajima,et al.  3-D data acquisition by Rainbow Range Finder , 1990, [1990] Proceedings. 10th International Conference on Pattern Recognition.

[13]  Ferdinando Silvestro Samaria,et al.  Face recognition using Hidden Markov Models , 1995 .

[14]  Gérard G. Medioni,et al.  3-D Surface Description from Binocular Stereo , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Ramin Zabih,et al.  Non-parametric Local Transforms for Computing Visual Correspondence , 1994, ECCV.

[16]  W. Eric L. Grimson,et al.  Computational Experiments with a Feature Based Stereo Algorithm , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Mark S. Nixon,et al.  A Hough transform for detecting the location and orientation of three-dimensional surfaces via color encoded spots , 1998, IEEE Trans. Syst. Man Cybern. Part B.

[18]  Nahum Kiryati,et al.  Toward optimal structured light patterns , 1999, Image Vis. Comput..

[19]  Huang Yumin,et al.  A PHYSICAL APPROACH TO COLOR IMAGE UNDERSTANDING , 1991 .

[20]  Minoru Maruyama,et al.  Range Sensing By Projecting Multiple Slits With Random Cuts , 1990, Other Conferences.

[21]  H Takasaki,et al.  Moiré topography. , 1973, Applied optics.

[22]  Yuan-Fang Wang,et al.  Computation of Surface Orientation and Structure of Objects Using Grid Coding , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Soung R. Yee,et al.  The use of a uniquely encoded light pattern for range data acquisition , 1991 .

[25]  Shahriar Negahdaripour,et al.  Revised Definition of Optical Flow: Integration of Radiometric and Geometric Cues for Dynamic Scene Analysis , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  F. A. Seiler,et al.  Numerical Recipes in C: The Art of Scientific Computing , 1989 .

[27]  Andreas F. Koschan,et al.  Color stereo vision using hierarchical block matching and active color illumination , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[28]  D. C. Douglas Hung,et al.  3D scene modelling by sinusoid encoded illumination , 1993, Image Vis. Comput..

[29]  K. Sato,et al.  Range imaging system utilizing nematic liquid crystal mask , 1987 .

[30]  Emanuele Trucco,et al.  Efficient stereo with multiple windowing , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[31]  M. Takeda,et al.  Fourier transform profilometry for the automatic measurement of 3-D object shapes. , 1983, Applied optics.

[32]  K. Torrance,et al.  Theory for off-specular reflection from roughened surfaces , 1967 .

[33]  Keith Phillips,et al.  Applications of Vector Fields to Image Processing , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Hsin-Chu Liu,et al.  Optical three-dimensional sensing by phase measuring profilometry , 1989 .

[35]  Martin D. Levine,et al.  Computer determination of depth maps , 1973, Comput. Graph. Image Process..

[36]  Kim L. Boyer,et al.  Color-Encoded Structured Light for Rapid Active Ranging , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Kazuyuki Hattori,et al.  Accurate rangefinder with laser pattern shifting , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[38]  Bernd Jähne,et al.  Digitale Bildverarbeitung (6. Aufl.) , 2005 .

[39]  David W. Capson,et al.  Surface profile measurement using color fringe projection , 1991, Machine Vision and Applications.

[40]  Peter M. Will,et al.  Grid Coding: A Preprocessing Technique for Robot and Machine Vision , 1971, IJCAI.

[41]  J. Politch,et al.  High-accuracy distance measurements with multiple-wavelength interferometry , 1995 .

[42]  W. N. Kellogg Porpoises and Sonar , 1963 .

[43]  Peter Lawrence,et al.  An Investigation of Methods for Determining Depth from Focus , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[44]  Jorge J. Moré,et al.  User Guide for Minpack-1 , 1980 .

[45]  George C. Stockman,et al.  Surface Orientation from a Projected Grid , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[46]  Ludwig Listl,et al.  Fast range imaging by CMOS sensor array through multiple double short time integration (MDSI) , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[47]  K. Creath V Phase-Measurement Interferometry Techniques , 1988 .

[48]  Thomas Luhmann,et al.  Nahbereichsphotogrammetrie : Grundlagen, Methoden und Anwendungen , 2000 .

[49]  R. Y. Tsai,et al.  An Efficient and Accurate Camera Calibration Technique for 3D Machine Vision , 1986, CVPR 1986.

[50]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[51]  F. MacWilliams,et al.  Pseudo-random sequences and arrays , 1976, Proceedings of the IEEE.

[52]  Kieran G. Larkin,et al.  Design and assessment of symmetrical phase-shifting algorithms , 1992 .

[53]  Luc Van Gool,et al.  Active acquisition of 3D shape for moving objects , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[54]  Olga Veksler,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[55]  Minoru Asada,et al.  Determining Surface Orientation by Projecting a Stripe Pattern , 1988, IEEE Trans. Pattern Anal. Mach. Intell..

[56]  Yoshiaki Shirai,et al.  Recognition of polyhedrons with a range finder , 1971, IJCAI.

[57]  Steven D. Blostein,et al.  Error Analysis in Stereo Determination of 3-D Point Positions , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[58]  J. L. Wayman,et al.  Best practices in testing and reporting performance of biometric devices. , 2002 .

[59]  C. Rathjen,et al.  Statistical properties of phase-shift algorithms , 1995 .

[60]  Paul R. Cohen,et al.  Camera Calibration with Distortion Models and Accuracy Evaluation , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[61]  Jacobus H. van Lint,et al.  Introduction to Coding Theory , 1982 .

[62]  de Ng Dick Bruijn A combinatorial problem , 1946 .

[63]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[64]  A. U.S.,et al.  Recovering Surface Shape and Orientation from Texture , 2002 .

[65]  Monson H. Hayes,et al.  A hidden markov model-based approach for face detection and recognition , 1999 .

[66]  G Häusler,et al.  Parallel three-dimensional sensing by color-coded triangulation. , 1993, Applied optics.

[67]  Y. F. WANG,et al.  Three-dimensional shape construction and recognition by fusing intensity and structured lighting , 1992, Pattern Recognit..

[68]  Rudolf Schwarte,et al.  New active 3D vision system based on rf-modulation interferometry of incoherent light , 1995, Other Conferences.

[69]  Alan M. McIvor,et al.  Accurate 3D measurement using a structured light system , 1998, Image Vis. Comput..

[70]  Bernd Radig,et al.  Real-time range imaging for dynamic scenes using colour-edge based structured light , 2002, Object recognition supported by user interaction for service robots.

[71]  Takeo Kanade,et al.  A Cooperative Algorithm for Stereo Matching and Occlusion Detection , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[72]  Zheng Geng,et al.  Rainbow three‐dimensional camera: new concept of high‐speed three‐dimensional vision systems , 1996 .

[73]  E. Lommel,et al.  Lehrbuch der Experimentalphysik , 1900 .

[74]  Takeo Kanade,et al.  A multibaseline stereo system with active illumination and real-time image acquisition , 1995, Proceedings of IEEE International Conference on Computer Vision.

[75]  Narendra Ahuja,et al.  Shape From Texture: Integrating Texture-Element Extraction and Surface Estimation , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[76]  Tomaso A. Poggio,et al.  On Edge Detection , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[77]  Oscar E. Agazzi,et al.  Keyword Spotting in Poorly Printed Documents using Pseudo 2-D Hidden Markov Models , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[78]  Yuan-Fang Wang,et al.  Characterizing Three-Dimensional Surface Structures from Visual Images , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[79]  Gongzhu Hu,et al.  3-D Surface Solution Using Structured Light and Constraint Propagation , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[80]  Berthold K. P. Horn Understanding Image Intensities , 1977, Artif. Intell..

[81]  Takeo Kanade,et al.  A Stereo Matching Algorithm with an Adaptive Window: Theory and Experiment , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[82]  Jeremy D. Pearson Automated visual measurement of body shape in scoliosis. , 1996 .

[83]  Minoru Asada,et al.  Utilization of a Stripe Pattern for Dynamic Scene Analysis , 1985, IJCAI.

[84]  Cengizhan Ozturk,et al.  Structured Light Using Pseudorandom Codes , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[85]  Nicholas Ayache,et al.  Artificial vision for mobile robots - stereo vision and multisensory perception , 1991 .

[86]  Marc Acheroy,et al.  3D Facial Surface Acquisition by Structured Light , 1999 .

[87]  D Marr,et al.  Cooperative computation of stereo disparity. , 1976, Science.

[88]  Bui Tuong Phong Illumination for computer generated pictures , 1975, Commun. ACM.

[89]  Joaquim Salvi,et al.  A robust-coded pattern projection for dynamic 3D scene measurement , 1998, Pattern Recognit. Lett..

[90]  Shojiro Sakata,et al.  Reconstruction Of Surfaces Of 3-D Objects By M-array Pattern Projection Method , 1988, [1988 Proceedings] Second International Conference on Computer Vision.

[91]  Jorge L. C. Sanz,et al.  Advances in Machine Vision , 1988, Springer Series in Perception Engineering.

[92]  Jeffrey L. Posdamer,et al.  Surface measurement by space-encoded projected beam systems , 1982, Comput. Graph. Image Process..

[93]  Bill Triggs,et al.  Autocalibration from Planar Scenes , 1998, ECCV.

[94]  B. Radig,et al.  Real-Time 3D and Color Camera , 2001 .

[95]  J P Frisby,et al.  PMF: A Stereo Correspondence Algorithm Using a Disparity Gradient Limit , 1985, Perception.

[96]  Azriel Rosenfeld,et al.  Improved Methods of Estimating Shape from Shading Using the Light Source Coordinate System , 1985, Artif. Intell..

[97]  André Oosterlinck,et al.  Range Image Acquisition with a Single Binary-Encoded Light Pattern , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[98]  Szymon Rusinkiewicz,et al.  Stripe boundary codes for real-time structured-light range scanning of moving objects , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[99]  Martin D. Altschuler,et al.  Measuring Surfaces Space-Coded By A Laser-Projected Dot Matrix , 1979, Other Conferences.

[100]  Aldo Cumani,et al.  Edge detection in multispectral images , 1991, CVGIP Graph. Model. Image Process..

[101]  Peter F. Sweeney Error control coding - an introduction , 1991 .

[102]  Shree K. Nayar,et al.  Real-Time Focus Range Sensor , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[103]  Thomas O. Binford,et al.  Computer Description of Curved Objects , 1973, IEEE Transactions on Computers.

[104]  Rama Chellappa,et al.  Estimation of Illuminant Direction, Albedo, and Shape from Shading , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[105]  Yoshiaki Shirai,et al.  Three-Dimensional Computer Vision , 1987, Symbolic Computation.

[106]  Riccardo March,et al.  Computation of stereo disparity using regularization , 1988, Pattern Recognit. Lett..

[107]  Richard Szeliski,et al.  A layered approach to stereo reconstruction , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[108]  Fu-Pen Chiang,et al.  Color-encoded digital fringe projection technique for high-speed three-dimensional surface contouring , 1999 .

[109]  Ping-Sing Tsai,et al.  Shape from Shading: A Survey , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[110]  Luc Van Gool,et al.  Reading between the lines—a method for extracting dynamic 3D with texture , 1997, VRST '97.

[111]  Shankar Chatterjee,et al.  A quantization error analysis for convergent stereo , 1994, Proceedings of 1st International Conference on Image Processing.

[112]  H. Barrow,et al.  Computational vision , 1981, Proceedings of the IEEE.

[113]  Jens Guehring,et al.  Dense 3D surface acquisition by structured light using off-the-shelf components , 2000, IS&T/SPIE Electronic Imaging.

[114]  T. Kanade,et al.  A Method of Time-Coded Parallel Planes of Light for Depth Measurement , 1981 .

[115]  Rachid Deriche,et al.  Dense Depth Map Reconstruction: A Minimization and Regularization Approach which Preserves Discontinuities , 1996, ECCV.

[116]  Robert Lange,et al.  3D time-of-flight distance measurement with custom solid-state image sensors in CMOS/CCD-technology , 2006 .

[117]  Laurence G. Hassebrook,et al.  Application of communication theory to high-speed structured light illumination , 1997, Other Conferences.

[118]  Carlo Tomasi,et al.  Surfaces with occlusions from layered stereo , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[119]  Joseph Shamir,et al.  Range Imaging With Adaptive Color Structured Light , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[120]  Steven A. Shafer,et al.  Using color to separate reflection components , 1985 .

[121]  D. Griffin Listening in the dark: The acoustic orientation of bats and men. , 1958 .

[122]  Roger Y. Tsai,et al.  A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses , 1987, IEEE J. Robotics Autom..

[123]  A. Hall Applied Optics. , 2022, Science.

[124]  Robert A. Hummel,et al.  Experiments with the intensity ratio depth sensor , 1985, Comput. Vis. Graph. Image Process..

[125]  Gunther Wyszecki,et al.  Color Science: Concepts and Methods, Quantitative Data and Formulae, 2nd Edition , 2000 .

[126]  Paul M. Griffin,et al.  Generation of uniquely encoded light patterns for range data acquisition , 1992, Pattern Recognit..

[127]  Takeo Kanade,et al.  Surface Reflection: Physical and Geometrical Perspectives , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[128]  Francois Blais,et al.  Compact three-dimensional camera for robotic applications , 1986 .

[129]  Solomon W. Golomb,et al.  Shift Register Sequences , 1981 .

[130]  Heinz Hugli,et al.  Generation And Use Of Color Pseudo Random Sequences For Coding Structured Light In Active Ranging , 1989, Other Conferences.

[131]  G. Stanke,et al.  Visuelle Echtzeitvermessung räumlicher Deformationsprozesse durch spektral codierte Szenenbeleuchtung , 2000 .

[132]  R. Woodham,et al.  Determining the movement of objects from a sequence of images , 1980, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[133]  Andrew K. C. Wong,et al.  Machine Intelligence and Knowledge Engineering for Robotic Applications , 1987, NATO ASI Series.

[134]  Takeo Kanade,et al.  A very fast VLSI rangefinder , 1991, Proceedings. 1991 IEEE International Conference on Robotics and Automation.

[135]  S. Nayar,et al.  Ordinal Measures for Image Correspondence , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[136]  Luc Van Gool,et al.  One-shot active 3D shape acquisition , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[137]  Ja-Ling Wu,et al.  Range data acquisition using color structured lighting and stereo vision , 1997, Image Vis. Comput..

[138]  Thomas S. Huang,et al.  Quantization errors in axial motion stereo on rectangular-tessellated image sensors , 1992, [1992] Proceedings. 11th IAPR International Conference on Pattern Recognition.

[139]  Berthold K. P. Horn,et al.  Shape from shading , 1989 .