Exploiting Wavelength Diversity for High Resolution Time-of-Flight 3D Imaging

State-of-the-art time-of-flight (ToF) based 3D sensors suffer from poor lateral and depth resolutions. In this work, we introduce a novel sensor concept that provides ToF-based 3D measurements of real world objects with depth precisions up to 35 micrometers and point cloud densities at the native sensor-resolutions of state-of-the-art CMOS/CCD cameras (up to several megapixels). Unlike other continuous-wave amplitude-modulated ToF principles, our approach exploits wavelength diversity for an interferometric surface measurement of macroscopic objects with rough or specular surfaces. Based on this principle, we introduce three different embodiments of prototype sensors, exploiting three different sensor architectures.

[1]  S. Beer,et al.  Real-time photon-noise limited optical coherence tomography based on pixel-level analog signal processing , 2006 .

[2]  Florian Willomitzer,et al.  Single-shot three-dimensional sensing with improved data density , 2015 .

[3]  Ramesh Raskar,et al.  Rethinking Machine Vision Time of Flight With GHz Heterodyning , 2017, IEEE Access.

[4]  Ian Coddington,et al.  Sub-micron absolute distance measurements in sub-millisecond times with dual free-running femtosecond Er fiber-lasers. , 2011, Optics express.

[5]  Xiang Huang,et al.  A Streamlined Photometric Stereo Framework for Cultural Heritage , 2016, ECCV Workshops.

[6]  Muralidhar Madabhushi Balaji,et al.  Indirect Imaging using Heterodyne Remote Digital Holography , 2018 .

[7]  R. Dändliker,et al.  Two-wavelength laser interferometry using superheterodyne detection. , 1988, Optics letters.

[8]  Florian Schiffers,et al.  Hand-guided qualitative deflectometry with a mobile device. , 2020, Optics express.

[9]  J Schwider,et al.  Phase shifting interferometry: reference phase error reduction. , 1989, Applied optics.

[10]  Yannick Caulier Inspection of complex surfaces by means of structured light patterns. , 2010, Optics express.

[11]  J. Goodman Statistical Properties of Laser Speckle Patterns , 1963 .

[12]  Prasanna Rangarajan,et al.  Synthetic Wavelength Holography: An Extension of Gabor's Holographic Principle to Imaging with Scattered Wavefronts , 2019, 1912.11438.

[13]  Edoardo Charbon,et al.  Megapixel time-gated SPAD image sensor for 2D and 3D imaging applications , 2019, Optica.

[14]  J. Goodman Speckle Phenomena in Optics: Theory and Applications , 2020 .

[15]  Daniel F. García,et al.  An improved 3D imaging system for dimensional quality inspection of rolled products in the metal industry , 2013, Comput. Ind..

[16]  Martin Schaffer,et al.  3D shape measurement of static and moving objects with adaptive spatiotemporal correlation. , 2014, Applied optics.

[17]  A. Tünnermann,et al.  High-speed three-dimensional shape measurement using GOBO projection , 2016 .

[18]  Prasanna Rangarajan,et al.  Spatiotemporal Holography Using Flutter Shutter Camera , 2017 .

[19]  Andrew W. Fitzgibbon,et al.  KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera , 2011, UIST.

[20]  Karsten Danzmann,et al.  Real-time phase-front detector for heterodyne interferometers. , 2007, Applied optics.

[21]  Florian Willomitzer,et al.  Flying Triangulation - a Motion-Robust Optical 3D Sensor for the Real-Time Shape Acquisition of Complex Objects , 2013 .

[22]  Florian Willomitzer,et al.  Single-shot 3D motion picture camera with a dense point cloud. , 2017, Optics express.

[23]  G. Ripandelli,et al.  Optical coherence tomography. , 1998, Seminars in ophthalmology.

[24]  Rob Fergus,et al.  Depth Map Prediction from a Single Image using a Multi-Scale Deep Network , 2014, NIPS.

[25]  Martin J. Dürst,et al.  Re , 1988 .

[26]  Laura Waller,et al.  DiffuserCam: Lensless Single-exposure 3D Imaging , 2017, ArXiv.

[27]  Jinwei Gu,et al.  Very Power Efficient Neural Time-of-Flight , 2020, 2020 IEEE Winter Conference on Applications of Computer Vision (WACV).

[28]  R. Lange,et al.  Solid-state time-of-flight range camera , 2001 .

[29]  Jason Geng,et al.  Structured-light 3D surface imaging: a tutorial , 2011 .

[30]  Antonios Gasteratos,et al.  Review of Stereo Vision Algorithms: From Software to Hardware , 2008 .

[31]  Prasanna Rangarajan,et al.  Mega-pixel time-of-flight imager with GHz modulation frequencies , 2019, Imaging and Applied Optics 2019 (COSI, IS, MATH, pcAOP).

[32]  Martin Schaffer,et al.  High-speed pattern projection for three-dimensional shape measurement using laser speckles. , 2010, Applied optics.

[33]  Jeanette G. Grasselli,et al.  “On the Relative Motion of the Earth and the Luminiferous Ether” , 1987 .

[34]  Ashok Veeraraghavan,et al.  Spatial Phase-Sweep: Increasing temporal resolution of transient imaging using a light source array , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[35]  J C Wyant,et al.  Two-wavelength phase shifting interferometry. , 1984, Applied optics.

[36]  Frédo Durand,et al.  Image and depth from a conventional camera with a coded aperture , 2007, ACM Trans. Graph..

[37]  M. Schmid Principles Of Optics Electromagnetic Theory Of Propagation Interference And Diffraction Of Light , 2016 .

[38]  Ramesh Raskar,et al.  Dynamic heterodyne interferometry , 2018, 2018 IEEE International Conference on Computational Photography (ICCP).

[39]  Munther A. Gdeisat,et al.  Fast two-dimensional phase-unwrapping algorithm based on sorting by reliability following a noncontinuous path. , 2002, Applied optics.

[40]  Franz J. T. Huber,et al.  3D body scanning with "Flying Triangulation" , 2011 .

[41]  Florian Willomitzer,et al.  Consequences of EEG electrode position error on ultimate beamformer source reconstruction performance , 2014, Front. Neurosci..

[42]  Ting Xu,et al.  Label-free evaluation of angiogenic sprouting in microengineered devices using ultrahigh-resolution optical coherence microscopy , 2014, Journal of biomedical optics.

[43]  Gordon Wetzstein,et al.  Deep End-to-End Time-of-Flight Imaging , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[44]  Marc Pollefeys,et al.  Motion Estimation for Self-Driving Cars with a Generalized Camera , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[45]  A F Fercher,et al.  Rough surface interferometry with a two-wavelength heterodyne speckle interferometer. , 1985, Applied optics.

[46]  James D. Siekierski,et al.  Space Shuttle thermal protection system inspection by 3D imaging laser radar , 2004, SPIE Defense + Commercial Sensing.

[47]  Helmut Fischer,et al.  New electro-optical mixing and correlating sensor: facilities and applications of the photonic mixer device (PMD) , 1997, Other Conferences.

[48]  Andreas Velten,et al.  High-depth-resolution range imaging with multiple-wavelength superheterodyne interferometry using 1550-nm lasers. , 2017, Applied optics.

[49]  C Polhemus,et al.  Two-wavelength interferometry. , 1973, Applied optics.

[50]  Oliver Cossairt,et al.  WISHED: Wavefront imaging sensor with high resolution and depth ranging , 2020, 2020 IEEE International Conference on Computational Photography (ICCP).

[51]  T. Yilmaz,et al.  Range resolved lidar for long distance ranging with sub-millimeter resolution. , 2010, Optics express.

[52]  Wei Xu,et al.  Phase-unwrapping of SAR interferogram with multi-frequency or multi-baseline , 1994, Proceedings of IGARSS '94 - 1994 IEEE International Geoscience and Remote Sensing Symposium.

[53]  Yoav Y. Schechner,et al.  Depth from Defocus vs. Stereo: How Different Really Are They? , 2004, International Journal of Computer Vision.

[54]  Amit K. Agrawal,et al.  Coded exposure photography: motion deblurring using fluttered shutter , 2006, ACM Trans. Graph..

[55]  S. Foix,et al.  Lock-in Time-of-Flight (ToF) Cameras: A Survey , 2011, IEEE Sensors Journal.

[56]  Sebastian Thrun,et al.  Towards fully autonomous driving: Systems and algorithms , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[57]  Ramesh Raskar,et al.  Polarized 3D: High-Quality Depth Sensing with Polarization Cues , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[58]  Michael Frankfurter,et al.  Optical Shop Testing , 2016 .

[59]  Prasanna Rangarajan,et al.  High Resolution Non-Line-of-Sight Imaging with Superheterodyne Remote Digital Holography , 2019, Imaging and Applied Optics 2019 (COSI, IS, MATH, pcAOP).

[60]  Ruigang Yang,et al.  Simultaneous Time-of-Flight sensing and photometric stereo with a single ToF sensor , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[61]  J. Streicher,et al.  Lidar: Range-Resolved Optical Remote Sensing of the Atmosphere , 2005 .

[62]  Michael Atlan,et al.  Full Field Holographic Vibrometry at Ultimate Limits , 2015 .

[63]  Andreas Velten,et al.  SH-ToF: Micro resolution time-of-flight imaging with superheterodyne interferometry , 2018, 2018 IEEE International Conference on Computational Photography (ICCP).

[64]  Prasanna Rangarajan,et al.  Synthetic Wavelength Holography: Snapshot Non-Line-of-Sight Imaging with High-Resolution and Wide Field of View , 2020 .