An Overview of Lidar Imaging Systems for Autonomous Vehicles

Lidar imaging systems are one of the hottest topics in the optronics industry. The need to sense the surroundings of every autonomous vehicle has pushed forward a race dedicated to deciding the final solution to be implemented. However, the diversity of state-of-the-art approaches to the solution brings a large uncertainty on the decision of the dominant final solution. Furthermore, the performance data of each approach often arise from different manufacturers and developers, which usually have some interest in the dispute. Within this paper, we intend to overcome the situation by providing an introductory, neutral overview of the technology linked to lidar imaging systems for autonomous vehicles, and its current state of development. We start with the main single-point measurement principles utilized, which then are combined with different imaging strategies, also described in the paper. An overview of the features of the light sources and photodetectors specific to lidar imaging systems most frequently used in practice is also presented. Finally, a brief section on pending issues for lidar development in autonomous vehicles has been included, in order to present some of the problems which still need to be solved before implementation may be considered as final. The reader is provided with a detailed bibliography containing both relevant books and state-of-the-art papers for further progress in the subject.

[1]  D. Dolfi,et al.  Frequency-modulated multifunction lidar for anemometry, range finding, and velocimetry-1. Theory and signal processing. , 2017, Applied optics.

[2]  H. Urey,et al.  MEMS Laser Scanners: A Review , 2014, Journal of Microelectromechanical Systems.

[3]  Byoungho Lee,et al.  Review of the present status of optical fiber sensors , 2003 .

[4]  Reinhard Koch,et al.  Technical Foundation and Calibration Methods for Time-of-Flight Cameras , 2013, Time-of-Flight and Depth Imaging.

[5]  Paul F. McManamon,et al.  Field Guide to Lidar , 2015 .

[6]  R. Paschotta Field Guide to Optical Fiber Technology , 2010 .

[7]  Ranjeet Kumar,et al.  High-resolution aliasing-free optical beam steering , 2016 .

[8]  Stephen Crouch Advantages of 3D Imaging Coherent Lidar for Autonomous Driving Applications , 2018 .

[9]  Zheng You,et al.  5 V Compatible Two-Axis PZT Driven MEMS Scanning Mirror with Mechanical Leverage Structure for Miniature LiDAR Application , 2017, Sensors.

[10]  A. Rogalski Infrared detectors: an overview , 2002 .

[11]  Francesc Rocadenbosch,et al.  Current Research in Lidar Technology Used for the Remote Sensing of Atmospheric Aerosols , 2017, Sensors.

[12]  David N. Payne,et al.  Er/sup 3+/-Yb/sup 3+/ and Er/sup 3+/ doped fiber lasers , 1989 .

[13]  Matti Kutila,et al.  Testing and Validation of Automotive Point-Cloud Sensors in Adverse Weather Conditions , 2019, Applied Sciences.

[14]  D. Renker Geiger-mode avalanche photodiodes, history, properties and problems , 2006 .

[15]  Shumeet Baluja,et al.  Evolution of an artificial neural network based autonomous land vehicle controller , 1996, IEEE Trans. Syst. Man Cybern. Part B.

[16]  David Pogue,et al.  Hail, Robo-Taxi. , 2016, Scientific American.

[17]  Hidetoshi Otono,et al.  Development and study of the multi pixel photon counter , 2007 .

[18]  F. Capasso Chapter 1 Physics of Avalanche Photodiodes , 1985 .

[19]  F. Bernardin,et al.  Light Transmission in Fog: The Influence of Wavelength on the Extinction Coefficient , 2019, Applied Sciences.

[20]  R. Lang,et al.  External optical feedback effects on semiconductor injection laser properties , 1980 .

[21]  Michael R. Watts,et al.  Large-scale nanophotonic phased array , 2013, Nature.

[22]  Brent Schwarz,et al.  LIDAR: Mapping the world in 3D , 2010 .

[23]  Ralph Helmar Rasshofer,et al.  Automotive Radar and Lidar Systems for Next Generation Driver Assistance Functions , 2005 .

[24]  T. Taira,et al.  Single-mode oscillation of laser-diode-pumped Nd:YVO(4) microchip lasers. , 1991, Optics letters.

[25]  Jean I. Montagu,et al.  Galvanometric and Resonant Scanners , 2004, Handbook of Optical and Laser Scanning.

[26]  Cristiano Premebida,et al.  Pedestrian detection combining RGB and dense LIDAR data , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[27]  J. Zayhowski,et al.  Topical Papers on Microchip Lasers and Applications. Passively Q-Switched Microchip Lasers and Applications. , 1998 .

[28]  M. Heck Highly integrated optical phased arrays: photonic integrated circuits for optical beam shaping and beam steering , 2017 .

[29]  Ralph Helmar Rasshofer,et al.  Influences of weather phenomena on automotive laser radar systems , 2011 .

[30]  Haiyin Sun,et al.  A Practical Guide to Handling Laser Diode Beams , 2015 .

[31]  Arthur A. Oliner,et al.  Phased array antennas , 1972 .

[32]  Werner Ritter,et al.  A Benchmark for Lidar Sensors in Fog: Is Detection Breaking Down? , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).

[33]  Alphan Sennaroglu,et al.  Solid-State Lasers and Applications , 2006 .

[34]  E. Charbon,et al.  SPAD-based Sensors , 2013 .

[35]  Guangcan Liu,et al.  Motion control of the wedge prisms in Risley-prism-based beam steering system for precise target tracking. , 2013, Applied optics.

[36]  Yongwan Park,et al.  Design and implementation of 3D LIDAR based on pixel-by-pixel scanning and DS-OCDMA , 2017, OPTO.

[37]  Edward A. Watson,et al.  A comparison flash lidar detector options , 2016, Defense + Security.

[38]  D. Kunze,et al.  Large deflection micromechanical scanning mirrors for linear scans and pattern generation , 2000, IEEE Journal of Selected Topics in Quantum Electronics.

[39]  Caglar Ataman,et al.  A dual-axis pointing mirror with moving-magnet actuation , 2013 .

[40]  Barry L. Stann,et al.  Progress on MEMS-scanned ladar , 2016, Defense + Security.

[41]  Christoph Stiller,et al.  Velodyne SLAM , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[42]  Reinhard Koch,et al.  Time‐of‐Flight Cameras in Computer Graphics , 2010, Comput. Graph. Forum.

[43]  John Strong,et al.  The infrared transmission of atmospheric windows , 1953 .

[44]  A. Vacheret,et al.  Impact of after-pulse, pixel crosstalk and recovery time in multi-pixel photon counter™ response , 2009 .

[45]  Suya You,et al.  Augmented virtual environments (AVE): dynamic fusion of imagery and 3D models , 2003, IEEE Virtual Reality, 2003. Proceedings..

[46]  Hironobu Narui,et al.  9-kW peak power and 150-fs duration blue-violet optical pulses generated by GaInN master oscillator power amplifier. , 2017, Optics express.

[47]  Krzysztof Kulpa,et al.  Two-dimensional Signal Processing in FMCW Radars , 1997 .

[48]  Pedro J. Navarro,et al.  A Systematic Review of Perception System and Simulators for Autonomous Vehicles Research , 2019, Sensors.

[49]  Franco Zappa,et al.  Evolution and prospects for single-photon avalanche diodes and quenching circuits , 2004 .

[50]  A. Gilerson,et al.  Range-resolved pulsed and CWFM lidars: potential capabilities comparison , 2006 .

[51]  S. Burak Gokturk,et al.  A Time-Of-Flight Depth Sensor - System Description, Issues and Solutions , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[52]  T. Taimre,et al.  Laser feedback interferometry: a tutorial on the self-mixing effect for coherent sensing , 2015 .

[53]  J. Faist,et al.  Quantum Cascade Laser , 1994, Science.

[54]  Michaël Sicard,et al.  Silicon photomultiplier detector for atmospheric lidar applications. , 2012, Optics letters.

[55]  Victor M. Brea,et al.  Distance Measurement Error in Time-of-Flight Sensors Due to Shot Noise , 2015, Sensors.

[56]  HoraudRadu,et al.  An overview of depth cameras and range scanners based on time-of-flight technologies , 2016 .

[57]  I. Kelson,et al.  Strongly pumped fiber lasers , 1998 .

[58]  Santiago Royo,et al.  Frequency-Modulated Optical Feedback Interferometry for Nanometric Scale Vibrometry , 2016, IEEE Photonics Technology Letters.

[59]  R. Lange,et al.  Solid-state time-of-flight range camera , 2001 .

[60]  William B. Spillman,et al.  Fiber optic sensors : an introduction for engineers and scientists , 2011 .

[61]  Michael Himmelsbach,et al.  LIDAR-based 3D Object Perception , 2008 .

[62]  Ming C. Wu,et al.  Lidar System Architectures and Circuits , 2017, IEEE Communications Magazine.

[63]  J. Zayhowski,et al.  Diode-pumped passively Q-switched picosecond microchip lasers. , 1994, Optics letters.

[64]  Paul F. McManamon,et al.  Review of ladar: a historic, yet emerging, sensor technology with rich phenomenology , 2012 .

[65]  Seth Johnson,et al.  Liquid crystal waveguides: new devices enabled by >1000 waves of optical phase control , 2010, OPTO.

[66]  J J Zayhowski Q-switched operation of microchip lasers. , 1991, Optics letters.

[67]  M. Amann,et al.  Laser ranging: a critical review of usual techniques for distance measurement , 2001 .

[68]  Jun Zhang,et al.  Fully integrated free-running InGaAs/InP single-photon detector for accurate lidar applications. , 2017, Optics express.

[69]  Yongwan Park,et al.  Mutual Interference on Mobile Pulsed Scanning LIDAR , 2017 .

[70]  J. Zayhowski,et al.  Diode-pumped microchip lasers electro-optically Q switched at high pulse repetition rates. , 1992, Optics letters.

[71]  Barry L. Stann,et al.  Low-cost compact MEMS scanning ladar system for robotic applications , 2012, Other Conferences.

[72]  B. Culshaw,et al.  Precision time domain reflectometry in optical fiber systems using a frequency modulated continuous wave ranging technique , 1985 .

[73]  Paul F. McManamon,et al.  LiDAR Technologies and Systems , 2019 .

[74]  A. Kersey,et al.  Bragg grating-based laser sensors systems with interferometric interrogation and wavelength division multiplexing , 1995 .

[75]  J. Młyńczak,et al.  Practical application of pulsed “eye-safe” microchip laser to laser rangefinders , 2013 .

[76]  George M. Williams Optimization of eyesafe avalanche photodiode lidar for automobile safety and autonomous navigation systems , 2017 .

[77]  Jochen Frey,et al.  Robust 3 D Measurement with PMD Sensors , 2005 .

[78]  Klaus Petermann,et al.  Solid-state lasers: status and future , 2010 .

[79]  Evan Ackerman,et al.  Hail, robo-taxi! [Top Tech 2017] , 2017, IEEE Spectrum.

[80]  S. Cova,et al.  Monolithic active-quenching and active-reset circuit for single-photon avalanche detectors , 2003, IEEE J. Solid State Circuits.

[81]  Scott P. Cryan,et al.  A Survey of LIDAR Technology and Its Use in Spacecraft Relative Navigation , 2013 .

[82]  Andriyan Bayu Suksmono,et al.  Stationary and moving targets detection on FMCW radar using GNU radio-based software defined radio , 2015, 2015 International Symposium on Intelligent Signal Processing and Communication Systems (ISPACS).

[83]  Aloysius Wehr,et al.  Airborne laser scanning—an introduction and overview , 1999 .

[84]  J. Hecht Lidar for Self-Driving Cars , 2018 .

[85]  A T Young,et al.  Use of photomultiplier tubes for photon counting. , 1971, Applied optics.

[86]  Risto A. Myllylae,et al.  Comparison of continuous-wave and pulsed time-of-flight laser range-finding techniques , 1992, Other Conferences.

[87]  Andreas Kolb,et al.  Pulse Based Time-of-Flight Range Sensing , 2018, Sensors.

[88]  H. Kogelnik,et al.  Coupled‐Wave Theory of Distributed Feedback Lasers , 1972 .

[89]  Takahide Mizuno,et al.  Study of two-dimensional scanning LIDAR for planetary explorer , 2008, Remote Sensing.

[90]  Myoungho Sunwoo,et al.  Development of Autonomous Car—Part I: Distributed System Architecture and Development Process , 2014, IEEE Transactions on Industrial Electronics.

[91]  Rajeev Thakur,et al.  Scanning LIDAR in Advanced Driver Assistance Systems and Beyond: Building a road map for next-generation LIDAR technology , 2016, IEEE Consumer Electronics Magazine.

[92]  Paul F. McManamon,et al.  Transmissive beam steering through electrowetting microprism arrays , 2010 .

[93]  R. Poprawe,et al.  High power diode lasers : technology and applications , 2007 .

[94]  Hakan Urey,et al.  Optical performance requirements for MEMS-scanner-based microdisplays , 2000, SPIE MOEMS-MEMS.

[95]  Kenichi Iga,et al.  Surface emitting semiconductor lasers , 1988 .

[96]  Yong Wang,et al.  Thermal effects in kilowatt fiber lasers , 2004, IEEE Photonics Technology Letters.

[97]  H. Melchior,et al.  Photodetectors for optical communication systems , 1970 .

[98]  Denise Wilson,et al.  A review of photodetectors for sensing light-emitting reporters in biological systems , 2003 .

[99]  Brian C. Redman,et al.  Flash lidar based on multiple-slit streak tube imaging lidar , 2002, SPIE Defense + Commercial Sensing.

[100]  Bertrand Douillard,et al.  On the segmentation of 3D LIDAR point clouds , 2011, 2011 IEEE International Conference on Robotics and Automation.

[101]  Reuben R. Rohrschneider,et al.  An Overview of Ball Flash LIDAR and Related Technology Development , 2013 .

[102]  Adrian Fiergolski,et al.  An Optical Interference Suppression Scheme for TCSPC Flash LiDAR Imagers , 2019, Applied Sciences.

[103]  A. Yalçinkaya,et al.  Two-axis electromagnetic microscanner for high resolution displays , 2006, Journal of Microelectromechanical Systems.

[104]  Qi Chen,et al.  LiDAR Remote Sensing and Applications , 2017 .

[105]  Robert Weigel,et al.  Challenges in miniaturized automotive long-range lidar system design , 2017, Commercial + Scientific Sensing and Imaging.

[106]  Michael Wahl,et al.  Time-Correlated Single Photon Counting , 2009 .

[107]  W. W. Chow,et al.  Semiconductor-Laser Fundamentals: Physics of the Gain Materials , 1999 .

[108]  Klaus Petermann,et al.  Solid-state lasers: status and future [Invited] , 2010 .

[109]  R. Baets,et al.  Off-chip beam steering with a one-dimensional optical phased array on silicon-on-insulator. , 2009, Optics letters.

[110]  S. Foix,et al.  Lock-in Time-of-Flight (ToF) Cameras: A Survey , 2011, IEEE Sensors Journal.

[111]  K. Sato,et al.  Development of Multi-Pixel Photon Counter (MPPC) , 2006, 2007 IEEE Nuclear Science Symposium Conference Record.

[112]  V. Gapontsev,et al.  100 kW CW fiber laser for industrial applications , 2014, 2014 International Conference Laser Optics.

[113]  H. Urey,et al.  High-resolution beam steering using microlens arrays. , 2006, Optics letters.

[114]  T. Ebisuzaki,et al.  A New LIDAR Method using MEMS Micromirror Array for the JEM-EUSO mission , 2009 .

[115]  Georg Schitter,et al.  MEMS-based lidar for autonomous driving , 2018, Elektrotech. Informationstechnik.

[116]  Ling Shao,et al.  Enhanced Computer Vision With Microsoft Kinect Sensor: A Review , 2013, IEEE Transactions on Cybernetics.

[117]  Stephen C. Cain,et al.  Direct-Detection Ladar Systems , 2010 .

[118]  L. Coldren,et al.  Diode Lasers and Photonic Integrated Circuits , 1995 .

[119]  Rita Mahon,et al.  Two-dimensional beam steering using a thermo-optic silicon photonic optical phased array , 2016 .

[120]  Michael A. Lefsky,et al.  The Electronically Steerable Flash Lidar: A Full Waveform Scanning System for Topographic and Ecosystem Structure Applications , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[121]  Radu Horaud,et al.  Time-of-Flight Cameras: Principles, Methods and Applications , 2012 .

[122]  Chao-I Chen,et al.  3D flash LIDAR vision systems for imaging in degraded visual environments , 2014, Defense + Security Symposium.