An Overview of Autonomous Vehicles Sensors and Their Vulnerability to Weather Conditions

Autonomous vehicles (AVs) rely on various types of sensor technologies to perceive the environment and to make logical decisions based on the gathered information similar to humans. Under ideal operating conditions, the perception systems (sensors onboard AVs) provide enough information to enable autonomous transportation and mobility. In practice, there are still several challenges that can impede the AV sensors’ operability and, in turn, degrade their performance under more realistic conditions that actually occur in the physical world. This paper specifically addresses the effects of different weather conditions (precipitation, fog, lightning, etc.) on the perception systems of AVs. In this work, the most common types of AV sensors and communication modules are included, namely: RADAR, LiDAR, ultrasonic, camera, and global navigation satellite system (GNSS). A comprehensive overview of their physical fundamentals, electromagnetic spectrum, and principle of operation is used to quantify the effects of various weather conditions on the performance of the selected AV sensors. This quantification will lead to several advantages in the simulation world by creating more realistic scenarios and by properly fusing responses from AV sensors in any object identification model used in AVs in the physical world. Moreover, it will assist in selecting the appropriate fading or attenuation models to be used in any X-in-the-loop (XIL, e.g., hardware-in-the-loop, software-in-the-loop, etc.) type of experiments to test and validate the manner AVs perceive the surrounding environment under certain conditions.

[1]  Jon Bjärkefur,et al.  Night vision animal detection , 2014, 2014 IEEE Intelligent Vehicles Symposium Proceedings.

[2]  Christian Steger,et al.  Next generation radar sensors in automotive sensor fusion systems , 2017, 2017 Sensor Data Fusion: Trends, Solutions, Applications (SDF).

[3]  Pedro J. Navarro,et al.  A Systematic Review of Perception System and Simulators for Autonomous Vehicles Research , 2019, Sensors.

[4]  Zygmunt Mierczyk,et al.  Comparison of 905 nm and 1550 nm semiconductor laser rangefinders’ performance deterioration due to adverse environmental conditions , 2014 .

[5]  Horst Mehl,et al.  GPS – Global Positioning System , 1996, Informatik-Spektrum.

[6]  V. Sreeja,et al.  Impact and mitigation of space weather effects on GNSS receiver performance , 2016, Geoscience Letters.

[7]  Onur Toker,et al.  A Python Based Testbed for Real-Time Testing and Visualization using TI's 77 GHz Automotive Radars , 2019, 2019 IEEE Vehicular Networking Conference (VNC).

[8]  Brian Y. Lattimer,et al.  Evaluation of Navigation Sensors in Fire Smoke Environments , 2014 .

[9]  Rahul Razdan,et al.  Conceptual Sensors Testing Framework for Autonomous Vehicles , 2018, 2018 IEEE Vehicular Networking Conference (VNC).

[10]  Keshav Bimbraw,et al.  Autonomous cars: Past, present and future a review of the developments in the last century, the present scenario and the expected future of autonomous vehicle technology , 2015, 2015 12th International Conference on Informatics in Control, Automation and Robotics (ICINCO).

[11]  Roger Lhermitte,et al.  Attenuation and Scattering of Millimeter Wavelength Radiation by Clouds and Precipitation , 1990 .

[13]  Peter V. Gehler,et al.  3D2PM - 3D Deformable Part Models , 2012, ECCV.

[14]  Xinghao Ding,et al.  Clearing the Skies: A Deep Network Architecture for Single-Image Rain Removal , 2016, IEEE Transactions on Image Processing.

[15]  Torsten Sattler,et al.  Obstacle detection for self-driving cars using only monocular cameras and wheel odometry , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[16]  Brendon Baker,et al.  Global Positioning System , 2010 .

[17]  Sergiu Nedevschi,et al.  Stereovision-Based Multiple Object Tracking in Traffic Scenarios Using Free-Form Obstacle Delimiters and Particle Filters , 2015, IEEE Transactions on Intelligent Transportation Systems.

[18]  Chuiqing Zeng,et al.  The impacts of environmental variables on water reflectance measured using a lightweight unmanned aerial vehicle (UAV)-based spectrometer system , 2017 .

[19]  Xiaohong Lu,et al.  Rain Backscattering Properties and Effects on the Radar Performance at MM Wave Band , 2001 .

[20]  L. Battan Radar Attenuation by Wet Ice Spheres , 1971 .

[21]  V. N. Pozhidaev,et al.  Estimation of attenuation and backscattering of millimeter radio waves in meteorological formations , 2010 .

[22]  Alexander Yarovoy,et al.  Analysis of rain clutter detections in commercial 77 GHz automotive radar , 2017, 2017 European Radar Conference (EURAD).

[23]  Erich Leitgeb,et al.  Spatial and Time Variability of Fog Attenuations for Optical Wireless Links in the Troposphere , 2009, 2009 IEEE 70th Vehicular Technology Conference Fall.

[24]  Hao Sun,et al.  Night Vision Pedestrian Detection Using a Forward-Looking Infrared Camera , 2011, 2011 International Workshop on Multi-Platform/Multi-Sensor Remote Sensing and Mapping.

[25]  Marin Marinov,et al.  A Study on Recent Developments and Issues with Obstacle Detection Systems for Automated Vehicles , 2020, Sustainability.

[26]  Seiya Nagaoka EVALUATION OF ATTENUATION OF ULTRASONIC WAVE IN AIR TO MEASURE CONCRETE ROUGHNESS USING AERIAL ULTRASONIC SENSOR , 2018 .

[27]  Nidhi Kalra,et al.  Driving to Safety , 2016 .

[28]  Ralph Helmar Rasshofer,et al.  Automotive Radar and Lidar Systems for Next Generation Driver Assistance Functions , 2005 .

[29]  Arturo de la Escalera,et al.  Far infrared pedestrian detection and tracking for night driving , 2011, Robotica.

[30]  Gennady P. Kulemin Influence of propagation effects on a millimeter-wave radar operation , 1999, Defense, Security, and Sensing.

[31]  Alexander Barth,et al.  Moving vehicle detection by optimal segmentation of the Dynamic Stixel World , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[32]  C. Lucianaz,et al.  77 GHz automotive anti-collision radar used for meteorological purposes , 2017, 2017 IEEE-APS Topical Conference on Antennas and Propagation in Wireless Communications (APWC).

[33]  Jiaolong Xu,et al.  Pedestrian Detection at Day/Night Time with Visible and FIR Cameras: A Comparison , 2016, Sensors.

[34]  Zheng Liu,et al.  Pedestrian detection in thermal images using adaptive fuzzy C-means clustering and convolutional neural networks , 2015, 2015 14th IAPR International Conference on Machine Vision Applications (MVA).

[35]  Wilhelm Stork,et al.  Weather Influence and Classification with Automotive Lidar Sensors , 2019, 2019 IEEE Intelligent Vehicles Symposium (IV).

[36]  Abu Bony Amin,et al.  Development of Sensors Testbed for Autonomous Vehicles , 2019, 2019 SoutheastCon.

[37]  Charles E. Thorpe,et al.  Sonar Based Outdoor Vehicle Navigation And Collision Avoidance , 1992, Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems.

[38]  Alebel Arage Hassen Indicators for the Signal Degradation and Optimization of Automotive Radar Sensors Under Adverse Weather Conditions , 2007 .

[39]  Mohamed Ali Kaafar,et al.  The Impact of Adverse Weather Conditions on Autonomous Vehicles: How Rain, Snow, Fog, and Hail Affect the Performance of a Self-Driving Car , 2019, IEEE Vehicular Technology Magazine.

[40]  Thomas B. Moeslund,et al.  Thermal cameras and applications: a survey , 2013, Machine Vision and Applications.

[41]  Zdenek Kalal,et al.  Tracking-Learning-Detection , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  Yi-Ping Hung,et al.  Tracking by Parts: A Bayesian Approach With Component Collaboration , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[43]  Ian D. Reid,et al.  PWP3D: Real-Time Segmentation and Tracking of 3D Objects , 2012, International Journal of Computer Vision.

[44]  Mokrane Hadj-Bachir,et al.  LIDAR sensor simulation in adverse weather condition for driving assistance development , 2019 .

[45]  H. B. Wallace,et al.  Millimeter-wave propagation measurements at the Ballistic Research Laboratory , 1988 .

[46]  Ismail Gultepe,et al.  Fog and Boundary Layer Clouds: Fog Visibility and Forecasting , 2007 .