Virtual Radar: Real-Time Millimeter-Wave Radar Sensor Simulation for Perception-Driven Robotics

This article presents ViRa<xref ref-type="fn" rid="fn1"><sup>1</sup></xref><fn id="fn1"><label><sup>1</sup></label><p>Available online: <uri>https://vira.aau.at/</uri>.</p></fn>, a real-time open-source millimeter-wave radar simulation framework for perception-driven robotic applications. ViRa provides <inline-formula><tex-math notation="LaTeX">$(i)$</tex-math></inline-formula> raw data of radar sensors in real-time, simulation of <inline-formula><tex-math notation="LaTeX">$(ii)$</tex-math></inline-formula> multi-antenna configurations for spatial estimation of objects, <inline-formula><tex-math notation="LaTeX">$(iii)$</tex-math></inline-formula> wave penetration of non-conductive objects to infer information in occluded situations, <inline-formula><tex-math notation="LaTeX">$(iv)$</tex-math></inline-formula> different radar beam patterns, and, <inline-formula><tex-math notation="LaTeX">$(v)$</tex-math></inline-formula> configurations of radar sensors as given by real-world radars. By using ViRa, researchers can simulate radar sensors in different robotic scenarios and investigate radars prior to the installation. This allows an acceleration in the development of radar sensors for robotic applications without the need of real hardware. Contrary to simple model abstractions, which only output loose features, ViRa generates raw radar data using computer graphics techniques on graphics processing unit (GPU) level embedded inside a game engine environment. ViRa allows to feed data directly into machine learning frameworks, which enables further improvement in novel research directions, such as safe human-robot interaction or agile drone flights in obstacle-rich environments. The proposed simulation framework is validated with data from different scenarios in robotics such as human tracking for human-robot interaction. The obtained results are compared with a reference simulation framework and show significantly higher correlation when compared to real-world measurement data.

[1]  Andrew Kramer,et al.  Radar-Inertial Ego-Velocity Estimation for Visually Degraded Environments , 2020, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[2]  Ingmar Posner,et al.  Under the Radar: Learning to Predict Robust Keypoints for Odometry Estimation and Metric Localisation in Radar , 2020, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[3]  Jitendra Malik,et al.  Habitat: A Platform for Embodied AI Research , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[4]  PowerFlex® 700H.  TECHNICAL DATA , 1949 .

[5]  Marcel Hoffmann,et al.  Pedestrian Classification with a 79 GHz Automotive Radar Sensor , 2018, 2018 19th International Radar Symposium (IRS).

[6]  Yeong Sang Park,et al.  PhaRaO: Direct Radar Odometry using Phase Correlation , 2020, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[7]  Matteo Lucchi,et al.  robo-gym – An Open Source Toolkit for Distributed Deep Reinforcement Learning on Real and Simulated Robots , 2020, 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[8]  Alberto Garcia-Garcia,et al.  UnrealROX: an extremely photorealistic virtual reality environment for robotics simulations and synthetic data generation , 2018, Virtual Reality.

[9]  Silvio Savarese,et al.  SURREAL: Open-Source Reinforcement Learning Framework and Robot Manipulation Benchmark , 2018, CoRL.

[10]  Erwin Biebl,et al.  A ray launching approach for modeling an FMCW radar system , 2017, 2017 18th International Radar Symposium (IRS).

[11]  Andrew J. Davison,et al.  RLBench: The Robot Learning Benchmark & Learning Environment , 2019, IEEE Robotics and Automation Letters.

[12]  Murat Torlak,et al.  Automotive Radars: A review of signal processing techniques , 2017, IEEE Signal Processing Magazine.

[13]  Paul Newman,et al.  Radar as a Teacher: Weakly Supervised Vehicle Detection using Radar Labels , 2020, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[14]  Nojun Kwak,et al.  Human detection by Neural Networks using a low-cost short-range Doppler radar sensor , 2017, 2017 IEEE Radar Conference (RadarConf).

[15]  W. Wiesbeck,et al.  Capability of 3-D Ray Tracing for Defining Parameter Sets for the Specification of Future Mobile Communications Systems , 2006, IEEE Transactions on Antennas and Propagation.

[16]  Dariu M. Gavrila,et al.  CNN Based Road User Detection Using the 3D Radar Cube , 2020, IEEE Robotics and Automation Letters.

[17]  Jan Barowski,et al.  Millimeter-Wave Characterization of Dielectric Materials Using Calibrated FMCW Transceivers , 2018, IEEE Transactions on Microwave Theory and Techniques.

[18]  Chris Van Hoof,et al.  Vital-sign monitoring and spatial tracking of multiple people using a contactless radar-based sensor , 2019, Nature Electronics.

[19]  Arien P. Sligar,et al.  Machine Learning-Based Radar Perception for Autonomous Vehicles Using Full Physics Simulation , 2020, IEEE Access.

[20]  Eric Haines,et al.  Real-time rendering , 2018 .

[21]  Thomas Zwick,et al.  A system simulation of a 77 GHz phased array radar sensor , 2011, 2011 12th International Radar Symposium (IRS).

[22]  Morgan Quigley,et al.  ROS: an open-source Robot Operating System , 2009, ICRA 2009.

[23]  Bing Wang,et al.  Heart Rate Sensing with a Robot Mounted mmWave Radar , 2020, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[24]  Sertac Karaman,et al.  FlightGoggles: Photorealistic Sensor Simulation for Perception-driven Robotics using Photogrammetry and Virtual Reality , 2019, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[25]  Werner Wiesbeck,et al.  Extraction of scattering centers of vehicles by ray-tracing simulations , 2007 .

[26]  Paul Newman,et al.  Probably Unknown: Deep Inverse Sensor Modelling Radar , 2018, 2019 International Conference on Robotics and Automation (ICRA).

[27]  D.A. White,et al.  Investigation of radar propagation in buildings: A 10 billion element Cartesian-mesh FETD simulation , 2008, 2008 IEEE Antennas and Propagation Society International Symposium.

[28]  Dietmar Kissinger,et al.  A millimeter-wave FMCW radar system simulator for automotive applications including nonlinear component models , 2011, 2011 8th European Radar Conference.

[29]  Hubert Zangl,et al.  Radar Sensors in Collaborative Robotics: Fast Simulation and Experimental Validation , 2020, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[30]  Craig Warren,et al.  A Realistic FDTD Numerical Modeling Framework of Ground Penetrating Radar for Landmine Detection , 2016, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[31]  D. Scaramuzza,et al.  Flightmare: A Flexible Quadrotor Simulator , 2020, CoRL.

[32]  M. Nowicki,et al.  FOR LEAP MOTION CONTROLLER , 2014 .

[33]  Billie F. Spencer,et al.  Simulation of ground penetrating radar on dispersive media by a finite element time domain algorithm , 2019, Journal of Applied Geophysics.

[34]  Jürgen Dickmann,et al.  Radar-based Feature Design and Multiclass Classification for Road User Recognition , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).