Flightmare: A Flexible Quadrotor Simulator

State-of-the-art quadrotor simulators have a rigid and highly-specialized structure: either are they really fast, physically accurate, or photo-realistic. In this work, we propose a novel quadrotor simulator: Flightmare. Flightmare is composed of two main components: a configurable rendering engine built on Unity and a flexible physics engine for dynamics simulation. Those two components are totally decoupled and can run independently of each other. This makes our simulator extremely fast: rendering achieves speeds of up to 230 Hz, while physics simulation of up to 200,000 Hz on a laptop. In addition, Flightmare comes with several desirable features: (i) a large multi-modal sensor suite, including an interface to extract the 3D point-cloud of the scene; (ii) an API for reinforcement learning which can simulate hundreds of quadrotors in parallel; and (iii) integration with a virtual-reality headset for interaction with the simulated environment. We demonstrate the flexibility of Flightmare by using it for two different robotic tasks: quadrotor control using deep reinforcement learning and collision-free path planning in a complex 3D environment.

[1]  Flavio Fontana,et al.  Automatic re-initialization and failure recovery for aggressive flight with a monocular vision-based quadrotor , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[2]  Roland Siegwart,et al.  RotorS—A Modular Gazebo MAV Simulator Framework , 2016 .

[3]  Germán Ros,et al.  CARLA: An Open Urban Driving Simulator , 2017, CoRL.

[4]  Sertac Karaman,et al.  FlightGoggles: Photorealistic Sensor Simulation for Perception-driven Robotics using Photogrammetry and Virtual Reality , 2019, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[5]  Marwan Mattar,et al.  Unity: A General Platform for Intelligent Agents , 2018, ArXiv.

[6]  Vladlen Koltun,et al.  Open3D: A Modern Library for 3D Data Processing , 2018, ArXiv.

[7]  Bernard Ghanem,et al.  Sim4CV: A Photo-Realistic Simulator for Computer Vision Applications , 2017, International Journal of Computer Vision.

[8]  Vladlen Koltun,et al.  Deep Drone Acrobatics , 2020, Robotics: Science and Systems.

[9]  L. Dagum,et al.  OpenMP: an industry standard API for shared-memory programming , 1998 .

[10]  Roland Siegwart,et al.  Mapping on the Fly: Real-Time 3D Dense Reconstruction, Digital Surface Map and Incremental Orthomosaic Generation for Unmanned Aerial Vehicles , 2017, FSR.

[11]  Antonio Franchi,et al.  Differential Flatness of Quadrotor Dynamics Subject to Rotor Drag for Accurate Tracking of High-Speed Trajectories , 2017, IEEE Robotics and Automation Letters.

[12]  Oskar von Stryk,et al.  Hector Open Source Modules for Autonomous Mapping and Navigation with Rescue Robots , 2013, RoboCup.

[13]  Jessica R. Cauchard,et al.  Collocated Human-Drone Interaction: Methodology and Approach Strategy , 2019, 2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[14]  Marco Hutter,et al.  Per-Contact Iteration Method for Solving Contact Dynamics , 2018, IEEE Robotics and Automation Letters.

[15]  Vladlen Koltun,et al.  Deep Drone Racing: From Simulation to Reality With Domain Randomization , 2019, IEEE Transactions on Robotics.

[16]  Davide Scaramuzza,et al.  Learning High-Level Policies for Model Predictive Control , 2020, 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[17]  Roland Siegwart,et al.  Control of a Quadrotor With Reinforcement Learning , 2017, IEEE Robotics and Automation Letters.

[18]  Sertac Karaman,et al.  The Blackbird UAV dataset , 2020, Int. J. Robotics Res..

[19]  Ashish Kapoor,et al.  AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles , 2017, FSR.

[20]  Titus Cieslewski,et al.  Rapid exploration with multi-rotors: A frontier selection method for high speed flight , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[21]  Yuval Tassa,et al.  MuJoCo: A physics engine for model-based control , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[22]  Antonio Sgorbissa,et al.  Real-Time Path Generation and Obstacle Avoidance for Multirotors: A Novel Approach , 2018, J. Intell. Robotic Syst..

[23]  M. Faessler,et al.  Thrust Mixing, Saturation, and Body-Rate Control for Accurate Aggressive Quadrotor Flight , 2017, IEEE Robotics and Automation Letters.

[24]  Lydia E. Kavraki,et al.  The Open Motion Planning Library , 2012, IEEE Robotics & Automation Magazine.

[25]  D. Scaramuzza,et al.  AlphaPilot: autonomous drone racing , 2020, Autonomous Robots.

[26]  Alec Radford,et al.  Proximal Policy Optimization Algorithms , 2017, ArXiv.

[27]  Andrew Howard,et al.  Design and use paradigms for Gazebo, an open-source multi-robot simulator , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).