AutoPilot: Automating SoC Design Space Exploration for SWaP Constrained Autonomous UAVs

Building domain-specific accelerators for autonomous unmanned aerial vehicles (UAVs) is challenging due to a lack of systematic methodology for designing onboard compute. Balancing a computing system for a UAV requires considering both the cyber (e.g., sensor rate, compute performance) and physical (e.g., payload weight) characteristics that affect overall performance. Iterating over the many component choices results in a combinatorial explosion of the number of possible combinations: from 10s of thousands to billions, depending on implementation details. Manually selecting combinations of these components is tedious and expensive. To navigate the cyber-physical design space efficiently, we introduce AutoPilot, a framework that automates full-system UAV co-design. AutoPilot uses Bayesian optimization to navigate a large design space and automatically select a combination of autonomy algorithm and hardware accelerator while considering the cross-product effect of other cyber and physical UAV components. We show that the AutoPilot methodology consistently outperforms general-purpose hardware selections like Xavier NX and Jetson TX2, as well as dedicated hardware accelerators built for autonomous UAVs, across a range of representative scenarios (three different UAV types and three deployment environments). Designs generated by AutoPilot increase the number of missions on average by up to 2.25×, 1.62×, and 1.43× for nano, micro, and mini-UAVs respectively over baselines. Our work demonstrates the need for holistic full-UAV co-design to achieve maximum overall UAV performance and the need for automated flows to simplify the design process for autonomous cyber-physical systems.

[1]  Jasper Snoek,et al.  Practical Bayesian Optimization of Machine Learning Algorithms , 2012, NIPS.

[2]  Alberto Elfes,et al.  Using occupancy grids for mobile robot perception and navigation , 1989, Computer.

[3]  Vijay Janapa Reddi,et al.  Sniffy Bug: A Fully Autonomous Swarm of Gas-Seeking Nano Quadcopters in Cluttered Environments , 2021, 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[4]  Luca Benini,et al.  A 64-mW DNN-Based Visual Navigation Engine for Autonomous Nano-Drones , 2018, IEEE Internet of Things Journal.

[5]  Yu Wang,et al.  Robotic Computing on FPGAs , 2021, Synthesis Lectures on Computer Architecture.

[6]  Nikolai Smolyanskiy,et al.  Toward low-flying autonomous MAV trail navigation using deep neural networks for environmental awareness , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[7]  Carlos R. del-Blanco,et al.  DroNet: Learning to Fly by Driving , 2018, IEEE Robotics and Automation Letters.

[8]  Gu-Yeon Wei,et al.  A case for efficient accelerator design space exploration via Bayesian optimization , 2017, 2017 IEEE/ACM International Symposium on Low Power Electronics and Design (ISLPED).

[9]  Yu Wang,et al.  A Survey of FPGA-Based Robotic Computing , 2020, IEEE Circuits and Systems Magazine.

[10]  Paul M Ness,et al.  Drone transportation of blood products , 2017, Transfusion.

[11]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[12]  George Konidaris,et al.  Robot Motion Planning on a Chip , 2016, Robotics: Science and Systems.

[13]  Frank Hutter,et al.  Neural Architecture Search: A Survey , 2018, J. Mach. Learn. Res..

[14]  Xin Zhang,et al.  End to End Learning for Self-Driving Cars , 2016, ArXiv.

[15]  Luca Carlone,et al.  Navion: A 2-mW Fully Integrated Real-Time Visual-Inertial Odometry Accelerator for Autonomous Navigation of Nano Drones , 2018, IEEE Journal of Solid-State Circuits.

[16]  Stefania Matteoli,et al.  Smart farming: Opportunities, challenges and technology enablers , 2018, 2018 IoT Vertical and Topical Summit on Agriculture - Tuscany (IOT Tuscany).

[17]  Jürgen Schmidhuber,et al.  A Machine Learning Approach to Visual Perception of Forest Trails for Mobile Robots , 2016, IEEE Robotics and Automation Letters.

[18]  Yao Zhang,et al.  Autonomous Flight Control of a Nano Quadrotor Helicopter in a GPS-Denied Environment Using On-Board Vision , 2015, IEEE Transactions on Industrial Electronics.

[19]  David Janz,et al.  Learning to Drive in a Day , 2018, 2019 International Conference on Robotics and Automation (ICRA).

[20]  Lydia Tapia,et al.  PRM-RL: Long-range Robotic Navigation Tasks by Combining Reinforcement Learning and Sampling-Based Planning , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[21]  Davide Scaramuzza,et al.  How Fast Is Too Fast? The Role of Perception Latency in High-Speed Sense and Avoid , 2019, IEEE Robotics and Automation Letters.

[22]  Aleksandra Faust,et al.  Air Learning: a deep reinforcement learning gym for autonomous aerial robot visual navigation , 2021, Mach. Learn..

[23]  Gerd Hirzinger,et al.  Energy-efficient Autonomous Four-rotor Flying Robot Controlled at 1 kHz , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[24]  Radu Bogdan Rusu,et al.  3D is here: Point Cloud Library (PCL) , 2011, 2011 IEEE International Conference on Robotics and Automation.

[25]  Srinivas Devadas,et al.  Robomorphic computing: a design methodology for domain-specific accelerators parameterized by robot morphology , 2021, ASPLOS.

[26]  Jung Ho Ahn,et al.  CACTI-P: Architecture-level modeling for SRAM-based structures with advanced leakage reduction techniques , 2011, 2011 IEEE/ACM International Conference on Computer-Aided Design (ICCAD).

[27]  Nando de Freitas,et al.  Taking the Human Out of the Loop: A Review of Bayesian Optimization , 2016, Proceedings of the IEEE.

[28]  Hadi Esmaeilzadeh,et al.  RoboX: An End-to-End Solution to Accelerate Autonomous Control in Robotics , 2018, 2018 ACM/IEEE 45th Annual International Symposium on Computer Architecture (ISCA).

[29]  Alberto Gonzalez-Sanchez,et al.  Estimation of vegetation fraction using RGB and multispectral images from UAV , 2018, International Journal of Remote Sensing.

[30]  H.-S. Philip Wong,et al.  On-Chip Memory Technology Design Space Explorations for Mobile Deep Neural Network Accelerators , 2019, 2019 56th ACM/IEEE Design Automation Conference (DAC).

[31]  Amir Yazdanbakhsh,et al.  Apollo: Transferable Architecture Exploration , 2021, ArXiv.

[32]  Guido C. H. E. de Croon,et al.  Tiny Robot Learning (tinyRL) for Source Seeking on a Nano Quadcopter , 2021, 2021 IEEE International Conference on Robotics and Automation (ICRA).

[33]  Arijit Raychowdhury,et al.  iELAS: An ELAS-Based Energy-Efficient Accelerator for Real-Time Stereo Matching on FPGA Platform , 2021, 2021 IEEE 3rd International Conference on Artificial Intelligence Circuits and Systems (AICAS).

[34]  Martin Buss,et al.  Visual tracking and control of a quadcopter using a stereo camera system and inertial sensors , 2009, 2009 International Conference on Mechatronics and Automation.

[35]  George Konidaris,et al.  The microarchitecture of a real-time robot motion planning accelerator , 2016, 2016 49th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO).

[36]  Sergey Levine,et al.  QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation , 2018, CoRL.

[37]  Hugh F. Durrant-Whyte,et al.  A solution to the simultaneous localization and map building (SLAM) problem , 2001, IEEE Trans. Robotics Autom..

[38]  Wojciech Zaremba,et al.  Domain randomization for transferring deep neural networks from simulation to the real world , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[39]  Gu-Yeon Wei,et al.  The Sky Is Not the Limit: A Visual Performance Model for Cyber-Physical Co-Design in Autonomous Machines , 2020, IEEE Computer Architecture Letters.

[40]  Emilio Frazzoli,et al.  Sampling-based algorithms for optimal motion planning , 2011, Int. J. Robotics Res..

[41]  Suki Kim,et al.  A multi-lane MIPI CSI receiver for mobile camera applications , 2010, IEEE Transactions on Consumer Electronics.

[42]  David González,et al.  A Review of Motion Planning Techniques for Automated Vehicles , 2016, IEEE Transactions on Intelligent Transportation Systems.

[43]  Vijay Kumar,et al.  High speed navigation for quadrotors with limited onboard sensing , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[44]  Sergey Levine,et al.  (CAD)$^2$RL: Real Single-Image Flight without a Single Real Image , 2016, Robotics: Science and Systems.

[45]  Arijit Raychowdhury,et al.  Autonomous Navigation via Deep Reinforcement Learning for Resource Constraint Edge Nodes Using Transfer Learning , 2019, IEEE Access.

[46]  Martial Hebert,et al.  Learning monocular reactive UAV control in cluttered natural environments , 2012, 2013 IEEE International Conference on Robotics and Automation.

[47]  Matthew Mattina,et al.  A Systematic Methodology for Characterizing Scalability of DNN Accelerators using SCALE-Sim , 2020, 2020 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS).

[48]  S. Lange,et al.  Autonomous Corridor Flight of a UAV Using a Low-Cost and Light-Weight RGB-D Camera , 2012 .

[49]  Azer Bestavros,et al.  Neuroflight: Next Generation Flight Control Firmware , 2019, ArXiv.

[50]  Bahar Asgari,et al.  Quantifying the design-space tradeoffs in autonomous drones , 2021, ASPLOS.

[51]  Vijay Kumar,et al.  Fast, autonomous flight in GPS‐denied and cluttered environments , 2017, J. Field Robotics.

[52]  Ya Wang,et al.  UAV navigation in high dynamic environments: A deep reinforcement learning approach , 2020 .

[53]  Wenzhi Cui,et al.  MAVBench: Micro Aerial Vehicle Benchmarking , 2018, 2018 51st Annual IEEE/ACM International Symposium on Microarchitecture (MICRO).

[54]  Wolfgang Ponweiser,et al.  Multiobjective Optimization on a Limited Budget of Evaluations Using Model-Assisted -Metric Selection , 2008, PPSN.

[55]  Christina Delimitrou,et al.  HiveMind: A Scalable and Serverless Coordination Control Platform for UAV Swarms , 2020, ArXiv.