MAVBench: Micro Aerial Vehicle Benchmarking

Unmanned Aerial Vehicles (UAVs) are getting closer to becoming ubiquitous in everyday life. Among them, Micro Aerial Vehicles (MAVs) have seen an outburst of attention recently, specifically in the area with a demand for autonomy. A key challenge standing in the way of making MAVs autonomous is that researchers lack the comprehensive understanding of how performance, power, and computational bottlenecks affect MAV applications. MAVs must operate under a stringent power budget, which severely limits their flight endurance time. As such, there is a need for new tools, benchmarks, and methodologies to foster the systematic development of autonomous MAVs. In this paper, we introduce the MAVBench' framework which consists of a closed-loop simulator and an end-to-end application benchmark suite. A closed-loop simulation platform is needed to probe and understand the intra-system (application data flow) and inter-system (system and environment) interactions in MAV applications to pinpoint bottlenecks and identify opportunities for hardware and software co-design and optimization. In addition to the simulator, MAVBench provides a benchmark suite, the first of its kind, consisting of a variety of MAV applications designed to enable computer architects to perform characterization and develop future aerial computing systems. Using our open source, end-to-end experimental platform, we uncover a hidden, and thus far unexpected compute to total system energy relationship in MAVs. Furthermore, we explore the role of compute by presenting three case studies targeting performance, energy and reliability. These studies confirm that an efficient system design can improve MAV's battery consumption by up to 1.8X.

[1]  Anass Benjebbour,et al.  Design considerations for a 5G network architecture , 2014, IEEE Communications Magazine.

[2]  Roland Siegwart,et al.  Receding Horizon "Next-Best-View" Planner for 3D Exploration , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[3]  M. Alam,et al.  Reliability analysis of phased-mission systems: a practical approach , 2006, RAMS '06. Annual Reliability and Maintainability Symposium, 2006..

[4]  Paul W. H. Chung,et al.  An efficient phased mission reliability analysis for autonomous vehicles , 2010, Reliab. Eng. Syst. Saf..

[5]  Xuan Zhang Reliable electrical systems for micro aerial vehicles and insect-scale robots , 2017 .

[6]  Alan L. Yuille,et al.  UnrealCV: Connecting Computer Vision to Unreal Engine , 2016, ECCV Workshops.

[7]  Bernard Ghanem,et al.  A Benchmark and Simulator for UAV Tracking , 2016, ECCV.

[8]  S. LaValle Rapidly-exploring random trees : a new tool for path planning , 1998 .

[9]  Wolfram Burgard,et al.  OctoMap: an efficient probabilistic 3D mapping framework based on octrees , 2013, Autonomous Robots.

[10]  AKHIL GUPTA,et al.  A Survey of 5G Network: Architecture and Emerging Technologies , 2015, IEEE Access.

[11]  Nidhi Kalra,et al.  Driving to Safety , 2016 .

[12]  Urbano Nunes,et al.  Platooning with DSRC-based IVC-enabled autonomous vehicles: Adding infrared communications for IVC reliability improvement , 2012, 2012 IEEE Intelligent Vehicles Symposium.

[13]  Nils J. Nilsson,et al.  A Formal Basis for the Heuristic Determination of Minimum Cost Paths , 1968, IEEE Trans. Syst. Sci. Cybern..

[14]  Hirohiko Suwa,et al.  An Emergency Medical Communications System by Low Altitude Platform at the Early Stages of a Natural Disaster in Indonesia , 2012, Journal of Medical Systems.

[15]  B. Faverjon,et al.  Probabilistic Roadmaps for Path Planning in High-Dimensional Con(cid:12)guration Spaces , 1996 .

[16]  Andrew Howard,et al.  Design and use paradigms for Gazebo, an open-source multi-robot simulator , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[17]  Giorgio C. Buttazzo,et al.  Energy-Aware Coverage Path Planning of UAVs , 2015, 2015 IEEE International Conference on Autonomous Robot Systems and Competitions.

[18]  Matthias Althoff,et al.  CommonRoad: Composable benchmarks for motion planning on roads , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[19]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[20]  Nicolas Mansard,et al.  Robot Motion Planning and Control: Is It More than a Technological Problem? , 2017, Geometric and Numerical Foundations of Movements.

[21]  Luca Carlone,et al.  Visual-Inertial Navigation Algorithm Development Using Photorealistic Camera Simulation in the Loop , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[22]  Michael F. P. O'Boyle,et al.  Introducing SLAMBench, a performance and accuracy benchmarking methodology for SLAM , 2014, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[23]  Min Chen,et al.  Accurate electrical battery model capable of predicting runtime and I-V performance , 2006, IEEE Transactions on Energy Conversion.

[24]  Vijay Kumar,et al.  High speed navigation for quadrotors with limited onboard sensing , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[25]  William Whittaker,et al.  Autonomous Driving in Traffic: Boss and the Urban Challenge , 2009, AI Mag..

[26]  Khaled M. Elbassioni,et al.  Flight Tour Planning with Recharging Optimization for Battery-operated Autonomous Drones , 2017, ArXiv.

[27]  Rui Caseiro,et al.  High-Speed Tracking with Kernelized Correlation Filters , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Chris Sanders,et al.  Team MIT Urban Challenge Technical Report , 2007 .

[29]  Ashish Kapoor,et al.  AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles , 2017, FSR.

[30]  Ali Farhadi,et al.  YOLO9000: Better, Faster, Stronger , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Juan D. Tardós,et al.  ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras , 2016, IEEE Transactions on Robotics.

[32]  Obbu Chandra Sekhar,et al.  Design and fabrication of coulomb counter for estimation of SOC of battery , 2016, 2016 IEEE International Conference on Power Electronics, Drives and Energy Systems (PEDES).

[33]  Joel S. Emer,et al.  The soft error problem: an architectural perspective , 2005, 11th International Symposium on High-Performance Computer Architecture.

[34]  Peter King,et al.  Odin: Team VictorTango's entry in the DARPA Urban Challenge , 2008, J. Field Robotics.

[35]  Oussama Khatib,et al.  Springer Handbook of Robotics , 2007, Springer Handbooks.

[36]  Shaojie Shen,et al.  VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator , 2017, IEEE Transactions on Robotics.