Robust navigation with tinyML for autonomous mini-vehicles

Autonomous navigation vehicles have rapidly improved thanks to the breakthroughs of Deep Learning. However, scaling autonomous driving to low-power and real-time systems deployed on dynamic environments poses several challenges that prevent their adoption. In this work, we show an end-to-end integration of data, algorithms, and deployment tools that enables the deployment of a family of tiny-CNNs on extra-low-power MCUs for autonomous driving mini-vehicles (image classification task). Our end-to-end environment enables a closed-loop learning system that allows the CNNs (learners) to learn through demonstration by imitating the original computer-vision algorithm (teacher) while doubling the throughput. Thereby, our CNNs gain robustness to lighting conditions and increase their accuracy up to 20% when deployed in the most challenging setup with a very fast-rate camera. Further, we leverage GAP8, a parallel ultra-low-power RISC-V SoC, to meet the real-time requirements. When running a family of CNN for an image classification task, GAP8 reduces their latency by over 20x compared to using an STM32L4 (Cortex-M4) or obtains +21.4% accuracy than an NXP k64f (Cortex-M4) solution with the same energy budget.

[1]  Jelena Kocić,et al.  An End-to-End Deep Neural Network for Autonomous Driving Designed for Embedded Automotive Platforms , 2019, Sensors.

[2]  David Patterson,et al.  Benchmarking TinyML Systems: Challenges and Direction , 2020, ArXiv.

[3]  Luca Benini,et al.  GAP-8: A RISC-V SoC for AI at the Edge of the IoT , 2018, 2018 IEEE 29th International Conference on Application-specific Systems, Architectures and Processors (ASAP).

[4]  Paolo Napoletano,et al.  Benchmark Analysis of Representative Deep Neural Network Architectures , 2018, IEEE Access.

[5]  Onur Mutlu,et al.  Processing Data Where It Makes Sense in Modern Computing Systems: Enabling In-Memory Computation , 2018, ACM Great Lakes Symposium on VLSI.

[6]  Vikas Chandra,et al.  CMSIS-NN: Efficient Neural Network Kernels for Arm Cortex-M CPUs , 2018, ArXiv.

[7]  Luca Benini,et al.  A 64-mW DNN-Based Visual Navigation Engine for Autonomous Nano-Drones , 2018, IEEE Internet of Things Journal.

[8]  Luca Benini,et al.  Work-in-Progress: Quantized NNs as the Definitive Solution for Inference on Low-Power ARM MCUs? , 2018, 2018 International Conference on Hardware/Software Codesign and System Synthesis (CODES+ISSS).

[9]  Luca Benini,et al.  PULP-NN: accelerating quantized neural networks on parallel ultra-low-power RISC-V processors , 2019, Philosophical Transactions of the Royal Society A.

[10]  Aakanksha Chowdhery,et al.  Visual Wake Words Dataset , 2019, ArXiv.

[11]  Lawrence D. Jackel,et al.  Explaining How a Deep Neural Network Trained with End-to-End Learning Steers a Car , 2017, ArXiv.

[12]  Yundong Zhang,et al.  Hello Edge: Keyword Spotting on Microcontrollers , 2017, ArXiv.

[13]  Dean Pomerleau,et al.  ALVINN, an autonomous land vehicle in a neural network , 2015 .

[14]  Lorenzo Keller,et al.  Bonseyes AI Pipeline—Bringing AI to You , 2019, ACM Trans. Internet Things.

[15]  Byron Boots,et al.  Agile Autonomous Driving using End-to-End Deep Imitation Learning , 2017, Robotics: Science and Systems.

[16]  Davide Maltoni,et al.  Continuous Learning in Single-Incremental-Task Scenarios , 2018, Neural Networks.

[17]  Luca Benini,et al.  Memory-Driven Mixed Low Precision Quantization For Enabling Deep Network Inference On Microcontrollers , 2019, MLSys.

[18]  Houssam Abbas,et al.  F1/10: An Open-Source Autonomous Cyber-Physical Platform , 2019, ArXiv.

[19]  Richard Bowden,et al.  A Survey of Deep Learning Applications to Autonomous Vehicle Control , 2019, IEEE Transactions on Intelligent Transportation Systems.