Novel Arithmetics in Deep Neural Networks Signal Processing for Autonomous Driving: Challenges and Opportunities

This article focuses on the trends, opportunities, and challenges of novel arithmetic for deep neural network (DNN) signal processing, with particular reference to assisted- and autonomous driving applications. Due to strict constraints in terms of the latency, dependability, and security of autonomous driving, machine perception (i.e., detection and decision tasks) based on DNNs cannot be implemented by relying on remote cloud access. These tasks must be performed in real time in embedded systems on board the vehicle, particularly for the inference phase (considering the use of DNNs pretrained during an offline step). When developing a DNN computing platform, the choice of the computing arithmetic matters. Moreover, functional safe applications, such as autonomous driving, impose severe constraints on the effect that signal processing accuracy has on the final rate of wrong detection/decisions. Hence, after reviewing the different choices and tradeoffs concerning arithmetic, both in academia and industry, we highlight the issues in implementing DNN accelerators to achieve accurate and lowcomplexity processing of automotive sensor signals (the latter coming from diverse sources, such as cameras, radar, lidar, and ultrasonics). The focus is on both general-purpose operations massively used in DNNs, such as multiplying, accumulating, and comparing, and on specific functions, including, for example, sigmoid or hyperbolic tangents used for neuron activation.

[1]  Marc Peter Deisenroth,et al.  Deep Reinforcement Learning: A Brief Survey , 2017, IEEE Signal Processing Magazine.

[2]  Jan-Michael Frahm,et al.  Re-Thinking CNN Frameworks for Time-Sensitive Autonomous-Driving Applications: Addressing an Industrial Challenge , 2019, 2019 IEEE Real-Time and Embedded Technology and Applications Symposium (RTAS).

[3]  Li Deng,et al.  Artificial Intelligence in the Rising Wave of Deep Learning: The Historical Path and Future Outlook [Perspectives] , 2018, IEEE Signal Processing Magazine.

[4]  Sergio Saponara,et al.  Radar Sensor Signal Acquisition and Multidimensional FFT Processing for Surveillance Applications in Transport Systems , 2017, IEEE Transactions on Instrumentation and Measurement.

[5]  Satoshi Matsuoka,et al.  Hardware Implementation of POSITs and Their Application in FPGAs , 2018, 2018 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW).

[6]  Yoshua Bengio,et al.  BinaryConnect: Training Deep Neural Networks with binary weights during propagations , 2015, NIPS.

[7]  Anastasios Tefas,et al.  Deep Convolutional Feature Histograms for Visual Object Tracking , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[8]  George Papandreou,et al.  Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation , 2018, ECCV.

[9]  Quoc V. Le,et al.  Searching for MobileNetV3 , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[10]  Eriko Nurvitadhi,et al.  Accelerating Deep Convolutional Networks using low-precision and sparsity , 2016, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[11]  Hayden Kwok-Hay So,et al.  Universal number posit arithmetic generator on FPGA , 2018, 2018 Design, Automation & Test in Europe Conference & Exhibition (DATE).

[12]  Pritish Narayanan,et al.  Deep Learning with Limited Numerical Precision , 2015, ICML.

[13]  Hayden Kwok-Hay So,et al.  PACoGen: A Hardware Posit Arithmetic Core Generator , 2019, IEEE Access.

[14]  H. Peter Hofstee,et al.  An Accelerator for Posit Arithmetic Targeting Posit Level 1 BLAS Routines and Pair-HMM , 2019, Proceedings of the Conference for Next Generation Arithmetic 2019.

[15]  Anastasios Tefas,et al.  Convolutional Neural Networks for Visual Information Analysis with Limited Computing Resources , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[16]  Luca Benini,et al.  FlexFloat: A Software Library for Transprecision Computing , 2020, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[17]  Norbert Wehn,et al.  The transprecision computing paradigm: Concept, design, and applications , 2018, 2018 Design, Automation & Test in Europe Conference & Exhibition (DATE).

[18]  John L. Gustafson,et al.  Deep Positron: A Deep Neural Network Using the Posit Number System , 2018, 2019 Design, Automation & Test in Europe Conference & Exhibition (DATE).

[19]  Zhihua Li,et al.  Survey on encoding schemes for genomic data representation and feature learning - from signal processing to machine learning , 2018, Big Data Min. Anal..

[20]  Bo Chen,et al.  Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[21]  Xin Wang,et al.  Flexpoint: An Adaptive Numerical Format for Efficient Training of Deep Neural Networks , 2017, NIPS.

[22]  Benoît Dupont de Dinechin.  Consolidating High-Integrity, High-Performance, and Cyber-Security Functions on a Manycore Processor , 2019, DAC.

[23]  Ran El-Yaniv,et al.  Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations , 2016, J. Mach. Learn. Res..

[24]  U. Kulisch.  An axiomatic approach to rounded computations , 1971 .

[25]  Michael Unser,et al.  Convolutional Neural Networks for Inverse Problems in Imaging: A Review , 2017, IEEE Signal Processing Magazine.

[26]  Lucia Lo Bello,et al.  Recent Advances and Trends in On-Board Embedded and Networked Automotive Systems , 2019, IEEE Transactions on Industrial Informatics.

[27]  Preeti Nagrath,et al.  Detection of Emergency Vehicles Using Modified Yolo Algorithm , 2020 .

[28]  Sergio Saponara,et al.  Exploiting Posit Arithmetic for Deep Neural Networks in Autonomous Driving Applications , 2018, 2018 International Conference of Electrical and Electronic Technologies for Automotive.

[29]  Nikolaos Doulamis,et al.  Deep Learning for Computer Vision: A Brief Review , 2018, Comput. Intell. Neurosci..

[30]  Michael J. Schulte,et al.  The interval logarithmic number system , 2003, Proceedings 2003 16th IEEE Symposium on Computer Arithmetic.

[31]  Baris Fidan,et al.  Localization for Autonomous Driving , 2019, Handbook of Position Location.

[32]  Francisco J. Cazorla,et al.  Assessing the Adherence of an Industrial Autonomous Driving Framework to ISO 26262 Software Guidelines , 2019, 2019 56th ACM/IEEE Design Automation Conference (DAC).

[33]  Nicolas Brunie,et al.  Modified Fused Multiply and Add for Exact Low Precision Product Accumulation , 2017, 2017 IEEE 24th Symposium on Computer Arithmetic (ARITH).

[34]  Pierre Vandergheynst,et al.  Geometric Deep Learning: Going beyond Euclidean data , 2016, IEEE Signal Process. Mag..

[35]  Sergio Saponara,et al.  Fast Approximations of Activation Functions in Deep Neural Networks when using Posit Arithmetic , 2020, Sensors.

[36]  Johannes Stallkamp,et al.  Man vs. computer: Benchmarking machine learning algorithms for traffic sign recognition , 2012, Neural Networks.

[37]  Xin Wang,et al.  Flexpoint: Predictive Numerics for Deep Learning , 2018, 2018 IEEE 25th Symposium on Computer Arithmetic (ARITH).

[38]  Yu Fan,et al.  Lidar Image Classification Based on Convolutional Neural Networks , 2017, 2017 International Conference on Computer Network, Electronic and Automation (ICCNEA).

[39]  Visar Berisha,et al.  Joint Optimization of Quantization and Structured Sparsity for Compressed Deep Neural Networks , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[40]  Dong Xu,et al.  Advanced Deep-Learning Techniques for Salient and Category-Specific Object Detection: A Survey , 2018, IEEE Signal Processing Magazine.

[41]  Carsten Rother,et al.  Conditional Random Fields Meet Deep Neural Networks for Semantic Segmentation: Combining Probabilistic Graphical Models with Deep Learning for Structured Prediction , 2018, IEEE Signal Processing Magazine.

[42]  Lina J. Karam,et al.  Quality Robust Mixtures of Deep Neural Networks , 2018, IEEE Transactions on Image Processing.

[43]  Tao Zhang,et al.  Model Compression and Acceleration for Deep Neural Networks: The Principles, Progress, and Challenges , 2018, IEEE Signal Processing Magazine.

[44]  Dominik Reinhardt,et al.  High Performance Processor Architecture for Automotive Large Scaled Integrated Systems within the European Processor Initiative Research Project , 2019 .

[45]  John L. Gustafson,et al.  Performance-Efficiency Trade-off of Low-Precision Numerical Formats in Deep Neural Networks , 2019, Proceedings of the Conference for Next Generation Arithmetic 2019.

[46]  Sergio Saponara,et al.  Novel Arithmetics to Accelerate Machine Learning Classifiers in Autonomous Driving Applications , 2019, 2019 26th IEEE International Conference on Electronics, Circuits and Systems (ICECS).

[47]  Van-Dung Hoang,et al.  Improving Traffic Signs Recognition Based Region Proposal and Deep Neural Networks , 2018, ACIIDS.

[48]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[49]  Rainer Leupers,et al.  Parameterized Posit Arithmetic Hardware Generator , 2018, 2018 IEEE 36th International Conference on Computer Design (ICCD).

[50]  John L. Gustafson,et al.  Beating Floating Point at its Own Game: Posit Arithmetic , 2017, Supercomput. Front. Innov..

[51]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[52]  Taxonomy and definitions for terms related to driving automation systems for on-road motor vehicles , 2022 .

[53]  Sebastian Ramos,et al.  The Cityscapes Dataset for Semantic Urban Scene Understanding , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[54]  Jeff Johnson,et al.  Rethinking floating point for deep learning , 2018, ArXiv.

[55]  Peter Lindstrom.  Universal Coding of the Reals using Bisection , 2019, CoNGA'19.

[56]  Patrizio Pelliccione,et al.  Autonomous Vehicles: State of the Art, Future Trends, and Challenges , 2019, Automotive Systems and Software Engineering.

[57]  Sergio Saponara,et al.  A Fast Approximation of the Hyperbolic Tangent When Using Posit Numbers and Its Application to Deep Neural Networks , 2019, ApplePies.