Bonseyes AI Pipeline—Bringing AI to You

Next generation of embedded Information and Communication Technology (ICT) systems are interconnected and collaborative systems able to perform autonomous tasks. The remarkable expansion of the embedded ICT market, together with the rise and breakthroughs of Artificial Intelligence (AI), have put the focus on the Edge as it stands as one of the keys for the next technological revolution: the seamless integration of AI in our daily life. However, training and deployment of custom AI solutions on embedded devices require a fine-grained integration of data, algorithms, and tools to achieve high accuracy and overcome functional and non-functional requirements. Such integration requires a high level of expertise that becomes a real bottleneck for small and medium enterprises wanting to deploy AI solutions on the Edge, which, ultimately, slows down the adoption of AI on applications in our daily life. In this work, we present a modular AI pipeline as an integrating framework to bring data, algorithms, and deployment tools together. By removing the integration barriers and lowering the required expertise, we can interconnect the different stages of particular tools and provide a modular end-to-end development of AI products for embedded devices. Our AI pipeline consists of four modular main steps: (i) data ingestion, (ii) model training, (iii) deployment optimization, and (iv) the IoT hub integration. To show the effectiveness of our pipeline, we provide examples of different AI applications during each of the steps. Besides, we integrate our deployment framework, Low-Power Deep Neural Network (LPDNN), into the AI pipeline and present its lightweight architecture and deployment capabilities for embedded devices. Finally, we demonstrate the results of the AI pipeline by showing the deployment of several AI applications such as keyword spotting, image classification, and object detection on a set of well-known embedded platforms, where LPDNN consistently outperforms all other popular deployment frameworks.

[1]  David Gregg,et al.  Performance-Oriented Neural Architecture Search , 2019, 2019 International Conference on High Performance Computing & Simulation (HPCS).

[2]  François Chollet,et al.  Keras: The Python Deep Learning library , 2018 .

[3]  Muhammad Shafique,et al.  An overview of next-generation architectures for machine learning: Roadmap, opportunities and challenges in the IoT era , 2018, 2018 Design, Automation & Test in Europe Conference & Exhibition (DATE).

[4]  Marshall Copeland,et al.  Microsoft Azure , 2015, Apress.

[5]  George Tzanetakis,et al.  Musical genre classification of audio signals , 2002, IEEE Trans. Speech Audio Process..

[6]  Colin Raffel,et al.  librosa: Audio and Music Signal Analysis in Python , 2015, SciPy.

[7]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[8]  Frank Hutter,et al.  Neural Architecture Search: A Survey , 2018, J. Mach. Learn. Res..

[9]  Fernando A. Mujica,et al.  An Empirical Evaluation of Deep Learning on Highway Driving , 2015, ArXiv.

[10]  Lei Wang,et al.  BONSEYES: Platform for Open Development of Systems of Artificial Intelligence: Invited paper , 2017, Conf. Computing Frontiers.

[11]  Luca Benini,et al.  Learning to infer: RL-based search for DNN primitive selection on Heterogeneous Embedded Systems , 2018, 2019 Design, Automation & Test in Europe Conference & Exhibition (DATE).

[12]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[13]  Alexandre Alahi,et al.  PifPaf: Composite Fields for Human Pose Estimation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Weng-Fai Wong,et al.  Exploiting half precision arithmetic in Nvidia GPUs , 2017, 2017 IEEE High Performance Extreme Computing Conference (HPEC).

[15]  Eunhyeok Park,et al.  Compression of Deep Convolutional Neural Networks for Fast and Low Power Mobile Applications , 2015, ICLR.

[16]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[17]  Geoffrey E. Hinton,et al.  Speech recognition with deep recurrent neural networks , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[18]  Luca Benini,et al.  QUENN: QUantization engine for low-power neural networks , 2018, CF.

[19]  Yundong Zhang,et al.  Hello Edge: Keyword Spotting on Microcontrollers , 2017, ArXiv.

[20]  Archan Misra,et al.  Eugene: Towards Deep Intelligence as a Service , 2019, 2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS).

[21]  V. Mathur,et al.  How Well Do We Know Pareto Optimality , 1991 .

[22]  Yoshua Bengio,et al.  Algorithms for Hyper-Parameter Optimization , 2011, NIPS.

[23]  Matthew Mattina,et al.  Efficient Winograd or Cook-Toom Convolution Kernel Implementation on Widely Used Mobile CPUs , 2019, 2019 2nd Workshop on Energy Efficient Machine Learning and Cognitive Computing for Embedded Applications (EMC2).

[24]  Bo Chen,et al.  MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications , 2017, ArXiv.

[25]  Yuxi Li,et al.  Deep Reinforcement Learning: An Overview , 2017, ArXiv.