Enabling Deep Learning on IoT Edge: Approaches and Evaluation

As we enter the Internet of Things (IoT) era, the size of mobile computing devices is largely reduced while their computing capability is dramatically improved. Meanwhile, machine learning technologies have been well developed and shown cutting edge performance in various tasks, leading to their wide adoption. As a result, moving machine learning, especially deep learning capability to the edge of the IoT is a trend happening today. But directly moving machine learning algorithms which originally run on PC platform is not feasible for IoT devices due to their relatively limited computing power. In this paper, we first reviewed several representative approaches for enabling deep learning on mobile/IoT devices. Then we evaluated the performance and impact of these methods on IoT platform equipped with integrated GPU and ARM processor. Our results show that we can enable the deep learning capability on the edge of the IoT if we apply these approaches in an efficient manner.

[1]  Joo-Hwee Lim,et al.  A Wearable Face Recognition System on Google Glass for Assisting Social Interactions , 2014, ACCV Workshops.

[2]  Xiangyu Zhang,et al.  Channel Pruning for Accelerating Very Deep Neural Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[3]  Pritish Narayanan,et al.  Deep Learning with Limited Numerical Precision , 2015, ICML.

[4]  Jiye Shi,et al.  The Cloudlet Accelerator: Bringing Mobile-Cloud Face Recognition into Real-Time , 2015, 2015 IEEE Globecom Workshops (GC Wkshps).

[5]  Chen Liu,et al.  Boosting Face in Video Recognition via CNN Based Key Frame Extraction , 2018, 2018 International Conference on Biometrics (ICB).

[6]  Omkar M. Parkhi,et al.  VGGFace2: A Dataset for Recognising Faces across Pose and Age , 2017, 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018).

[7]  Song Han,et al.  Deep Compression: Compressing Deep Neural Network with Pruning, Trained Quantization and Huffman Coding , 2015, ICLR.

[8]  Jean-Luc Gaudiot,et al.  Enabling Embedded Inference Engine with ARM Compute Library: A Case Study , 2017, ArXiv.

[9]  Chen Liu,et al.  GPU-Accelerated Key Frame Analysis for Face Detection in Video , 2015, 2015 IEEE 7th International Conference on Cloud Computing Technology and Science (CloudCom).

[10]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[12]  Kwang-Ting Cheng,et al.  Using mobile GPU for general-purpose computing – a case study of face recognition on smartphones , 2011, Proceedings of 2011 International Symposium on VLSI Design, Automation and Test.

[13]  Natalie D. Enright Jerger,et al.  Reduced-Precision Strategies for Bounded Memory in Deep Neural Nets , 2015, ArXiv.

[14]  Wendi B. Heinzelman,et al.  Cloud-Vision: Real-time face recognition using a mobile-cloudlet-cloud acceleration architecture , 2012, 2012 IEEE Symposium on Computers and Communications (ISCC).