Resource Characterisation of Personal-Scale Sensing Models on Edge Accelerators

Edge accelerator is a class of brand-new purpose-built System On a Chip (SoC) for running deep learning models efficiently on edge devices. These accelerators offer various benefits such as ultra-low latency, sensitive data protection, and high availability due to their locality and are opening up interminable opportunities for building sensory systems in the real world. Naturally, in the context of sensory awareness systems, e.g., IoT, wearables, and other sensory devices, the emergence of edge accelerators is pushing us to rethink how we design these systems at a personal-scale. To this end, in this paper we take a closer look at the performance of a set of edge accelerators in running a collection of personal-scale sensory deep learning models. We benchmark eight different models with varying architectures and tasks (i.e., motion, audio, and vision) across seven platform configurations with three different accelerators including Google Coral, NVidia Jetson Nano, and Intel Neural Compute Stick. We report on their execution performance concerning latency, memory, and power consumption while discussing their current workflows and limitations. The results and insights lay an empirical foundation for the development of sensory systems on edge accelerators.

[1]  Ling Chen,et al.  AROMA , 2018, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[2]  Tadashi Okoshi,et al.  Situation-Aware Conversational Agent with Kinetic Earables (demo) , 2019, MobiSys.

[3]  Quoc V. Le,et al.  EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks , 2019, ICML.

[4]  Forrest N. Iandola,et al.  SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <1MB model size , 2016, ArXiv.

[5]  Yurong Liu,et al.  A survey of deep neural network architectures and their applications , 2017, Neurocomputing.

[6]  Nicholas D. Lane,et al.  An Early Resource Characterization of Deep Learning on Wearables, Smartphones and Internet-of-Things Devices , 2015, IoT-App@SenSys.

[7]  Ilias Leontiadis,et al.  EmBench: Quantifying Performance Variations of Deep Neural Networks across Modern Commodity Devices , 2019, EMDL '19.

[8]  Soheil Ghiasi,et al.  Hardware-oriented Approximation of Convolutional Neural Networks , 2016, ArXiv.

[9]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Bo Chen,et al.  MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications , 2017, ArXiv.

[11]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Fanglin Chen,et al.  StudentLife: assessing mental health, academic performance and behavioral trends of college students using smartphones , 2014, UbiComp.

[13]  Bo Chen,et al.  Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[14]  Hongnian Yu,et al.  Elderly activities recognition and classification for applications in assisted living , 2013, Expert Syst. Appl..

[15]  Nicholas D. Lane,et al.  Mic2Mic: using cycle-consistent generative adversarial networks to overcome microphone variability in speech systems , 2019, IPSN.