A Case For Adaptive Deep Neural Networks in Edge Computing

Edge computing offers an additional layer of compute infrastructure closer to the data source before raw data from privacy-sensitive and performance-critical applications is transferred to a cloud data center. Deep Neural Networks (DNNs) are one class of applications that are reported to benefit from collaboratively computing between the edge and the cloud. A DNN is partitioned such that specific layers of the DNN are deployed onto the edge and the cloud to meet performance and privacy objectives. However, there is limited understanding of: (a) whether and how evolving operational conditions (increased CPU and memory utilization at the edge or reduced data transfer rates between the edge and the cloud) affect the performance of already deployed DNNs, and (b) whether a new partition configuration is required to maximize performance. A DNN that adapts to changing operational conditions is referred to as an `adaptive DNN'. This paper investigates whether there is a case for adaptive DNNs in edge computing by considering three questions: (i) Are DNNs sensitive to operational conditions? (ii) How sensitive are DNNs to operational conditions? (iii) Do individual or a combination of operational conditions equally affect DNNs? The exploration is carried out in the context of 8 pre-trained DNN models and the results presented are from analyzing nearly 2 million data points. The results highlight that network conditions affects DNN performance more than CPU or memory related operational conditions. Repartitioning is noted to provide a performance gain in a number of cases, thus demonstrating the need for adaptive DNNs.

[1]  Mario Di Francesco,et al.  Distributed Inference Acceleration with Adaptive DNN Partitioning and Offloading , 2020, IEEE INFOCOM 2020 - IEEE Conference on Computer Communications.

[2]  Yiran Chen,et al.  MoDNN: Local distributed mobile computing system for Deep Neural Network , 2017, Design, Automation & Test in Europe Conference & Exhibition (DATE), 2017.

[3]  Feng Qian,et al.  DeepWear: Adaptive Local Offloading for On-Wearable Deep Learning , 2017, IEEE Transactions on Mobile Computing.

[4]  Weisong Shi,et al.  Edge Computing: Vision and Challenges , 2016, IEEE Internet of Things Journal.

[5]  Xu Chen,et al.  Edge Intelligence: Paving the Last Mile of Artificial Intelligence With Edge Computing , 2019, Proceedings of the IEEE.

[6]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[7]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[8]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Chengzhu Yu,et al.  Context adaptive deep neural networks for fast acoustic model adaptation in noisy conditions , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[10]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[11]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Trevor N. Mudge,et al.  Neurosurgeon: Collaborative Intelligence Between the Cloud and Mobile Edge , 2017, ASPLOS.

[13]  Ruzhuo Wang.  Adaptive Solution to Compress Deep Neural Networks for Resource-Constrained Devices , 2019 .

[14]  Mahadev Satyanarayanan,et al.  The Emergence of Edge Computing , 2017, Computer.

[15]  Dan Wang,et al.  Dynamic Adaptive DNN Surgery for Inference Acceleration on the Edge , 2019, IEEE INFOCOM 2019 - IEEE Conference on Computer Communications.

[16]  Michael S. Ryoo,et al.  Musical Chair: Efficient Real-Time Recognition Using Collaborative IoT Devices , 2018, ArXiv.

[17]  Andreas Gerstlauer,et al.  DeepThings: Distributed Adaptive Deep Learning Inference on Resource-Constrained IoT Edge Clusters , 2018, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[18]  Xiaofei Wang,et al.  Convergence of Edge Computing and Deep Learning: A Comprehensive Survey , 2019, IEEE Communications Surveys & Tutorials.

[19]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[20]  Peter Kilpatrick,et al.  Challenges and Opportunities in Edge Computing , 2016, 2016 IEEE International Conference on Smart Cloud (SmartCloud).

[21]  Weisong Shi,et al.  LAVEA: latency-aware video analytics on edge computing platform , 2017, SEC.

[22]  Hao Wen,et al.  Distributing Deep Neural Networks with Containerized Partitions at the Edge , 2019, HotEdge.

[23]  Florian Metze,et al.  Speaker Adaptive Training of Deep Neural Network Acoustic Models Using I-Vectors , 2015, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[24]  Yehia El-khatib,et al.  Adaptive deep learning model selection on embedded systems , 2018, LCTES.

[25]  Jie Xu,et al.  Privacy-Aware Edge Computing Based on Adaptive DNN Partitioning , 2019, 2019 IEEE Global Communications Conference (GLOBECOM).

[26]  Bo Chen,et al.  MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications , 2017, ArXiv.

[27]  Ada Gavrilovska,et al.  Couper: DNN model slicing for visual analytics containers at the edge , 2019, SEC.

[28]  Fang Dong,et al.  ADDA: Adaptive Distributed DNN Inference Acceleration in Edge Computing Environment , 2019, 2019 IEEE 25th International Conference on Parallel and Distributed Systems (ICPADS).

[29]  Weisong Shi,et al.  A Comparison of Communication Mechanisms in Vehicular Edge Computing , 2020, HotEdge.

[30]  Rajkumar Buyya,et al.  Next generation cloud computing: New trends and research directions , 2017, Future Gener. Comput. Syst..

[31]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[32]  Mahadev Satyanarayanan,et al.  The Seminal Role of Edge-Native Applications , 2019, 2019 IEEE International Conference on Edge Computing (EDGE).

[33]  Mahadev Satyanarayanan,et al.  Towards wearable cognitive assistance , 2014, MobiSys.