Collaborative Inference for Mobile Deep Learning Applications

Deep learning makes people enjoy a more convenient as well as smarter mobile life. It is an interesting yet much challenging topic to enable efficient mobile deep learning applications. Traditional approach to tackle this challenge is to employ cloud computing by offloading the computation tasks to remote servers, which has weaknesses of high bandwidth requirements and transmission latency. In this paper, we propose to enable collaborative inference among local mobile devices. Instead of sending deep learning inference tasks to cloud, we let mobile devices collaboratively share the computation workloads. This is based on an important observation that batching inference tasks on GPUs can accelerate the processing speed. To achieve efficient collaboration, we design an algorithm based on partial swarm optimization (PSO) that is a versatile population-based stochastic optimization technique. Moreover, extensive simulations are conducted to evaluate the performance of the designed algorithm. The simulation results show that the collaborative inference scheme can reduce global dealing time in given field compared with offloading tasks to cloud.

[1]  Harris Drucker,et al.  Learning algorithms for classification: A comparison on handwritten digit recognition , 1995 .

[2]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[3]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[4]  Song Han,et al.  Deep Compression: Compressing Deep Neural Network with Pruning, Trained Quantization and Huffman Coding , 2015, ICLR.

[5]  John Tran,et al.  cuDNN: Efficient Primitives for Deep Learning , 2014, ArXiv.

[6]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[7]  Eugenio Culurciello,et al.  An Analysis of Deep Neural Network Models for Practical Applications , 2016, ArXiv.

[8]  Qinghua Li,et al.  Privacy-aware and trustworthy data aggregation in mobile sensing , 2015, 2015 IEEE Conference on Communications and Network Security (CNS).

[9]  Song Guo,et al.  Energy-Efficient Transmission Scheduling in Mobile Phones Using Machine Learning and Participatory Sensing , 2015, IEEE Transactions on Vehicular Technology.

[10]  Demetrios Zeinalipour-Yazti,et al.  Crowdsourcing with Smartphones , 2012, IEEE Internet Computing.

[11]  Song Guo,et al.  Crowdsourcing on mobile cloud: Cost minimization of joint data acquisition and processing , 2014, 2014 IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS).

[12]  Hai Jin,et al.  Using Crowdsourcing to Provide QoS for Mobile Cloud Computing , 2019, IEEE Transactions on Cloud Computing.

[13]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  C. Rossi,et al.  On the evaluation of cloud web services for crowdsourcing mobile applications , 2016, 2016 2nd International Conference on Cloud Computing Technologies and Applications (CloudTech).

[15]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.