DeepWear: Adaptive Local Offloading for On-Wearable Deep Learning

Due to their on-body and ubiquitous nature, wearables can generate a wide range of unique sensor data creating countless opportunities for deep learning tasks. We propose DeepWear, a deep learning (DL) framework for wearable devices to improve the performance and reduce the energy footprint. DeepWear strategically offloads DL tasks from a wearable device to its paired handheld device through local network connectivity such as Bluetooth. Compared to the remote-cloud-based offloading, DeepWear requires no Internet connectivity, consumes less energy, and is robust to privacy breach. DeepWear provides various novel techniques such as context-aware offloading, strategic model partition, and pipelining support to efficiently utilize the processing capacity from nearby paired handhelds. Deployed as a user-space library, DeepWear offers developer-friendly APIs that are as simple as those in traditional DL libraries such as TensorFlow. We have implemented DeepWear on the Android OS and evaluated it on COTS smartphones and smartwatches with real DL models. DeepWear brings up to 5.08X and 23.0X execution speedup, as well as 53.5 and 85.5 percent energy saving compared to wearable-only and handheld-only strategies, respectively.

[1]  Nicholas D. Lane,et al.  Sparsification and Separation of Deep Learning Layers for Constrained Resource Inference on Wearables , 2016, SenSys.

[2]  Alec Wolman,et al.  MAUI: making smartphones last longer with code offload , 2010, MobiSys '10.

[3]  Hamed Haddadi,et al.  A Hybrid Deep Learning Architecture for Privacy-Preserving Mobile Analytics , 2017, IEEE Internet of Things Journal.

[4]  Nicholas D. Lane,et al.  DeepX: A Software Accelerator for Low-Power Deep Learning Inference on Mobile Devices , 2016, 2016 15th ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN).

[5]  Lei Yang,et al.  Accurate online power estimation and automatic battery behavior based power model generation for smartphones , 2010, 2010 IEEE/ACM/IFIP International Conference on Hardware/Software Codesign and System Synthesis (CODES+ISSS).

[6]  Trevor N. Mudge,et al.  Neurosurgeon: Collaborative Intelligence Between the Cloud and Mobile Edge , 2017, ASPLOS.

[7]  Andres Calvo.  The Wearable Data Layer API , 2015 .

[8]  Ming Liu,et al.  Novel wavelet neural network algorithm for continuous and noninvasive dynamic estimation of blood pressure from photoplethysmography , 2016, Science China Information Sciences.

[9]  Ge Yu,et al.  A novel cross-modal hashing algorithm based on multimodal deep learning , 2015, Science China Information Sciences.

[10]  Xuanzhe Liu,et al.  DeepType , 2018, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[11]  Mahesh K. Marina,et al.  Towards multimodal deep learning for activity recognition on mobile devices , 2016, UbiComp Adjunct.

[12]  Nicholas D. Lane,et al.  DeepEar: robust smartphone audio sensing in unconstrained acoustic environments using deep learning , 2015, UbiComp.

[13]  김종영.  구글 TensorFlow 소개 , 2015 .

[14]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Di Wang,et al.  A Long Short-Term Memory Model for Answer Sentence Selection in Question Answering , 2015, ACL.

[16]  Xu Chen,et al.  COMET: Code Offload by Migrating Execution Transparently , 2012, OSDI.

[17]  Alec Wolman,et al.  MCDNN: An Approximation-Based Execution Framework for Deep Stream Processing Under Resource Constraints , 2016, MobiSys.

[18]  Nicholas D. Lane,et al.  DeepEye: Resource Efficient Local Execution of Multiple Deep Vision Models using Wearable Commodity Hardware , 2017, MobiSys.

[19]  Xuanzhe Liu,et al.  DeepCache: Principled Cache for Mobile Deep Vision , 2017, MobiCom.

[20]  Di Huang,et al.  Dust: Real-Time Code Offloading System for Wearable Computing , 2014, 2015 IEEE Global Communications Conference (GLOBECOM).

[21]  Nicholas D. Lane,et al.  Can Deep Learning Revolutionize Mobile Sensing? , 2015, HotMobile.

[22]  Feng Qian,et al.  Characterizing Smartwatch Usage in the Wild , 2017, MobiSys.

[23]  Jian Cheng,et al.  Quantized Convolutional Neural Networks for Mobile Devices , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Alex Graves,et al.  Generating Sequences With Recurrent Neural Networks , 2013, ArXiv.

[25]  Joan Bruna,et al.  Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation , 2014, NIPS.

[26]  Byung-Gon Chun,et al.  CloneCloud: elastic execution between mobile device and cloud , 2011, EuroSys '11.

[27]  Bo Chen,et al.  MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications , 2017, ArXiv.

[28]  Ninghui Sun,et al.  DianNao: a small-footprint high-throughput accelerator for ubiquitous machine-learning , 2014, ASPLOS.

[29]  Xuanzhe Liu,et al.  ShuffleDog: Characterizing and Adapting User-Perceived Latency of Android Apps , 2017, IEEE Transactions on Mobile Computing.

[30]  Young-June Choi,et al.  Poster: A Novel Computation Offloading Technique for Reducing Energy Consumption of Smart Watch , 2016, MobiSys '16 Companion.

[31]  Cecilia Mascolo,et al.  LEO: scheduling sensor inference algorithms across heterogeneous mobile processors and network resources , 2016, MobiCom.

[32]  Scott A. Mahlke,et al.  Accelerating Mobile Applications through Flip-Flop Replication , 2015, MobiSys.

[33]  Ying Zhang,et al.  Refactoring android Java code for on-demand computation offloading , 2012, OOPSLA '12.

[34]  Jason Cong,et al.  Optimizing FPGA-based Accelerator Design for Deep Convolutional Neural Networks , 2015, FPGA.

[35]  Hongfei Lin,et al.  Convolutional neural networks for expert recommendation in community question answering , 2017, Science China Information Sciences.

[36]  Heiga Zen,et al.  WaveNet: A Generative Model for Raw Audio , 2016, SSW.

[37]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[38]  Song Han,et al.  EIE: Efficient Inference Engine on Compressed Deep Neural Network , 2016, 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA).

[39]  Ji Yang,et al.  Offloading Guidelines for Augmented Reality Applications on Wearable Devices , 2015, ACM Multimedia.

[40]  Jiannong Cao,et al.  Joint Computation Partitioning and Resource Allocation for Latency Sensitive Applications in Mobile Edge Clouds , 2017, 2017 IEEE 10th International Conference on Cloud Computing (CLOUD).

[41]  Erik McDermott,et al.  Deep neural networks for small footprint text-dependent speaker verification , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[42]  Mahadev Satyanarayanan,et al.  Towards wearable cognitive assistance , 2014, MobiSys.

[43]  Xuanzhe Liu,et al.  A First Look at Deep Learning Apps on Smartphones , 2018, WWW.

[44]  Shaohan Hu,et al.  DeepSense: A Unified Deep Learning Framework for Time-Series Mobile Sensing Data Processing , 2016, WWW.

[45]  Hui Liu,et al.  On-Demand Deep Model Compression for Mobile Devices: A Usage-Driven Model Selection Framework , 2018, MobiSys.

[46]  Panos Kalnis,et al.  GRAMI: Frequent Subgraph and Pattern Mining in a Single Large Graph , 2014, Proc. VLDB Endow..

[47]  Moustafa Alzantot,et al.  RSTensorFlow: GPU Enabled TensorFlow for Deep Learning on Commodity Android Devices , 2017, EMDL '17.

[48]  Jiannong Cao,et al.  Network Aware Multi-User Computation Partitioning in Mobile Edge Clouds , 2017, 2017 46th International Conference on Parallel Processing (ICPP).

[49]  Ying Gao,et al.  Quantifying the Impact of Edge Computing on Mobile Applications , 2016, APSys.

[50]  Wei Sun,et al.  Towards Service Composition Based on Mashup , 2007, 2007 IEEE Congress on Services (Services 2007).

[51]  Georg Heigold,et al.  Small-footprint keyword spotting using deep neural networks , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).