DeCaf: Iterative Collaborative Processing over the Edge

The increase in privacy concerns among the users has led to edge based analytics applications such as federated learning which trains machine learning models in an iterative and collaborative fashion on the edge devices without sending the raw private data to the central cloud. In this paper, we propose a system for enabling iterative collaborative processing (ICP) in resource constrained edge environments. We first identify the unique systems challenges posed by ICP, which are not addressed by the existing distributed machine learning frameworks such as the parameter server. We then propose the system components necessary for ICP to work well in highly distributed edge environments. Based on this, we propose a system design for enabling such applications over the edge. We show the benefits of our proposed system components with a preliminary evaluation.

[1]  Trevor N. Mudge,et al.  Neurosurgeon: Collaborative Intelligence Between the Cloud and Mobile Edge , 2017, ASPLOS.

[2]  Alexander J. Smola,et al.  Scaling Distributed Machine Learning with the Parameter Server , 2014, OSDI.

[3]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[4]  Paramvir Bahl,et al.  Low Latency Geo-distributed Data Analytics , 2015, SIGCOMM.

[5]  Donghan Yu,et al.  Smartphone App Usage Prediction Using Points of Interest , 2017, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[6]  Nikolaos G. Bourbakis,et al.  A Survey on Wearable Sensor-Based Systems for Health Monitoring and Prognosis , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[7]  Alec Wolman,et al.  MCDNN: An Approximation-Based Execution Framework for Deep Stream Processing Under Resource Constraints , 2016, MobiSys.

[8]  Qiang Xu,et al.  Identifying diverse usage behaviors of smartphone apps , 2011, IMC '11.

[9]  Suman Banerjee,et al.  Final report from the NSF Workshop on Future Directions in Wireless Networking , 2013 .

[10]  Soo-Mook Moon,et al.  IONN: Incremental Offloading of Neural Network Computations from Mobile Devices to Edge Servers , 2018, SoCC.

[11]  Gwilym M. Jenkins,et al.  Time series analysis, forecasting and control , 1971 .

[12]  Zhaohui Wu,et al.  Discovering different kinds of smartphone users through their application usage behaviors , 2016, UbiComp.

[13]  Hubert Eichner,et al.  Towards Federated Learning at Scale: System Design , 2019, SysML.

[14]  Feng Xu,et al.  AppUsage2Vec: Modeling Smartphone App Usage for Prediction , 2019, 2019 IEEE 35th International Conference on Data Engineering (ICDE).

[15]  Davide Anguita,et al.  A Public Domain Dataset for Human Activity Recognition using Smartphones , 2013, ESANN.

[16]  Blaise Agüera y Arcas,et al.  Federated Learning of Deep Networks using Model Averaging , 2016, ArXiv.

[17]  Rachid Guerraoui,et al.  Personalized and Private Peer-to-Peer Machine Learning , 2017, AISTATS.

[18]  Scott Shenker,et al.  Monarch: Gaining Command on Geo-Distributed Graph Analytics , 2018, HotCloud.

[19]  Abhishek Chandra,et al.  Multi-Query Optimization in Wide-Area Streaming Analytics , 2018, SoCC.

[20]  Zheng Zhang,et al.  MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems , 2015, ArXiv.

[21]  Minlan Yu,et al.  Wide-area analytics with multiple resources , 2018, EuroSys.

[22]  Anit Kumar Sahu,et al.  On the Convergence of Federated Optimization in Heterogeneous Networks , 2018, ArXiv.

[23]  Hubert Eichner,et al.  Federated Learning for Mobile Keyboard Prediction , 2018, ArXiv.

[24]  Reynold Xin,et al.  GraphX: Graph Processing in a Distributed Dataflow Framework , 2014, OSDI.

[25]  Aart J. C. Bik,et al.  Pregel: a system for large-scale graph processing , 2010, SIGMOD Conference.

[26]  Jin-Hyuk Hong,et al.  Understanding and prediction of mobile application usage for smart phones , 2012, UbiComp.

[27]  Onur Mutlu,et al.  Gaia: Geo-Distributed Machine Learning Approaching LAN Speeds , 2017, NSDI.

[28]  Stefano Secci,et al.  Characterizing and predicting mobile application usage , 2016, Comput. Commun..

[29]  Paramvir Bahl,et al.  VideoEdge: Processing Camera Streams using Hierarchical Clusters , 2018, 2018 IEEE/ACM Symposium on Edge Computing (SEC).

[30]  Hubert Eichner,et al.  APPLIED FEDERATED LEARNING: IMPROVING GOOGLE KEYBOARD QUERY SUGGESTIONS , 2018, ArXiv.

[31]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[32]  Tim Kraska,et al.  MLbase: A Distributed Machine-learning System , 2013, CIDR.

[33]  Marc'Aurelio Ranzato,et al.  Large Scale Distributed Deep Networks , 2012, NIPS.