Crowd-ML: A library for privacy-preserving machine learning on smart devices

When user-generated data such as audio and video signals are used to train machine learning algorithms, users' privacy must be considered before the learned model is released. In this work, we present an open-source library for privacy-preserving machine learning framework on smart devices. The library allows Android and iOS devices to collectively learn a common classifier/regression model from distributed data with differential privacy, using a variant of minibatch stochastic gradient descent method. The library allows researchers and developers to easily implement and deploy customized tasks that use on-device sensors to collect sensitive data for machine learning.

[1]  Mikhail Belkin,et al.  Crowd-ML: A Privacy-Preserving Learning Framework for a Crowd of Smart Devices , 2015, 2015 IEEE 35th International Conference on Distributed Computing Systems.

[2]  Blaise Agüera y Arcas,et al.  Federated Learning of Deep Networks using Model Averaging , 2016, ArXiv.

[3]  Arun Rajkumar,et al.  A Differentially Private Stochastic Gradient Descent Algorithm for Multiparty Classification , 2012, AISTATS.

[4]  Aaron Roth,et al.  The Algorithmic Foundations of Differential Privacy , 2014, Found. Trends Theor. Comput. Sci..

[5]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..

[6]  Ian Goodfellow,et al.  Deep Learning with Differential Privacy , 2016, CCS.

[7]  M. Athans.  Distributed Asynchronous Deterministic and . . . , 1986 .

[8]  Mikhail Belkin,et al.  Learning privately from multiparty data , 2016, ICML.

[9]  Larry A. Wasserman,et al.  Differential privacy for functions and functional data , 2012, J. Mach. Learn. Res..

[10]  Cynthia Dwork,et al.  Privacy-Preserving Datamining on Vertically Partitioned Databases , 2004, CRYPTO.

[11]  Raef Bassily,et al.  Differentially Private Empirical Risk Minimization: Efficient Algorithms and Tight Error Bounds , 2014, 1405.7085.

[12]  Vitaly Shmatikov,et al.  Privacy-preserving deep learning , 2015, 2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[13]  John C. Duchi,et al.  Distributed delayed stochastic optimization , 2011, 2012 IEEE 51st IEEE Conference on Decision and Control (CDC).

[14]  John N. Tsitsiklis,et al.  Distributed Asynchronous Deterministic and Stochastic Gradient Optimization Algorithms , 1984, 1984 American Control Conference.

[15]  Cynthia Dwork,et al.  Calibrating Noise to Sensitivity in Private Data Analysis , 2006, TCC.

[16]  Anand D. Sarwate,et al.  Stochastic gradient descent with differentially private updates , 2013, 2013 IEEE Global Conference on Signal and Information Processing.

[17]  Zhenqi Huang,et al.  Differentially Private Distributed Optimization , 2014, ICDCN.

[18]  Ohad Shamir,et al.  Optimal Distributed Online Prediction , 2011, ICML.

[19]  Anand D. Sarwate,et al.  Differentially Private Empirical Risk Minimization , 2009, J. Mach. Learn. Res..

[20]  Cynthia Dwork,et al.  Differential Privacy , 2006, ICALP.