Pushing Analytics to the Edge

Edge or Fog computing is emerging as a new computing paradigm where the data processing, networking, storage and analytics are performed closer to the devices (IoT) and applications. The edge of a network plays an important role in the IoT system. It is an optimal site for off-loading bandwidth hungry IoT data. In order to generate business value out of the large volume of data on the edge, we need decentralized machine learning (ML) algorithms. These algorithms must enable edge devices to communicate autonomously and deliver information seamlessly to the decision makers. In this paper, we present EdgeSGD a decentralized stochastic gradient descent method to solve a large linear regression problem on the edge network. The solution is applied to the seismic imaging use-case and evaluated using an edge computing testbed. The proposed algorithm avoids sending raw data to the cloud, and offers faster and balanced computation. We compare the algorithm with the existing methods such as MapReduce, DGD and EXTRA. The results show that the EdgeSGD converges faster to the optimal value and is robust towards node/link failure. Finally, we test the algorithm using real-world seismic data trace from Parkfield, CA and show that our proposed algorithm can illuminate the underlying fault region of San-Andreas.

[1]  Stephen J. Wright,et al.  Hogwild: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent , 2011, NIPS.

[2]  Lei Shi,et al.  Distributed travel-time seismic tomography in large-scale sensor networks , 2016, J. Parallel Distributed Comput..

[3]  Alexander J. Smola,et al.  Parallelized Stochastic Gradient Descent , 2010, NIPS.

[4]  Wilfried N. Gansterer,et al.  Distributed QR Factorization Based on Randomized Algorithms , 2011, PPAM.

[5]  Qing Ling,et al.  On the Convergence of Decentralized Gradient Descent , 2013, SIAM J. Optim..

[6]  Soummya Kar,et al.  Gossip Algorithms for Distributed Signal Processing , 2010, Proceedings of the IEEE.

[7]  Johannes Gehrke,et al.  Gossip-based computation of aggregate information , 2003, 44th Annual IEEE Symposium on Foundations of Computer Science, 2003. Proceedings..

[8]  Goutham Kamath,et al.  Distributed Randomized Kaczmarz and Applications to Seismic Imaging in Sensor Network , 2015, 2015 International Conference on Distributed Computing in Sensor Systems.

[9]  Qing Ling,et al.  On the Linear Convergence of the ADMM in Decentralized Consensus Optimization , 2013, IEEE Transactions on Signal Processing.

[10]  Dan Gordon,et al.  Component-Averaged Row Projections: A Robust, Block-Parallel Scheme for Sparse Linear Systems , 2005, SIAM J. Sci. Comput..

[11]  Robbert van Renesse,et al.  Astrolabe: A robust and scalable technology for distributed system monitoring, management, and data mining , 2003, TOCS.

[12]  Lei Shi,et al.  Component-Average Based Distributed Seismic Tomography in Sensor Networks , 2013, 2013 IEEE International Conference on Distributed Computing in Sensor Systems.

[13]  Wen-Zhan Song,et al.  Volcanic earthquake timing using wireless sensor networks , 2013, 2013 ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN).

[14]  Stephen J. Wright,et al.  An asynchronous parallel stochastic coordinate descent algorithm , 2013, J. Mach. Learn. Res..

[15]  Qing Ling,et al.  EXTRA: An Exact First-Order Algorithm for Decentralized Consensus Optimization , 2014, 1404.6264.

[16]  Gideon S. Mann,et al.  Efficient Large-Scale Distributed Training of Conditional Maximum Entropy Models , 2009, NIPS.

[17]  Léon Bottou,et al.  Large-Scale Machine Learning with Stochastic Gradient Descent , 2010, COMPSTAT.

[18]  C. Thurber,et al.  Joint inversion for Vp, Vs, and Vp/Vs at SAFOD, Parkfield, California , 2009 .

[19]  Ivan Stojmenovic,et al.  The Fog computing paradigm: Scenarios and security issues , 2014, 2014 Federated Conference on Computer Science and Information Systems.

[20]  Stephen P. Boyd,et al.  Randomized gossip algorithms , 2006, IEEE Transactions on Information Theory.