Fundamental Limits of Wireless Distributed Computing Networks

We consider a wireless distributed computing network, where all computing nodes (workers) are connected via wireless medium obeying the seminal protocol channel model. In particular, we focus on the MapReduce-type platform, where each worker is assigned to compute some arbitrary output functions from $F$ input files, which are distributively cached in all workers. The overall computation is decomposed into computing a set of “Map” and “Reduce” functions across all workers. The goal is to characterize the minimum computing latency as a function of the computation load. Unlike other related works, which consider either wireline settings or restrict the communication among workers to single-hop, here we focus on the wireless scenario and do not constrain any communication schemes. We propose a data set cache strategy based on a deterministic assignment of Maximum Distance Separable (MDS)-coded date sets over all input files, and a coded multicast transmission strategy where the workers send linearly coded computing results to each other in order to collectively satisfy their assigned tasks. We show that our approach can achieve a scalable communication latency, outperform the state of the art schemes in the order sense, and achieve the information theoretic outer bound within a multiplicative constant factor in practical parameter regimes.

[1]  Dimitris S. Papailiopoulos,et al.  Speeding up distributed machine learning using codes , 2016, ISIT.

[2]  Alexandros G. Dimakis,et al.  Gradient Coding: Avoiding Stragglers in Distributed Learning , 2017, ICML.

[3]  Khaled Ben Letaief,et al.  Mobile Edge Computing: Survey and Research Outlook , 2017, ArXiv.

[4]  A. Salman Avestimehr,et al.  A Fundamental Tradeoff Between Computation and Communication in Distributed Computing , 2016, IEEE Transactions on Information Theory.

[5]  Mohammad Ali Maddah-Ali,et al.  Communication-aware computing for edge processing , 2017, 2017 IEEE International Symposium on Information Theory (ISIT).

[6]  Ilan Shomorony,et al.  Degrees of Freedom of Two-Hop Wireless Networks: Everyone Gets the Entire Cake , 2012, IEEE Transactions on Information Theory.

[7]  A. Salman Avestimehr,et al.  A Scalable Framework for Wireless Distributed Computing , 2016, IEEE/ACM Transactions on Networking.

[8]  Ramtin Pedarsani,et al.  Latency analysis of coded computation schemes over wireless networks , 2017, 2017 55th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[9]  Hairong Kuang,et al.  The Hadoop Distributed File System , 2010, 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST).

[10]  Urs Niesen,et al.  Fundamental Limits of Caching , 2014, IEEE Trans. Inf. Theory.

[11]  Panganamala Ramana Kumar,et al.  RHEINISCH-WESTFÄLISCHE TECHNISCHE HOCHSCHULE AACHEN , 2001 .

[12]  Ravi Tandon,et al.  On the worst-case communication overhead for distributed data shuffling , 2016, 2016 54th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[13]  Gábor Lugosi,et al.  Concentration Inequalities - A Nonasymptotic Theory of Independence , 2013, Concentration Inequalities.

[14]  Scott Shenker,et al.  Spark: Cluster Computing with Working Sets , 2010, HotCloud.

[15]  Sanjay Ghemawat,et al.  MapReduce: Simplified Data Processing on Large Clusters , 2004, OSDI.

[16]  Giuseppe Caire,et al.  Fundamental Limits of Caching in Wireless D2D Networks , 2014, IEEE Transactions on Information Theory.

[17]  Xiang-Yang Li Multicast capacity of wireless ad hoc networks , 2009, TNET.

[18]  Giuseppe Caire,et al.  The Throughput-Outage Tradeoff of Wireless One-Hop Caching Networks , 2013, IEEE Transactions on Information Theory.

[19]  Tao Zhang,et al.  Fog and IoT: An Overview of Research Opportunities , 2016, IEEE Internet of Things Journal.