Latency Optimization for Coded Computation Straggled by Wireless Transmission

Although distributed computing is an efficient solution to large-scale computational tasks, its performance is determined by the slowest computing nodes, i.e., the computation stragglers among distributed nodes. To mitigate their impact, coded computation has emerged as a promising technique by introducing clever computational redundancy. Most existing works only considered the computation latency, however, transmission latency is a serious concern in wireless computing networks. In such a case, the performance is straggled by not only local computation but also wireless transmission. In this letter, we consider the wireless distributed computing network where nodes have different transmission rates and different computation capabilities. Unlike computation stragglers always existing, we derive a straggling factor for each node to indicate whether the network is straggled by transmission. To deal with both stragglers, the wireless coded computation (WCC) algorithm is proposed, which generalizes existing methods. By comparing WCC with four benchmark schemes, Uniform Uncoded Allocation, Non-Uniform Uncoded Allocation, Uniform Coded Allocation, and Heterogeneous Coded Matrix Multiplication, it shows that WCC results in significant speedups of up to 72%, 66%, 64% and 47% over the four aforementioned benchmark schemes, respectively.

[1]  S. Simić ON AN UPPER BOUND FOR JENSEN'S INEQUALITY , 2009 .

[2]  Ramtin Pedarsani,et al.  Latency analysis of coded computation schemes over wireless networks , 2017, 2017 55th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[3]  Gregory W. Wornell,et al.  Efficient task replication for fast response times in parallel computation , 2014, SIGMETRICS '14.

[4]  Amir Salman Avestimehr,et al.  Coded computation over heterogeneous clusters , 2017, 2017 IEEE International Symposium on Information Theory (ISIT).

[5]  Kannan Ramchandran,et al.  Speeding Up Distributed Machine Learning Using Codes , 2015, IEEE Transactions on Information Theory.

[6]  Shancheng Zhao,et al.  A Node-Selection-Based Sub-Task Assignment Method for Coded Edge Computing , 2019, IEEE Communications Letters.

[7]  Osvaldo Simeone,et al.  Coded Network Function Virtualization: Fault Tolerance via In-Network Coding , 2016, IEEE Wireless Communications Letters.

[8]  Jeffrey H. Reed,et al.  Wireless distributed computing: a survey of research challenges , 2012, IEEE Communications Magazine.

[9]  Sanjay Ghemawat,et al.  MapReduce: Simplified Data Processing on Large Clusters , 2004, OSDI.

[10]  Herbert A. David,et al.  Order Statistics , 2011, International Encyclopedia of Statistical Science.

[11]  Gaston H. Gonnet,et al.  On the LambertW function , 1996, Adv. Comput. Math..

[12]  Scott Shenker,et al.  Spark: Cluster Computing with Working Sets , 2010, HotCloud.