Coded Computation of Multiple Functions

We consider the problem of evaluating arbitrary multivariate polynomials over several massive datasets in a distributed computing system with a single master node and multiple worker nodes. We focus on the general case when each multivariate polynomial is evaluated over its dataset and propose a generalization of the Lagrange Coded Computing framework (Yu et al. 2019) to provide robustness against stragglers who do not respond in time, adversarial workers who respond with wrong computation and information-theoretic security of dataset against colluding workers. Our scheme introduces a small computation overhead which results in a reduction in download cost and also offers comparable resistance to stragglers over existing solutions.

[1]  Songze Li,et al.  Generalized Lagrange Coded Computing: A Flexible Computation-Communication Tradeoff , 2022, 2022 IEEE International Symposium on Information Theory (ISIT).

[2]  A. Salman Avestimehr,et al.  List-Decodable Coded Computing: Breaking the Adversarial Toleration Barrier , 2021, IEEE Journal on Selected Areas in Information Theory.

[3]  A. Salman Avestimehr,et al.  Coded Computing for Resilient, Secure, and Privacy-Preserving Distributed Matrix Multiplication , 2021, IEEE Transactions on Communications.

[4]  A. Salman Avestimehr,et al.  Privacy-Preserving Distributed Learning in the Analog Domain , 2020, ArXiv.

[5]  Amir Salman Avestimehr,et al.  Lagrange Coded Computing: Optimal Design for Resiliency, Security and Privacy , 2018, AISTATS.

[6]  Mohammad Ali Maddah-Ali,et al.  Limited-Sharing Multi-Party Computation for Massive Matrix Operations , 2018, 2018 IEEE International Symposium on Information Theory (ISIT).

[7]  Mohammad Ali Maddah-Ali,et al.  Straggler Mitigation in Distributed Matrix Multiplication: Fundamental Limits and Optimal Coding , 2018, 2018 IEEE International Symposium on Information Theory (ISIT).

[8]  Hessam Mahdavifar,et al.  Distributed Multi-User Secret Sharing , 2018, 2018 IEEE International Symposium on Information Theory (ISIT).

[9]  Hua Sun,et al.  The Capacity of Private Computation , 2018, 2018 IEEE International Conference on Communications (ICC).

[10]  Mohammad Ali Maddah-Ali,et al.  Polynomial Codes: an Optimal Design for High-Dimensional Coded Matrix Multiplication , 2017, NIPS.

[11]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[12]  A. Salman Avestimehr,et al.  A Fundamental Tradeoff Between Computation and Communication in Distributed Computing , 2016, IEEE Transactions on Information Theory.

[13]  Kannan Ramchandran,et al.  Speeding Up Distributed Machine Learning Using Codes , 2015, IEEE Transactions on Information Theory.

[14]  Lawrence H. Ozarow,et al.  Wire-tap channel II , 1984, AT&T Bell Laboratories Technical Journal.

[15]  Chunyan Miao,et al.  A Comprehensive Survey on Coded Distributed Computing: Fundamentals, Challenges, and Networking Applications , 2021, IEEE Communications Surveys & Tutorials.

[16]  A. Barron,et al.  Coded Computing: Mitigating Fundamental Bottlenecks in Large-Scale Distributed Computing and Machine Learning , 2020 .

[17]  Avi Wigderson,et al.  Completeness theorems for non-cryptographic fault-tolerant distributed computation , 1988, STOC '88.

[18]  Shu Lin,et al.  Error control coding : fundamentals and applications , 1983 .