Communication for Omniscience

This paper considers the communication for omniscience (CO) problem: A set of users observe a discrete memoryless multiple source and want to recover the entire multiple source via noise-free broadcast communications. We study the problem of how to attain omniscience with the minimum sum-rate, the total number of communications, and determine a corresponding optimal rate vector. The results cover both asymptotic and non-asymptotic models where the transmission rates are real and integral, respectively. Based on the concepts of submodularity and Dilworth truncation, we formulate a maximization problem. The maximum is the highest Slepian-Wolf constraint over all multi-way cuts of the user set, which determines the minimum sum-rate. For solving this maximization problem and searching for an optimal rate vector, we propose a modified decomposition algorithm (MDA) and a sum-rate increment algorithm (SIA) for asymptotic and non-asymptotic models, respectively, both of which complete in polynomial time. For solving the Dilworth truncation problem as the subroutine in both algorithms, we propose a fusion method to implement the existing coordinate saturation capacity (CoordSatCap) algorithm, where the submodular function minimization (SFM) is done over a merged user set. We show by experimental results that this fusion method contributes to a reduction in computation complexity as compared to the original CoordSatCap algorithm.

[1]  Rodney A. Kennedy,et al.  Fairness in communication for omniscience , 2016, 2016 IEEE International Symposium on Information Theory (ISIT).

[2]  Michael J. Todd,et al.  Polynomial Algorithms for Linear Programming , 1988 .

[3]  Satoru Iwata,et al.  A combinatorial strongly polynomial algorithm for minimizing submodular functions , 2001, JACM.

[4]  Chung Chan,et al.  Multivariate Mutual Information Inspired by Secret-Key Agreement , 2015, Proceedings of the IEEE.

[5]  Alexander Schrijver,et al.  A Combinatorial Algorithm Minimizing Submodular Functions in Strongly Polynomial Time , 2000, J. Comb. Theory B.

[6]  Vladimir Kolmogorov,et al.  A Faster Algorithm for Computing the Principal Sequence of Partitions of a Graph , 2010, Algorithmica.

[7]  J. G. Pierce,et al.  Geometric Algorithms and Combinatorial Optimization , 2016 .

[8]  Srinivas Shakkottai,et al.  Harnessing multiple wireless interfaces for guaranteed QoS in proximate P2P networks , 2012, 2012 1st IEEE International Conference on Communications in China (ICCC).

[9]  Chung Chan,et al.  Linear perfect secret key agreement , 2011, 2011 IEEE Information Theory Workshop.

[10]  R. E. Bixby,et al.  The Partial Order of a Polymatroid Extreme Point , 1985, Math. Oper. Res..

[11]  Satoru Fujishige,et al.  Theory of Principal Partitions Revisited , 2008, Bonn Workshop of Combinatorial Optimization.

[12]  Bike Xie,et al.  Optimal exchange of packets for universal recovery in broadcast networks , 2010, 2010 - MILCOM 2010 MILITARY COMMUNICATIONS CONFERENCE.

[13]  H. Narayanan.  Submodular functions and electrical networks , 1997 .

[14]  R. Wesel,et al.  Efficient universal recovery in broadcast networks , 2010, 2010 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[15]  Rodney A. Kennedy,et al.  A game-theoretic perspective on communication for omniscience , 2015, 2016 Australian Communications Theory Workshop (AusCTW).

[16]  Alexander Sprintson,et al.  A randomized algorithm and performance bounds for coded cooperative data exchange , 2010, 2010 IEEE International Symposium on Information Theory.

[17]  B. Korte,et al.  Mathematical Programming the State of the Art: Bonn 1982 , 1983 .

[18]  Alexander Sprintson,et al.  On coding for cooperative data exchange , 2010, 2010 IEEE Information Theory Workshop on Information Theory (ITW 2010, Cairo).

[19]  Swanand Kadhe,et al.  Successive omniscience , 2015, 2015 International Symposium on Network Coding (NetCod).

[20]  Kannan Ramchandran,et al.  Deterministic algorithm for the cooperative data exchange problem , 2011, 2011 IEEE International Symposium on Information Theory Proceedings.

[21]  Eddie Cheng,et al.  A Faster Algorithm for Computing the Strength of a Network , 1994, Inf. Process. Lett..

[22]  H. Narayanan,et al.  The principal lattice of partitions of a submodular function , 1991 .

[23]  Philip Wolfe,et al.  Finding the nearest point in A polytope , 1976, Math. Program..

[24]  Toshihide Ibaraki,et al.  Resource allocation problems - algorithmic approaches , 1988, MIT Press series in the foundations of computing.

[25]  R. P. Dilworth.  Dependence Relations in a Semi-Modular Lattice , 1944 .

[26]  Kiyohito Nagano,et al.  A strongly polynomial algorithm for line search in submodular polyhedra , 2007, Discret. Optim..

[27]  Kazuyuki Aihara,et al.  Equivalence of convex minimization problems over base polytopes , 2012 .

[28]  Kannan Ramchandran,et al.  Efficient algorithms for the data exchange problem under fairness constraints , 2012, 2012 50th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[29]  Kannan Ramchandran,et al.  Efficient Algorithms for the Data Exchange Problem , 2015, IEEE Transactions on Information Theory.

[30]  Jack Edmonds,et al.  Submodular Functions, Matroids, and Certain Polyhedra , 2001, Combinatorial Optimization.

[31]  László Lovász,et al.  Submodular functions and convexity , 1982, ISMP.

[32]  S. Fujishige,et al.  A Submodular Function Minimization Algorithm Based on the Minimum-Norm Base ⁄ , 2009 .

[33]  Raymond W. Yeung,et al.  Information Theory and Network Coding , 2008 .

[34]  Alexander Schrijver,et al.  Combinatorial optimization. Polyhedra and efficiency. , 2003 .

[35]  Rodney A. Kennedy,et al.  Estimating minimum sum-rate for cooperative data exchange , 2015, 2015 IEEE International Symposium on Information Theory (ISIT).

[36]  Rodney A. Kennedy,et al.  Iterative merging algorithm for cooperative data exchange , 2015, 2015 International Symposium on Network Coding (NetCod).

[37]  William H. Cunningham,et al.  Optimal attack and reinforcement of a network , 1985, JACM.

[38]  Chung Chan,et al.  Info-Clustering: A Mathematical Theory for Data Clustering , 2016, IEEE Transactions on Molecular, Biological and Multi-Scale Communications.

[39]  Andreas Krause,et al.  SFO: A Toolbox for Submodular Function Optimization , 2010, J. Mach. Learn. Res..

[40]  Satoru Fujishige,et al.  Structures of polyhedra determined by submodular functions on crossing families , 1984, Math. Program..

[41]  Prakash Narayan,et al.  Perfect Omniscience, Perfect Secrecy, and Steiner Tree Packing , 2010, IEEE Transactions on Information Theory.

[42]  Richard D. Wesel,et al.  Weighted universal recovery, practical secrecy, and an efficient algorithm for solving both , 2011, 2011 49th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[43]  David R. Karger,et al.  Deterministic network coding by matrix completion , 2005, SODA '05.

[44]  Parastoo Sadeghi,et al.  A Generalized Model for Cost and Fairness Analysis in Coded Cooperative Data Exchange , 2011, 2011 International Symposium on Networking Coding.

[45]  Satoru Iwata,et al.  Submodular function minimization , 2007, Math. Program..

[46]  藤重 悟.  Submodular functions and optimization , 1991 .

[47]  Satoru Fujishige,et al.  Principal structures of submodular systems , 1980, Discret. Appl. Math..

[48]  Yoshinobu Kawahara,et al.  Structured Convex Optimization under Submodular Constraints , 2013, UAI.

[49]  Axthonv G. Oettinger,et al.  IEEE Transactions on Information Theory , 1998 .

[50]  Satoko Moriguchi,et al.  On Hochbaum's Proximity-Scaling Algorithm for the General Resource Allocation Problem , 2004, Math. Oper. Res..

[51]  Rodney A. Kennedy,et al.  A Faster Algorithm for Asymptotic Communication for Omniscience , 2016, 2016 IEEE Globecom Workshops (GC Wkshps).

[52]  Rodney A. Kennedy,et al.  Fairest constant sum-rate transmission for cooperative data exchange: An M-convex minimization approach , 2015, 2015 22nd International Conference on Telecommunications (ICT).

[53]  James B. Orlin,et al.  A faster strongly polynomial time algorithm for submodular function minimization , 2007, Math. Program..

[54]  Satoru Fujishige,et al.  Polymatroidal Dependence Structure of a Set of Random Variables , 1978, Inf. Control..

[55]  Satoru Fujishige,et al.  Lexicographically Optimal Base of a Polymatroid with Respect to a Weight Vector , 1980, Math. Oper. Res..

[56]  Jeff A. Bilmes,et al.  Q-Clustering , 2005, NIPS.

[57]  F. Barahona,et al.  Partition Inequalities: Separation, Extensions and Network Design, , 2011 .

[58]  Martin Grötschel,et al.  The ellipsoid method and its consequences in combinatorial optimization , 1981, Comb..

[59]  Satoru Iwata,et al.  Minimum Average Cost Clustering , 2010, NIPS.

[60]  Nobuyuki Tsuchimura,et al.  M-Convex Function Minimization by Continuous Relaxation Approach: Proximity Theorem and Algorithm , 2011, SIAM J. Optim..

[61]  Yoshio Okamoto,et al.  Submodular fractional programming for balanced clustering , 2011, Pattern Recognit. Lett..

[62]  Alexander Sprintson,et al.  An algorithm for cooperative data exchange with cost criterion , 2011, 2011 Information Theory and Applications Workshop.

[63]  Richard D. Wesel,et al.  Coded Cooperative Data Exchange in Multihop Networks , 2012, IEEE Transactions on Information Theory.

[64]  Imre Csiszár,et al.  Secrecy capacities for multiple terminals , 2004, IEEE Transactions on Information Theory.