Overhead Performance Tradeoffs—A Resource Allocation Perspective

A key aspect of many resource allocation problems is the need for the resource controller to compute a function, such as the max or arg max, of the competing users' metrics. Information must be exchanged between the competing users and the resource controller in order for this function to be computed. In many practical resource controllers, the competing users' metrics are communicated to the resource controller, which then computes the desired extremization function. However, in this paper, it is shown that information rate savings can be obtained by recognizing that the controller only needs to determine the result of this extremization function. If the extremization function is to be computed losslessly, the rate savings are shown in most cases to be at most 2 b independent of the number of competing users. Motivated by the small savings in the lossless case, simple achievable schemes for both the lossy and interactive variants of this problem are considered. It is shown that both of these approaches have the potential to realize large rate savings, especially in the case where the number of competing users is large. For the lossy variant, it is shown that the proposed simple achievable schemes are in fact close to the fundamental limit given by the rate distortion function.

[1]  Richard E. Blahut,et al.  Computation of channel capacity and rate-distortion functions , 1972, IEEE Trans. Inf. Theory.

[2]  H. S. WITSENHAUSEN,et al.  The zero-error side information problem and chromatic numbers (Corresp.) , 1976, IEEE Trans. Inf. Theory.

[3]  Prakash Ishwar,et al.  Interaction strictly improves the Wyner-Ziv rate-distortion function , 2010, 2010 IEEE International Symposium on Information Theory.

[4]  Michael Gastpar,et al.  The Wyner-Ziv problem with multiple sources , 2004, IEEE Transactions on Information Theory.

[5]  Venkat Anantharam,et al.  An improved outer bound for multiterminal source coding , 2008, IEEE Transactions on Information Theory.

[6]  D. Katabi,et al.  A framework for scalable global IP-anycast (GIA) , 2001, CCRV.

[7]  S.D. Servetto Achievable Rates for Multiterminal Source Coding with Scalar Quantizers , 2005, Conference Record of the Thirty-Ninth Asilomar Conference onSignals, Systems and Computers, 2005..

[8]  D. Gesbert,et al.  Exploiting multiuser diversity using multiple feedback thresholds , 2005, 2005 IEEE 61st Vehicular Technology Conference.

[9]  R. Gallager Information Theory and Reliable Communication , 1968 .

[10]  Toby Berger,et al.  Multiterminal Source Coding with High Resolution , 1999, IEEE Trans. Inf. Theory.

[11]  Piyush Gupta,et al.  Interactive Source Coding for Function Computation in Collocated Networks , 2012, IEEE Transactions on Information Theory.

[12]  Alon Orlitsky,et al.  Coding for computing , 1995, Proceedings of IEEE 36th Annual Foundations of Computer Science.

[13]  Amiram H. Kaspi,et al.  Two-way source coding with a fidelity criterion , 1985, IEEE Trans. Inf. Theory.

[14]  Devavrat Shah,et al.  Functional Compression Through Graph Coloring , 2010, IEEE Transactions on Information Theory.

[15]  Pramod Viswanath,et al.  Rate Region of the Quadratic Gaussian Two-Encoder Source-Coding Problem , 2005, IEEE Transactions on Information Theory.

[16]  Vivek K. Goyal,et al.  Distributed Scalar Quantization for Computing: High-Resolution Analysis and Extensions , 2008, IEEE Transactions on Information Theory.

[17]  Aslan Tchamkerten,et al.  Distributed Function Computation Over a Rooted Directed Tree , 2016, IEEE Transactions on Information Theory.

[18]  Jie Ren,et al.  Computing the Rate Distortion Region for the CEO Problem With Independent Sources , 2015, IEEE Transactions on Signal Processing.

[19]  Vinod M. Prabhakaran,et al.  Rate region of the quadratic Gaussian CEO problem , 2004, International Symposium onInformation Theory, 2004. ISIT 2004. Proceedings..

[20]  Jeremy Kleidosty,et al.  Theory of Justice , 2017 .

[21]  Michael Luby,et al.  LT codes , 2002, The 43rd Annual IEEE Symposium on Foundations of Computer Science, 2002. Proceedings..

[22]  Prakash Ishwar,et al.  Some Results on Distributed Source Coding for Interactive Function Computation , 2011, IEEE Transactions on Information Theory.

[23]  Jack K. Wolf,et al.  Noiseless coding of correlated information sources , 1973, IEEE Trans. Inf. Theory.

[24]  Toby Berger,et al.  The CEO problem [multiterminal source coding] , 1996, IEEE Trans. Inf. Theory.

[25]  Jie Ren,et al.  Interactive communication for resource allocation , 2014, 2014 48th Annual Conference on Information Sciences and Systems (CISS).

[26]  Giuseppe Longo,et al.  The information theory approach to communications , 1977 .

[27]  M. Médard,et al.  Graph Coloring and Conditional Graph Entropy , 2006, 2006 Fortieth Asilomar Conference on Signals, Systems and Computers.

[28]  Aaron D. Wyner,et al.  Coding Theorems for a Discrete Source With a Fidelity CriterionInstitute of Radio Engineers, International Convention Record, vol. 7, 1959. , 1993 .

[29]  Shlomo Shamai,et al.  Variable-Rate Channel Capacity , 2010, IEEE Transactions on Information Theory.

[30]  Aslan Tchamkerten,et al.  Distributed function computation over a tree network , 2013, 2013 IEEE Information Theory Workshop (ITW).

[31]  Aaron D. Wyner,et al.  The rate-distortion function for source coding with side information at the decoder , 1976, IEEE Trans. Inf. Theory.

[32]  Imre Csiszár,et al.  On the computation of rate-distortion functions (Corresp.) , 1974, IEEE Trans. Inf. Theory.

[33]  Suguru Arimoto,et al.  An algorithm for computing the capacity of arbitrary discrete memoryless channels , 1972, IEEE Trans. Inf. Theory.

[34]  Yasutada Oohama,et al.  Rate-distortion theory for Gaussian multiterminal source coding systems with several side informations at the decoder , 2005, IEEE Transactions on Information Theory.

[35]  Gregory W. Wornell,et al.  Rateless Coding for Gaussian Channels , 2007, IEEE Transactions on Information Theory.

[36]  Nariman Farvardin,et al.  Optimum quantizer performance for a class of non-Gaussian memoryless sources , 1984, IEEE Trans. Inf. Theory.

[37]  P. Boukris An upper bound on the speed of convergence of the Blahut algorithm for computing rate-distortion functions (Corresp.) , 1973, IEEE Trans. Inf. Theory.

[38]  Ravi Sankar,et al.  A Survey of Intrusion Detection Systems in Wireless Sensor Networks , 2014, IEEE Communications Surveys & Tutorials.

[39]  Mohamed-Slim Alouini,et al.  How much feedback is multi-user diversity really worth? , 2004, 2004 IEEE International Conference on Communications (IEEE Cat. No.04CH37577).

[40]  G. Blelloch Introduction to Data Compression * , 2022 .

[41]  Toby Berger,et al.  Rate distortion theory : a mathematical basis for data compression , 1971 .

[42]  Masoud Salehi,et al.  Communication Systems Engineering , 1994 .

[43]  Steven P. Weber,et al.  Distributed scalar quantizers for subband allocation , 2014, 2014 48th Annual Conference on Information Sciences and Systems (CISS).

[44]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[45]  E. Maasland,et al.  Auction Theory , 2021, Springer Texts in Business and Economics.