Average Age of Information With Hybrid ARQ Under a Resource Constraint

Scheduling the transmission of status updates over an error-prone communication channel is studied in order to minimize the long-term average age of information at the destination under a constraint on the average number of transmissions at the source node. After each transmission, the source receives an instantaneous ACK/NACK feedback, and decides on the next update without prior knowledge on the success of future transmissions. The optimal scheduling policy is first studied under different feedback mechanisms when the channel statistics are known; in particular, the standard automatic repeat request (ARQ) and hybrid ARQ (HARQ) protocols are considered. The structural results are derived for the optimal policy under HARQ, while the optimal policy is determined analytically for ARQ. For the case of unknown environments, an average-cost reinforcement learning algorithm is proposed that learns the system parameters and the transmission policy in real time. The effectiveness of the proposed methods is verified through the numerical results.

[1]  Linn I. Sennott,et al.  Constrained Average Cost Markov Decision Chains , 1993, Probability in the Engineering and Informational Sciences.

[2]  Elif Uysal-Biyikoglu,et al.  Age of information under energy replenishment constraints , 2015, 2015 Information Theory and Applications Workshop (ITA).

[3]  Deniz Gündüz,et al.  Average age of information with hybrid ARQ under a resource constraint , 2017, 2018 IEEE Wireless Communications and Networking Conference (WCNC).

[4]  Roy D. Yates,et al.  Real-time status: How often should one update? , 2012, 2012 Proceedings IEEE INFOCOM.

[5]  Stefan Parkvall,et al.  Performance comparison of HARQ with Chase combining and incremental redundancy for HSDPA , 2001, IEEE 54th Vehicular Technology Conference. VTC Fall 2001. Proceedings (Cat. No.01CH37211).

[6]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[7]  Roy D. Yates,et al.  Update or wait: How to keep your data fresh , 2016, IEEE INFOCOM 2016 - The 35th Annual IEEE International Conference on Computer Communications.

[8]  Parimal Parag,et al.  On Real-Time Status Updates over Symbol Erasure Channels , 2017, 2017 IEEE Wireless Communications and Networking Conference (WCNC).

[9]  E. Altman.  Constrained Markov Decision Processes , 1999 .

[10]  Eytan Modiano,et al.  Age of information: Design and analysis of optimal scheduling algorithms , 2016, 2017 IEEE International Symposium on Information Theory (ISIT).

[11]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[12]  Singh Rahul,et al.  Minimizing the Age of Information in broadcast wireless networks , 2016 .

[13]  Deniz Gündüz,et al.  Designing intelligent energy harvesting communication systems , 2014, IEEE Communications Magazine.

[14]  Sheldon M. Ross,et al.  Introduction to Probability Models (4th ed.). , 1990 .

[15]  Xavier Lagrange,et al.  Throughput of HARQ protocols on a block fading channel , 2010, IEEE Communications Letters.

[16]  Roy D. Yates,et al.  Status updates through M/G/1/1 queues with HARQ , 2017, 2017 IEEE International Symposium on Information Theory (ISIT).

[17]  Rachid El Azouzi,et al.  Optimal sensing policies for smartphones in hybrid networks: A POMDP approach , 2012, 6th International ICST Conference on Performance Evaluation Methodologies and Tools.

[18]  Sanjit Krishnan Kaul,et al.  Minimizing age of information in vehicular networks , 2011, 2011 8th Annual IEEE Communications Society Conference on Sensor, Mesh and Ad Hoc Communications and Networks.

[19]  Harold J. Kushner,et al.  Stochastic Approximation Algorithms and Applications , 1997, Applications of Mathematics.

[20]  SRIDHAR MAHADEVAN,et al.  Average Reward Reinforcement Learning: Foundations, Algorithms, and Empirical Results , 2005, Machine Learning.

[21]  Michael L. Honig,et al.  Reliability-based type II hybrid ARQ schemes , 2003, IEEE International Conference on Communications, 2003. ICC '03..

[22]  Eitan Altman,et al.  Forever Young: Aging Control For Smartphones In Hybrid Networks , 2010 .

[23]  Kun Chen,et al.  Age-of-information in the presence of error , 2016, 2016 IEEE International Symposium on Information Theory (ISIT).

[24]  Roy D. Yates,et al.  Timely updates over an erasure channel , 2017, 2017 IEEE International Symposium on Information Theory (ISIT).

[25]  Sheldon M. Ross.  Introduction to Probability Models. , 1995 .

[26]  Rachid El Azouzi,et al.  Forever Young: Aging Control For Hybrid Networks , 2010, MobiHoc.

[27]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[28]  Linn I. Sennott,et al.  Average Cost Optimal Stationary Policies in Infinite State Markov Decision Processes with Unbounded Costs , 1989, Oper. Res..