Radio Resource Management with MDP

The GC scheme was proved by Ramjee et al.[6] to be optimal for an objective function formed by a linear weighting of the new call blocking and the handoff dropping probabilities. However, the GC scheme is not optimal for multiple traffic classes. In order to achieve better QoS, it is necessary to have a stochastic control policy. One well known stochastic control approach is the Markov Decision Process (MDP), which is a branch from operation research. MDP models are widely used in diverse research areas and practical applications such as ecology [7, 8], economics [9], and network routing [10]. Extensive application and examples are given in [11, 12]. In this chapter, we focus on the application of MDP to the design of the call admission control policy.

[1]  Ronald A. Howard,et al.  Dynamic Programming and Markov Processes , 1960 .

[2]  R. Bellman.  Dynamic programming. , 1957, Science.

[3]  Narendra Karmarkar,et al.  A new polynomial-time algorithm for linear programming , 1984, Comb..

[4]  Stephen S. Rappaport,et al.  Traffic model and performance analysis for cellular mobile radio telephone systems with prioritized and nonprioritized handoff procedures , 1986, IEEE Transactions on Vehicular Technology.

[5]  Henk Tijms,et al.  Stochastic modelling and analysis: a computational approach , 1986 .

[6]  Alasdair I. Houston,et al.  Singing to attract a mate: a stochastic dynamic game , 1987 .

[7]  C. Clark.  The lazy, adaptable lions: a Markovian model of group foraging , 1987, Animal Behaviour.

[8]  James Renegar,et al.  A polynomial-time algorithm, based on Newton's method, for linear programming , 1988, Math. Program..

[9]  C. C. Gonzaga,et al.  An Algorithm for Solving Linear Programming Problems in O(n 3 L) Operations , 1989 .

[10]  Pravin M. Vaidya,et al.  An algorithm for linear programming which requires O(((m+n)n2+(m+n)1.5n)L) arithmetic operations , 1990, Math. Program..

[11]  Andrew J. Viterbi,et al.  Erlang Capacity of a Power Controlled CDMA System , 1993, IEEE J. Sel. Areas Commun..

[12]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[13]  Armand M. Makowski,et al.  Stochastic Control of Handoffs in Cellular Networks , 1995, IEEE J. Sel. Areas Commun..

[14]  Leslie Pack Kaelbling,et al.  On the Complexity of Solving Markov Decision Problems , 1995, UAI.

[15]  Don Towsley,et al.  On optimal call admission control in cellular networks , 1996, Proceedings of IEEE INFOCOM '96. Conference on Computer Communications.

[16]  Moshe Sidi,et al.  New call blocking versus handoff blocking in cellular networks , 1997, Wirel. Networks.

[17]  David McFarlane,et al.  IMT-2000: service provider's perspective , 1997, IEEE Wirel. Commun..

[18]  Masami Yabusaki,et al.  IMT-2000 standards: network aspects , 1997, IEEE Wirel. Commun..

[19]  Judy Goldsmith,et al.  Complexity issues in Markov decision processes , 1998, Proceedings. Thirteenth Annual IEEE Conference on Computational Complexity (Formerly: Structure in Complexity Theory Conference) (Cat. No.98CB36247).

[20]  Irene Katzela.  Modeling and Simulating Communication Networks: A Hands-on Approach Using Opnet , 1998 .

[21]  Chin-Tau Lea,et al.  Finding better call admission policies in wireless networks , 1998, VTC '98. 48th IEEE Vehicular Technology Conference. Pathway to Global Wireless Revolution (Cat. No.98CH36151).

[22]  Kurt M. Anstreicher,et al.  Linear Programming in O([n3/ln n]L) Operations , 1999, SIAM J. Optim..

[23]  Ronald E. Miller.  Optimization: Foundations and Applications , 1999 .

[24]  C.-C. Jay Kuo,et al.  Differentiated QoS-aware priority handoff in cell-based multimedia wireless network , 2000, Electronic Imaging.

[25]  Yanghee Choi,et al.  Call admission control for multimedia services in mobile cellular networks: a Markov decision approach , 2000, Proceedings ISCC 2000. Fifth IEEE Symposium on Computers and Communications.

[26]  Yan Wang,et al.  An optimal distributed call admission control for adaptive multimedia in wireless/mobile networks , 2000, Proceedings 8th International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (Cat. No.PR00728).

[27]  Magnus Almgren,et al.  Radio Resource Management for Wireless Networks , 2001 .

[28]  C.-C. Jay Kuo,et al.  Dynamic call admission control scheme for QoS priority handoff in multimedia cellular systems , 2002, 2002 IEEE Wireless Communications and Networking Conference Record. WCNC 2002 (Cat. No.02TH8609).