A dynamic channel assignment policy through Q-learning

One of the fundamental issues in the operation of a mobile communication system is the assignment of channels to cells and to calls. Since the number of channels allocated to a mobile communication system is limited, efficient utilization of these communication channels by using efficient channel assignment strategies is not only desirable but also imperative. This paper presents a novel approach to solving the dynamic channel assignment (DCA) problem by using a form of realtime reinforcement learning known as Q-learning in conjunction with neural network representation. Instead of relying on a known teacher, the system is designed to learn an optimal channel assignment policy by directly interacting with the mobile communication environment. The performance of the Q-learning-based DCA was examined by extensive simulation studies on a 49-cell mobile communication system under various conditions. Comparative studies with the fixed channel assignment (FCA) scheme and one of the best dynamic channel assignment strategies, MAXAVAIL, have revealed that the proposed approach is able to perform better than the FCA in various situations and capable of achieving a performance similar to that achieved by the MAXIAVIAL, but with a significantly reduced computational complexity.

[1]  R. Bellman Dynamic programming. , 1957, Science.

[2]  D. O. Reudink,et al.  Dynamic channel assignment in two-dimensional large-scale mobile Radio systems , 1972 .

[3]  V. H. Mac Donald,et al.  Advanced mobile phone service: The cellular concept , 1979, The Bell System Technical Journal.

[4]  Roger L. Freeman,et al.  Telecommunication System Engineering , 1980 .

[5]  Ming Zhang,et al.  Comparisons of channel-assignment strategies in cellular mobile telephone systems , 1989 .

[6]  Kumar N. Sivarajan,et al.  Dynamic channel assignment in cellular radio , 1990, 40th IEEE Conference on Vehicular Technology.

[7]  D. Kunz,et al.  Channel assignment for cellular radio using neural networks , 1991 .

[8]  S. Tekinay,et al.  Handover and channel assignment in mobile cellular networks , 1991, IEEE Communications Magazine.

[9]  Ming Zhang,et al.  The nonuniform compact pattern allocation algorithm cellular mobile systems , 1991 .

[10]  Justin C. Chuang,et al.  Performance issues and algorithms for dynamic channel assignment , 1992, [Conference Record] GLOBECOM '92 - Communications for Global Users: IEEE.

[11]  Gerald Tesauro,et al.  Practical Issues in Temporal Difference Learning , 1992, Mach. Learn..

[12]  Dietmar Kunz,et al.  Channel assignment for cellular radio using simulated annealing , 1993 .

[13]  Justin C.-I. Chuang Performance Issues and Algorithms for Dynamic Channel Assignment , 1993, IEEE J. Sel. Areas Commun..

[14]  D. D. Dimitrijevic,et al.  Design and performance analysis of the algorithms for channel allocation in cellular networks , 1993 .

[15]  S. Hyakin,et al.  Neural Networks: A Comprehensive Foundation , 1994 .

[16]  D. Everitt,et al.  Traffic engineering of the radio interface for cellular mobile networks , 1994, Proc. IEEE.

[17]  Marimuthu Palaniswami,et al.  Neural network-based dynamic channel assignment for cellular mobile communication systems , 1994 .

[18]  Ben J. A. Kröse,et al.  Learning from delayed rewards , 1995, Robotics Auton. Syst..

[19]  Derek A. Linkens,et al.  Fuzzy-neural control: principles, algorithms and applications , 1995 .

[20]  Andrew G. Barto,et al.  Improving Elevator Performance Using Reinforcement Learning , 1995, NIPS.

[21]  Andrea Baiocchi,et al.  The geometric dynamic channel allocation as a practical strategy in mobile networks with bursty user mobility , 1995 .

[22]  Andrew G. Barto,et al.  Learning to Act Using Real-Time Dynamic Programming , 1995, Artif. Intell..

[23]  William C. Y. Lee,et al.  Mobile Cellular Telecommunications , 1995 .

[24]  Romano Fantacci,et al.  A dynamic channel allocation technique based on Hopfield neural networks , 1996 .

[25]  Thomas G. Dietterich What is machine learning? , 2020, Archives of Disease in Childhood.

[26]  G. G. Coghill,et al.  Channel assignment through evolutionary optimization , 1996 .