Distributed Q-Learning for Interference Mitigation in Self-Organised Femtocell Networks: Synchronous or Asynchronous?

Femtocells promise to improve the quality of indoor wireless communications substantially. However, a serious interference problem arises with universal frequency reuse. In this paper, an asynchronous dynamic power allocation among femtocells based on Q-learning is proposed to mitigate the interference in the network. Simulation results show that in the high femtocells density deployment, asynchronous decision-making process has better performance than the synchronous one in terms of both performance degradation of the macrocell and average capacity of femtocells. In addition, it is shown that our method has superiority to smart power control algorithm proposed by 3GPP when femtocell occupation ratio is over 53 %.

[1]  Jeffrey G. Andrews,et al.  Downlink Femtocell Networks: Open or Closed? , 2011, 2011 IEEE International Conference on Communications (ICC).

[2]  Jeffrey G. Andrews,et al.  Spectrum allocation in tiered cellular networks , 2009, IEEE Transactions on Communications.

[3]  Jie Zhang,et al.  Access control mechanisms for femtocells , 2010, IEEE Communications Magazine.

[4]  Sean Luke,et al.  Cooperative Multi-Agent Learning: The State of the Art , 2005, Autonomous Agents and Multi-Agent Systems.

[5]  Kai Ma,et al.  Power Control Based on Maximum Power Adaptation in Two-Tier Femtocell Networks , 2013, Wirel. Pers. Commun..

[6]  Ana Galindo-Serrano,et al.  Femtocell systems with self organization capabilities , 2011, International Conference on NETwork Games, Control and Optimization (NetGCooP 2011).

[7]  Pin-Han Ho,et al.  A Novel Approach for Co-Channel Interference Mitigation in Femtocell Networks , 2011, 2011 IEEE Global Telecommunications Conference - GLOBECOM 2011.

[8]  Holger Claussen,et al.  Performance of Macro- and Co-Channel Femtocells in a Hierarchical Cell Structure , 2007, 2007 IEEE 18th International Symposium on Personal, Indoor and Mobile Radio Communications.

[9]  Ana Galindo-Serrano,et al.  Downlink femto-to-macro interference management based on Fuzzy Q-Learning , 2011, 2011 International Symposium of Modeling and Optimization of Mobile, Ad Hoc, and Wireless Networks.

[10]  Ana Galindo-Serrano,et al.  On implementation requirements and performances of Q-Learning for self-organized femtocells , 2011, 2011 IEEE GLOBECOM Workshops (GC Wkshps).

[11]  Oriol Sallent,et al.  Intercell Interference Management in OFDMA Networks: A Decentralized Approach Based onReinforcement Learning , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[12]  Yanzan Sun,et al.  Uplink Interference Mitigation for OFDMA Femtocell Networks , 2012, IEEE Transactions on Wireless Communications.

[13]  K. P. Sycara.  Multiagent systems : Special issue on agents , 1998 .

[14]  Kamran Arshad,et al.  Interference Management in Femtocells , 2013, IEEE Communications Surveys & Tutorials.

[15]  Christian Ibars,et al.  Distributed interference control in OFDMA-based femtocells , 2010, 21st Annual IEEE International Symposium on Personal, Indoor and Mobile Radio Communications.

[16]  H. Vincent Poor,et al.  Compressive autonomous sensing (CASe) for wideband spectrum sensing , 2012, 2012 IEEE International Conference on Communications (ICC).

[17]  Gerhard Weiss,et al.  Multiagent Systems , 1999 .

[18]  Jie Zhang,et al.  OFDMA femtocells: A roadmap on interference avoidance , 2009, IEEE Communications Magazine.

[19]  R. Bellman.  Dynamic programming. , 1957, Science.

[20]  Peter Dayan,et al.  Technical Note: Q-Learning , 2004, Machine Learning.

[21]  Wei-Chen Pao,et al.  Power Allocation Schemes in OFDM-Based Femtocell Networks , 2013, Wirel. Pers. Commun..

[22]  Jeffrey G. Andrews,et al.  Open vs. Closed Access Femtocells in the Uplink , 2010, IEEE Transactions on Wireless Communications.