Direct Reinforcement Learning for Autonomous Power Configuration and Control in Wireless Networks

In this paper, non deterministic Direct Reinforcement Learning (RL) for controlling the transmission times and power of a Wireless Sensor Network (WSN) node is presented. RL allows for truly autonomous optimal behaviour of agents by requiring no models or supervision to learn. Optimal actions are learnt by repeated interactions with the environment. Performance results are presented for Monte Carlo, TD0 and TDλ. The resultant optimal learned policies are shown to out perform static power control in a stochastic environment.

[1]  Vikas Kawadia,et al.  Power control and clustering in ad hoc networks , 2003, IEEE INFOCOM 2003. Twenty-second Annual Joint Conference of the IEEE Computer and Communications Societies (IEEE Cat. No.03CH37428).

[2]  Ram Ramanathan,et al.  Topology control of multihop wireless networks using transmit power adjustment , 2000, Proceedings IEEE INFOCOM 2000. Conference on Computer Communications. Nineteenth Annual Joint Conference of the IEEE Computer and Communications Societies (Cat. No.00CH37064).

[3]  S. M. Heemstra de Groot,et al.  Power-aware routing in mobile ad hoc networks , 1998, MobiCom '98.

[4]  P. R. Kumar,et al.  Power Control in Ad-Hoc Networks: Theory, Architecture, Algorithm and Implementation of the COMPOW Protocol , 2002 .

[5]  Li Li,et al.  Distributed topology control for power efficient operation in multihop wireless ad hoc networks , 2001, Proceedings IEEE INFOCOM 2001. Conference on Computer Communications. Twentieth Annual Joint Conference of the IEEE Computer and Communications Society (Cat. No.01CH37213).

[6]  Mário Serafim Nunes,et al.  Minimizing Interference in TDMA MAC Protocols for WSN Operating in Shadow-Fading Channels , 2008, EuroNGI Workshop.

[7]  Yu-Chee Tseng,et al.  Intelligent medium access for mobile ad hoc networks with busy tones and power control , 2000, IEEE Journal on Selected Areas in Communications.

[8]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[9]  A. Speranzon,et al.  On power control for wireless sensor networks: System model, middleware component and experimental evaluation , 2007, 2007 European Control Conference (ECC).

[10]  Andrea J. Goldsmith,et al.  Capacity and optimal resource allocation for fading broadcast channels - Part I: Ergodic capacity , 2001, IEEE Trans. Inf. Theory.

[11]  Mohamed F. Younis,et al.  A survey on routing protocols for wireless sensor networks , 2005, Ad Hoc Networks.

[12]  Srikanth V. Krishnamurthy,et al.  Power management for throughput enhancement in wireless ad-hoc networks , 2000, 2000 IEEE International Conference on Communications. ICC 2000. Global Convergence Through Communications. Conference Record.

[13]  Ramesh Govindan,et al.  Understanding packet delivery performance in dense wireless sensor networks , 2003, SenSys '03.

[14]  Vaduvur Bharghavan,et al.  A power controlled multiple access protocol for wireless packet networks , 2001, Proceedings IEEE INFOCOM 2001. Conference on Computer Communications. Twentieth Annual Joint Conference of the IEEE Computer and Communications Society (Cat. No.01CH37213).

[15]  Madhavi W. Subbarao.  Dynamic power-conscious routing for MANETs: an initial approach , 1999, Gateway to 21st Century Communications Village. VTC 1999-Fall. IEEE VTS 50th Vehicular Technology Conference (Cat. No.99CH36324).

[16]  Gang Zhou,et al.  RID: radio interference detection in wireless sensor networks , 2005, Proceedings IEEE 24th Annual Joint Conference of the IEEE Computer and Communications Societies..

[17]  Qun Li,et al.  Online power-aware routing in wireless Ad-hoc networks , 2001, MobiCom '01.

[18]  P. R. Kumar,et al.  Principles and protocols for power control in wireless ad hoc networks , 2005 .

[19]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .

[20]  Klaus D. McDonald-Maier,et al.  Partially Observable Markov Decision Process for Transmitter Power Control in Wireless Sensor Networks , 2008, 2008 Bio-inspired, Learning and Intelligent Systems for Security.

[21]  Richard S. Sutton,et al.  Dimensions of Reinforcement Learning , 1998 .

[22]  Andreas Savvides,et al.  An Empirical Characterization of Radio Signal Strength Variability in 3-D IEEE 802.15.4 Networks Using Monopole Antennas , 2006, EWSN.

[23]  Teresa H. Y. Meng,et al.  Minimum energy mobile wireless networks , 1999, IEEE J. Sel. Areas Commun..

[24]  Yu-Chee Tseng,et al.  Intelligent medium access for mobile ad hoc networks with busy tones and power control , 2000, IEEE J. Sel. Areas Commun..

[25]  Antonio Alfredo Ferreira Loureiro,et al.  Transmission power control in MAC protocols for wireless sensor networks , 2005, MSWiM '05.