A Reinforcement-Learning Approach to Power Management

We describe an adaptive, mid-level approach to the wireless device power management problem. Our approach is based on reinforcement learning, a machine learning framework for autonomous agents. We describe how our framework can be applied to the power management problem in both infrastructure and ad hoc wireless networks. From this thesis we conclude that mid-level power management policies can outperform low-level policies and are more convenient to implement than high-level policies. We also conclude that power management policies need to adapt to the user and network, and that a mid-level power management framework based on reinforcement learning fulfills these requirements. Thesis Supervisor: Leslie Pack Kaelbling Title: Professor

[1]  Robert Tappan Morris,et al.  Span: An Energy-Efficient Coordination Algorithm for Topology Maintenance in Ad Hoc Wireless Networks , 2001, MobiCom '01.

[2]  Mahmoud Naghshineh,et al.  Bluetooth: vision, goals, and architecture , 1998, MOCO.

[3]  Luca Benini,et al.  Event-driven power management of portable systems , 1999, Proceedings 12th International Symposium on System Synthesis.

[4]  Suresh Singh,et al.  PAMAS—power aware multi-access protocol with signalling for ad hoc networks , 1998, CCRV.

[5]  J. Flinn,et al.  Energy-aware adaptation for mobile applications , 1999, SOSP.

[6]  Luca Benini,et al.  Policy optimization for dynamic power management , 1998, Proceedings 1998 Design and Automation Conference. 35th DAC. (Cat. No.98CH36175).

[7]  Jerome H. Saltzer,et al.  End-to-end arguments in system design , 1984, TOCS.

[8]  Andrew S. Tanenbaum,et al.  Computer networks, third edition , 1996 .

[9]  Massoud Pedram,et al.  Dynamic power management based on continuous-time Markov decision processes , 1999, DAC '99.

[10]  Robin Kravets,et al.  Application‐driven power management for mobile communication , 2000, Wirel. Networks.

[11]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .

[12]  Carla Schlatter Ellis,et al.  The case for higher-level power management , 1999, Proceedings of the Seventh Workshop on Hot Topics in Operating Systems.

[13]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[14]  Martin Nilsson,et al.  Investigating the energy consumption of a wireless network interface in an ad hoc networking environment , 2001, Proceedings IEEE INFOCOM 2001. Conference on Computer Communications. Twentieth Annual Joint Conference of the IEEE Computer and Communications Society (Cat. No.01CH37213).

[15]  Krishna M. Sivalingam,et al.  A Survey of Energy Efficient Network Protocols for Wireless Networks , 2001, Wirel. Networks.

[16]  Ronny Krashinsky Maintaining Performance while Saving Energy on Wireless LANs , 2001 .

[17]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[18]  Randy H. Katz,et al.  Measuring and Reducing Energy Consumption of Network Interfaces in Hand-Held Devices (Special Issue on Mobile Computing) , 1997 .

[19]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[20]  Giovanni De Micheli,et al.  Energy efficient design of portable wireless systems , 2000, ISLPED'00: Proceedings of the 2000 International Symposium on Low Power Electronics and Design (Cat. No.00TH8514).