Reinforcement Learning Algorithm for Solving Load Commitment Problem Considering a General Load Model

Under the smart grid paradigm, in the near future all consumers will be exposed to variable pricing schemes introduced by utilities. Hence, there is a need to develop algorithms which could be used by the consumers to schedule their loads. In this paper, load scheduling problem is formulated as a LCP (load commitment problem). The load model is general and can model atomic and non-atomic loads. Furthermore, it can also take into consideration the relative discomfort caused by delay in scheduling any load. For this purpose, a single parameter “udc” is introduced in the load model which captures the relative discomfort caused by delay in scheduling a particular load. Guidelines for choosing this parameter are given. All the other parameters of the proposed load model can be easily specified by the consumer. The paper shows that the general LCP can be viewed as multi-stage decision making problem or a MDP (Markov decision problem). RL (reinforcement learning) based algorithm is developed to solve this problem. The efficacy of the algorithm is investigated when the price of electricity is available in advance as well as for the case when it is random. The scalability of the approach is also investigated.

[1]  C.W. Gellings,et al.  The concept of demand-side management for electric utilities , 1985, Proceedings of the IEEE.

[2]  Mohammed H. Albadi,et al.  A summary of demand response in electricity markets , 2008 .

[3]  P. S. Nagendra Rao,et al.  A reinforcement learning approach to automatic generation control , 2002 .

[4]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[5]  Q.B. Dam,et al.  Intelligent Demand Response Scheme for Customer Side Load Management , 2008, 2008 IEEE Energy 2030 Conference.

[6]  H. Vincent Poor,et al.  Scheduling Power Consumption With Price Uncertainty , 2011, IEEE Transactions on Smart Grid.

[7]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[8]  E. A. Jasmin,et al.  Reinforcement Learning approaches to Economic Dispatch problem , 2011 .

[9]  James Won-Ki Hong,et al.  Near optimal demand-side energy management under real-time demand-response pricing , 2010, 2010 International Conference on Network and Service Management.

[10]  Allen J. Wood,et al.  Power Generation, Operation, and Control , 1984 .

[11]  M. Thathachar,et al.  Networks of Learning Automata: Techniques for Online Stochastic Optimization , 2003 .

[12]  Hamed Mohsenian Rad,et al.  Optimal Residential Load Control With Price Prediction in Real-Time Electricity Pricing Environments , 2010, IEEE Transactions on Smart Grid.

[13]  Kathleen L. Spees,et al.  Demand Response and Electricity Market Efficiency , 2007 .

[14]  Marco Levorato,et al.  Residential Demand Response Using Reinforcement Learning , 2010, 2010 First IEEE International Conference on Smart Grid Communications.

[15]  Ning Lu,et al.  Appliance Commitment for Household Load Scheduling , 2011, IEEE Transactions on Smart Grid.