Q-Learning with Eligibility Traces to Solve Non-Convex Economic Dispatch Problems

Economic Dispatch is one of the most important power system management tools. It is used to allocate an amount of power generation to the generating units to meet the load demand. The Economic Dispatch problem is a large scale nonlinear constrained optimization problem. In general, heuristic optimization techniques are used to solve non-convex Economic Dispatch problem. In this paper, ideas from Reinforcement Learning are proposed to solve the non-convex Economic Dispatch problem. QLearning is a reinforcement learning techniques where each generating unit learn the optimal schedule of the generated power that minimizes the generation cost function. The eligibility traces are used to speed up the Q-Learning process. Q-Learning with eligibility traces is used to solve Economic Dispatch problems with valve point loading effect, multiple fuel options, and power transmission losses. Keywords—Economic Dispatch, Non-Convex Cost Functions, Valve Point Loading Effect, Q-Learning, Eligibility Traces.

[1]  T. Jayabarathi,et al.  Evolutionary programming‐based economic dispatch for units with multiple fuel options , 2007 .

[2]  Chao-Lung Chiang,et al.  Improved genetic algorithm for power economic dispatch of units with valve-point effects and multiple fuels , 2005 .

[3]  E. Kyriakides,et al.  A GA-API Solution for the Economic Dispatch of Generation in Power System Operation , 2012, IEEE Transactions on Power Systems.

[4]  G. L. Viviani,et al.  Hierarchical Economic Dispatch for Piecewise Quadratic Cost Functions , 1984, IEEE Transactions on Power Apparatus and Systems.

[5]  Dick Duffey,et al.  Power Generation , 1932, Transactions of the American Institute of Electrical Engineers.

[6]  A. Immanuel Selvakumar,et al.  Optimization using civilized swarm: Solution to economic dispatch with multiple minima , 2009 .

[7]  J. Nanda,et al.  ECONOMIC-EMISSION LOAD DISPHTCH THROUGH GOAL PROGRAMMING TECHNIIJUES , 1988 .

[8]  P. K. Chattopadhyay,et al.  Solving complex economic load dispatch problems using biogeography-based optimization , 2010, Expert Syst. Appl..

[9]  Nima Amjady,et al.  Economic dispatch using an efficient real-coded genetic algorithm , 2009 .

[10]  M. Pandit,et al.  Self-Organizing Hierarchical Particle Swarm Optimization for Nonconvex Economic Dispatch , 2008, IEEE Transactions on Power Systems.

[11]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.

[12]  Essam A. Al-Ammar,et al.  Reinforcement learning solution to economic dispatch using pursuit algorithm , 2011, 2011 IEEE GCC Conference and Exhibition (GCC).

[13]  Kwang Y. Lee,et al.  Economic load dispatch for piecewise quadratic cost function using Hopfield neural network , 1993 .

[15]  Chao-Lung Chiang,et al.  Improved genetic algorithm for power economic dispatch of units with valve-point effects and multiple fuels , 2005, IEEE Transactions on Power Systems.

[16]  W. Marsden.  I and J , 2012 .

[17]  S. Khamsawang,et al.  DSPSO–TSA for economic dispatch problem with nonsmooth and noncontinuous cost functions , 2010 .

[18]  Paul J. Werbos,et al.  Approximate dynamic programming for real-time control and neural modeling , 1992 .

[19]  G. Granelli,et al.  Security-constrained economic dispatch using dual quadratic programming , 2000 .

[20]  Chun Che Fung,et al.  Simulated annealing based economic dispatch algorithm , 1993 .

[21]  Lenka Lhotská,et al.  Learning in Multi-Agent Systems: Theoretical Issues , 1997, EUROCAST.

[22]  Nima Amjady,et al.  Solution of nonconvex and nonsmooth economic dispatch by a new Adaptive Real Coded Genetic Algorithm , 2010, Expert Syst. Appl..

[23]  R. Stephenson.  A and V , 1962, The British journal of ophthalmology.

[24]  S. Rao Rayapudi.  An Intelligent Water Drop Algorithm for Solving Economic Load Dispatch Problem , 2011 .

[25]  P. S. Kannan,et al.  Penalty parameter-less constraint handling scheme based evolutionary algorithm solutions to economic dispatch , 2008 .

[26]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[27]  E.A. Jasmin,et al.  A Reinforcement Learning algorithm to economic dispatch considering transmission losses , 2008, TENCON 2008 - 2008 IEEE Region 10 Conference.

[28]  Jong-Bae Park,et al.  An Improved Particle Swarm Optimization for Nonconvex Economic Dispatch Problems , 2010, IEEE Transactions on Power Systems.

[29]  Hong-Tzer Yang,et al.  Evolutionary programming based economic dispatch for units with non-smooth fuel cost functions , 1996 .

[30]  L. Coelho,et al.  Combining of chaotic differential evolution and quadratic programming for economic dispatch optimization with valve-point effect , 2006, IEEE Transactions on Power Systems.

[31]  Whei-Min Lin,et al.  An Improved Tabu Search for Economic Dispatch with Multiple Minima , 2002, IEEE Power Engineering Review.

[32]  Bijaya Ketan Panigrahi,et al.  Adaptive particle swarm optimization approach for static and dynamic economic load dispatch , 2008 .

[33]  Serhat Duman,et al.  GRAVITATIONAL SEARCH ALGORITHM FOR ECONOMIC DISPATCH WITH VALVE-POINT EFFECTS , 2010 .

[34]  P. Werbos,et al.  Beyond Regression : "New Tools for Prediction and Analysis in the Behavioral Sciences , 1974 .

[35]  Zwe-Lee Gaing,et al.  Particle swarm optimization to solving the economic dispatch considering the generator constraints , 2003 .

[36]  Serhat Duman,et al.  A Hybrid GA-PSO Approach Based on Similarity for Various Types of Economic Dispatch Problems , 2011 .

[37]  Whei-Min Lin,et al.  Bid-based dynamic economic dispatch with an efficient interior point algorithm , 2002 .

[38]  Bart De Schutter,et al.  Multi-Agent Reinforcement Learning: A Survey , 2006, 2006 9th International Conference on Control, Automation, Robotics and Vision.

[39]  June Ho Park,et al.  Adaptive Hopfield neural networks for economic load dispatch , 1998 .