A stochastic game approach for modeling wholesale energy bidding in deregulated power markets

It has been noted in a recent report (Dec. 2002) from the General Accounting Office of the U.S. that "various design flaws in wholesale markets and transmission services have created operational problems within and between wholesale markets." This paper presents a novel methodology to analyze and design the wholesale energy bidding aspect of a deregulated power market. We adopt a system-wide approach that considers many of the relevant features including transmission congestion. We develop a two-stage model: 1) a nonzero sum stochastic game with average reward for the wholesale energy market operation, and 2) a nonlinear programming (NLP) model for the unit-commitment (UC) and the optimal power-flow aspects. The solution approach for the two-stage model is based on a reinforcement-learning (RL) algorithm, which is designed to obtain Nash equilibrium policies. We use a three-retailer/three-supplier power network with and without congestion to test and benchmark our methodology.

[1]  Allen J. Wood,et al.  Power Generation, Operation, and Control , 1984 .

[2]  Benjamin F. Hobbs,et al.  An assessment of the deregulation of electric power generation using network models of imperfect spatial markets , 1985 .

[3]  P. Klemperer,et al.  Supply Function Equilibria in Oligopoly under Uncertainty , 1989 .

[4]  John Moody,et al.  Learning rate schedules for faster stochastic gradient search , 1992, Neural Networks for Signal Processing II Proceedings of the 1992 IEEE Workshop.

[5]  W. Hogan.  Contract networks for electric power transmission , 1992 .

[6]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[7]  A. Rudkevich,et al.  Modeling Electricity Pricing in a Deregulated Generation Industry: The Potential for Oligopoly Pricing in a Poolco , 1998 .

[8]  William W. Hogan.  COMPETITIVE ELECTRICITY MARKET DESIGN : A WHOLESALE , 1998 .

[9]  James Bushnell,et al.  Market Power in Electricity Markets: Beyond Concentration Measures , 1999 .

[10]  S. Hao.  A study of basic bidding strategy in clearing pricing auctions , 1999, Proceedings of the 21st International Conference on Power Industry Computer Applications. Connecting Utilities. PICA 99. To the Millennium and Beyond (Cat. No.99CH36351).

[11]  Frank A. Wolak,et al.  Diagnosing Market Power in California's Deregulated Wholesale Electricity Market , 2000 .

[12]  J. Pang,et al.  Strategic gaming analysis for electric power systems: an MPEC approach , 2000 .

[13]  Robert J. Michaels,et al.  Games or Opportunities: Bidding in the California Markets , 2001 .

[14]  Irene Mavrommati,et al.  Design principles , 2001 .

[15]  M. Joung,et al.  A Continuous Strategy Game for Power Transactions Analysis in Competitive Electricity Markets , 2001, IEEE Power Engineering Review.

[16]  Yves Smeers,et al.  Alternative Models of Restructured Electricity Systems, Part 1: No Market Power , 2001, Oper. Res..

[17]  James Bushnell,et al.  Trading Inefficiencies in California&Apos;S Electricity Markets , 2001 .

[18]  Benjamin F. Hobbs,et al.  Equilibrium market power modeling for large scale power systems , 2001, 2001 Power Engineering Society Summer Meeting. Conference Proceedings (Cat. No.01CH37262).

[19]  Per Brinch Hansen,et al.  Design principles , 2002 .

[20]  J. Sweeney.  The California Electricity Crisis , 2002 .

[21]  An Analysis of Generation Market Power in the Midwest Interconnect , 2002 .

[22]  J. Pang,et al.  Oligopolistic Competition in Power Networks: A Conjectured Supply Function Approach , 2002, IEEE Power Engineering Review.

[23]  William W. Hogan,et al.  Electricity Market Restructuring: Reforms of Reforms , 2002 .

[24]  S. A. Soman,et al.  Application of Actor-Critic Learning Algorithm for Optimal Bidding Problem of a Genco , 2002, IEEE Power Engineering Review.

[25]  Tapas K. Das,et al.  A simulation-based approach to study stochastic inventory-planning games , 2003, Int. J. Syst. Sci..

[26]  Kiran Kumar Ravulapati,et al.  A reinforcement learning approach to stochastic business games , 2004 .

[27]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.