Agent-Based Analysis of Capacity Withholding and Tacit Collusion in Electricity Markets

This paper employs agent-based simulation to study energy market performance and, in particular, capacity withholding and the emergence of tacit collusion among the market participants. The energy market is formulated as a repeated game, where each stage game corresponds to an hourly energy auction. Each hourly energy auction is cleared using locational marginal pricing. Generators are modeled as adaptive agents capable of learning through the interaction with their environment, following a reinforcement learning algorithm. The SA-Q-learning algorithm, a modified version of the popular Q-Learning, is used. Test results on a two-node power system with two and eight competing generator-agents, demonstrate the development of tacit collusion among generators even under competitive conditions.

[1]  C. Ribeiro,et al.  Reinforcement Learning Agents , 2002, Artificial Intelligence Review.

[2]  S. Borenstein,et al.  Measuring Market Inefficiencies in California's Restructured Wholesale Electricity Market , 2002 .

[3]  W. Hamilton,et al.  The evolution of cooperation. , 1984, Science.

[4]  Natalia Fabra,et al.  Tacit Collusion in Repeated Auctions: Uniform Versus Discriminatory , 2003 .

[5]  C. Aliprantis,et al.  Games and Decision Making , 1999 .

[6]  GLOSSARY OF INDUSTRIAL ORGANISATION ECONOMICS AND COMPETITION LAW , 1999 .

[7]  C. W. Smith.  Auctions: The Social Construction of Value , 1989 .

[8]  Gaofeng Xiong,et al.  An electricity supplier bidding strategy through Q-Learning , 2002, IEEE Power Engineering Society Summer Meeting,.

[9]  Damien Ernst,et al.  Nash Equilibria and Reinforcement Learning for Active Decision Maker Modelling in Power Markets , 2004 .

[10]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[11]  Yoav Shoham,et al.  Multi-Agent Reinforcement Learning:a critical survey , 2003 .

[12]  A co-evolutionary approach to the tacit collusion of generators in oligopolistic electricity markets: piecewise linear bidding structure case , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[13]  M. Rothkopf.  Daily Repetition: A Neglected Factor in the Analysis of Electricity Auctions , 1999 .

[14]  S. Okuma,et al.  Multi-agent based experiments on uniform price and pay-as-bid electricity auction markets , 2004, 2004 IEEE International Conference on Electric Utility Deregulation, Restructuring and Power Technologies. Proceedings.

[15]  S. Borenstein,et al.  Measuring Market Inefficiencies in California's Deregulated Electricity Industry , 2002 .

[16]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[17]  Michael L. Littman,et al.  Markov Games as a Framework for Multi-Agent Reinforcement Learning , 1994, ICML.

[18]  R. Axelrod.  The Emergence of Cooperation among Egoists , 1981, American Political Science Review.

[19]  Damien Ernst,et al.  A comparison of Nash equilibria analysis and agent-based modelling for power markets , 2006 .

[20]  Yang Liu,et al.  A new Q-learning algorithm based on the metropolis criterion , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[21]  Market power analysis in electricity markets using supply function equilibrium model , 2004 .

[22]  Shmuel S. Oren,et al.  Economic Inefficiency of Passive Transmission Rights in Congested Electricity Systems with Competitive Generation , 1997 .

[23]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .

[24]  Michael P. Wellman,et al.  Nash Q-Learning for General-Sum Stochastic Games , 2003, J. Mach. Learn. Res..

[25]  Manuela Veloso,et al.  An Analysis of Stochastic Game Theory for Multiagent Reinforcement Learning , 2000 .

[26]  S. Oren,et al.  How good are supply function equilibrium models: an empirical analysis of the ERCOT balancing market , 2007 .

[27]  Derek W. Bunn,et al.  Agent-based simulation-an application to the new electricity trading arrangements of England and Wales , 2001, IEEE Trans. Evol. Comput..

[28]  Robert Gibbons,et al.  A primer in game theory , 1992 .

[29]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[30]  Simon Haykin,et al.  A dynamic channel assignment policy through Q-learning , 1999, IEEE Trans. Neural Networks.

[31]  Emmanuel Dechenaux,et al.  Tacit collusion and capacity withholding in repeated uniform price auctions , 2007 .

[32]  Peter Dayan,et al.  Technical Note: Q-Learning , 2004, Machine Learning.