Electricity Demand Response under Real-Time Pricing: A Multi-armed Bandit Game

Real-time electricity pricing (RTP) for consumers has long been argued to be key to realize the many envisioned benefits of a smart energy grid. However, there has not been a consensus on how to best implement RTP in an organized, competitive wholesale market with active demand participation. Since most of such markets implement a two-settlement system, with day-ahead electricity price forecasts guiding physical transactions in the next day and real-time ex post prices settling any realtime imbalances, it is a natural idea to let consumers respond to the day-ahead prices. We show in this paper through simulation that naive responsive behaviors to the day-ahead price signals can lead to high price volatility, which will increase not only the risk of system instability, but also the financial risks faced by the consumers. To overcome this issue, we propose a game-theoretic framework in which each consumer solves a multi-armed bandit problem; that is, each consumer learns from the history of the game and attempts to minimize the cumulative regrets. We show through simulation that such a framework leads to drastically reduced volatility on real-time prices and much flatter load curves for the entire grid.