Beyond theory: Experimental results of a self-learning air conditioning unit

This paper demonstrates the application of a data-driven approach, based on fitted Q-iteration, in a living lab with an air conditioning unit and a photovoltaic system. More specifically, the objective is to minimize the quadratic difference between the locally produced photovoltaic power and the power consumption of the air conditioning unit. A first simulation-based experiment assesses the performance of the data-driven approach by comparing its performance with the default thermostat and a model-based method. The simulation-based results indicate that the data-driven control method was able to achieve near-optimal policies within approximately 15 days of operation. In a second experiment, the proposed control method is applied to a living lab. The qualitative results indicate that the control method was able to successfully reduce the peak power of the photovoltaic system that is injected into the grid.

[1]  Pierre Geurts,et al.  Tree-Based Batch Mode Reinforcement Learning , 2005, J. Mach. Learn. Res..

[2]  Bart De Schutter,et al.  Residential Demand Response of Thermostatically Controlled Loads Using Batch Reinforcement Learning , 2017, IEEE Transactions on Smart Grid.

[3]  Gregor P. Henze,et al.  Evaluation of Reinforcement Learning Control for Thermal Energy Storage Systems , 2003 .

[4]  Soummya Kar,et al.  Using smart devices for system-level management and control in the smart grid: A reinforcement learning framework , 2012, 2012 IEEE Third International Conference on Smart Grid Communications (SmartGridComm).

[5]  Martin A. Riedmiller.  Neural Fitted Q Iteration - First Experiences with a Data Efficient Neural Reinforcement Learning Method , 2005, ECML.

[6]  Wei Zhang,et al.  Aggregate model for heterogeneous thermostatically controlled loads with demand response , 2012, 2012 IEEE Power and Energy Society General Meeting.

[7]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[8]  Koen Vanthournout,et al.  LINEAR breakthrough project: Large-scale implementation of smart grid technologies in distribution grids , 2012, 2012 3rd IEEE PES Innovative Smart Grid Technologies Europe (ISGT Europe).

[9]  Í. Ciglera,et al.  Beyond theory : the challenge of implementing Model Predictive Control in buildings Ji ř , 2013 .

[10]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[11]  K. Schneider,et al.  GridLAB-D: An open-source power systems modeling and simulation environment , 2008, 2008 IEEE/PES Transmission and Distribution Conference and Exposition.

[12]  W. Blyth,et al.  Energy Security and Climate Change Policy Interactions - An Assessment Framework , 2005 .

[13]  Ronnie Belmans,et al.  Demand response of a heterogeneous cluster of electric water heaters using batch reinforcement learning , 2014, 2014 Power Systems Computation Conference.

[14]  Zheng Wen,et al.  Optimal Demand Response Using Device-Based Reinforcement Learning , 2014, IEEE Transactions on Smart Grid.

[15]  Aimee Girdwood.  Moving towards a low-carbon economy : financial law , 2011 .

[16]  H. Madsen,et al.  From probabilistic forecasts to statistical scenarios of short-term wind power production , 2009 .