User comfort and energy efficiency in HVAC systems by Q-learning

This study focuses on applying Q-learning techniques for an HVAC agent where the agent learns to find the optimal sequence of ventilator rate variations to satisfy user comfort and energy efficiency simultaneously. On-Off and Setpoint control methods are investigated besides the proposed control method under different occupant number. The results show the advantage of the proposed Q-learning method to keep the Indoor Air Quality (IAQ), i.e., the indoor CO2 concentration, at the desired level while operating the HVAC efficiently.

[1]  C. Verhelst.  Model Predictive Control of Ground Coupled Heat Pump Systems in Office Buildings (Modelgebaseerde regeling van grondgekoppelde warmtepompsystemen in kantoorgebouwen) , 2012 .

[2]  Peter Dayan,et al.  Technical Note: Q-Learning , 2004, Machine Learning.

[3]  Phillipp Beiter,et al.  2016 Renewable Energy Data Book , 2017 .

[4]  Tao Jiang,et al.  Online Energy Management for a Sustainable Smart Home With an HVAC Load and Random Occupancy , 2017, IEEE Transactions on Smart Grid.

[5]  Anthony A. Maciejewski,et al.  A Partially Observable Markov Decision Process Approach to Residential Home Energy Management , 2018, IEEE Transactions on Smart Grid.

[6]  Shuhui Li,et al.  An Optimal and Learning-Based Demand Response and Home Energy Management System , 2016, IEEE Transactions on Smart Grid.

[7]  A. Persily,et al.  Carbon dioxide generation rates for building occupants , 2017, Indoor air.

[8]  Tao Lu,et al.  Estimation of Space Air Change Rates and CO2 Generation Rates for Mechanically-Ventilated Buildings , 2011 .

[9]  Philip Haves,et al.  Model predictive control for the operation of building cooling systems , 2010, Proceedings of the 2010 American Control Conference.

[10]  José R. Vázquez-Canteli,et al.  Balancing comfort and energy consumption of a heat pump using batch reinforcement learning with fitted Q-iteration , 2017 .

[11]  Ermyas Abebe,et al.  Day Ahead Scheduling to Optimize Industrial HVAC Energy Cost Based ON Peak/OFF-Peak Tariff and Weather Forecasting , 2017, IEEE Access.

[12]  Gregor P. Henze,et al.  Evaluation of Reinforcement Learning Control for Thermal Energy Storage Systems , 2003 .