An uncertainty-aware deep reinforcement learning framework for residential air conditioning energy management

Abstract Most existing methods for controlling the energy consumption of air conditioning (AC), focus on either scheduling the switching (on/off) of compressors or optimizing the overall energy consumption of AC system of an entire building. Unlike commercial buildings, residential apartments typically house separate ACs in individual rooms occupied by people with different thermal comfort preferences. Fortunately, the advancement of Internet-of-Things (IoT) technology has enabled the exploitation of sensory data to intelligently control the set-point temperature of ACs in individual rooms based on environmental conditions and occupant’s preferences, improving the energy efficiency of residential buildings. Indeed, control decisions based on sensory data may suffer from uncertainties due to error in data measurement and contribute to model uncertainty. This work proposes a data-driven uncertainty-aware approach to control split-type inverter ACs of residential buildings. First, information from similar AC and residential units are aggregated to reduce data imbalances, and Bayesian-Convolutional-Neural-Networks (BCNNs) are utilized to model the performance and uncertainty of the ACs from the aggregated data. Second, a Q-learning based reinforcement learning algorithm for set-point decision making is designed for setpoint optimization with transitions sampled from the BCNN models. Third, a case study is simulated based on such a framework to show that the control actions taken by the uncertainty-aware agent perform better in terms of discomfort management and energy savings compared to the uncertainty unaware agent. Further, the agent could also be adjusted to capture the trade-off between energy savings and comfort levels for varying degrees of energy and discomfort savings.