A review of reinforcement learning methodologies on control systems for building energy

The usage of energy directly leads to a great amount of consumption of the non-renewable fossil resources. Exploiting fossil resources energy can influence both climate and health via ineluctable e ...

[1]  Sean R Eddy,et al.  What is dynamic programming? , 2004, Nature Biotechnology.

[2]  Wenjian Cai,et al.  PID autotuner and its application in HVAC systems , 2001, Proceedings of the 2001 American Control Conference. (Cat. No.01CH37148).

[3]  Madeleine Gibescu,et al.  Enabling Cooperative Behavior for Building Demand Response Based on Extended Joint Action Learning , 2018, IEEE Transactions on Industrial Informatics.

[4]  Zhen Yu,et al.  Online tuning of a supervisory fuzzy controller for low-energy building system using reinforcement learning , 2010 .

[5]  Athanasios V. Vasilakos,et al.  A Survey of Self-Organization Mechanisms in Multiagent Systems , 2017, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[6]  Bart De Schutter,et al.  Multi-agent Reinforcement Learning: An Overview , 2010 .

[7]  R. J. Williams,et al.  Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning , 2004, Machine Learning.

[8]  Antonio Messineo,et al.  Coupling a neural network temperature predictor and a fuzzy logic controller to perform thermal comfort regulation in an office building , 2014 .

[9]  Nursyarizal Mohd Nor,et al.  A review on optimized control systems for building energy and comfort management of smart sustainable buildings , 2014 .

[10]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[11]  Martin A. Riedmiller,et al.  Reinforcement learning for robot soccer , 2009, Auton. Robots.

[12]  Douglas C. Hittle,et al.  Synthesis of reinforcement learning, neural networks and PI control applied to a simulated heating coil , 1997, Artificial Intelligence in Engineering.

[13]  Weimin Wang,et al.  Applying multi-objective genetic algorithms in green building design optimization , 2005 .

[14]  M. Zaheer-uddin,et al.  Dynamic simulation of energy management control functions for HVAC systems in buildings , 2006 .

[15]  Kaamran Raahemifar,et al.  Artificial neural network (ANN) based model predictive control (MPC) and optimization of HVAC systems: A state of the art review and case study of a residential HVAC system , 2017 .

[16]  R. Belmans,et al.  Reinforcement Learning Applied to an Electric Water Heater: From Theory to Practice , 2015, IEEE Transactions on Smart Grid.

[17]  Lingfeng Wang,et al.  Multi-zone building energy management using intelligent control and optimization , 2013 .

[18]  R. Bellman.  A Markovian Decision Process , 1957 .

[19]  Gregor P. Henze,et al.  Adaptive Optimal Control of a Grid-Independent Photovoltaic System , 2002 .

[20]  Robert Babuska,et al.  A Survey of Actor-Critic Reinforcement Learning: Standard and Natural Policy Gradients , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[21]  Mahesan Niranjan,et al.  On-line Q-learning using connectionist systems , 1994 .

[22]  Douglas C. Hittle,et al.  MIMO Robust Control for HVAC Systems , 2008, IEEE Transactions on Control Systems Technology.

[23]  James E. Braun,et al.  Reducing energy costs and peak electrical demand through optimal control of building thermal storage , 1990 .

[24]  Mischa Schmidt,et al.  Optimizing legacy building operation: The evolution into data-driven predictive cyber-physical systems , 2017 .

[25]  Lingfeng Wang,et al.  Occupancy pattern based intelligent control for improving energy efficiency in buildings , 2012, 2012 IEEE International Conference on Automation Science and Engineering (CASE).

[26]  Kirk H. Drees,et al.  Development and Evaluation of a Rule-Based Control Strategy for Ice Storage Systems , 1996 .

[27]  Bart De Schutter,et al.  A Comprehensive Survey of Multiagent Reinforcement Learning , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[28]  G. Henze,et al.  Control of Air-Cooled Chiller Condenser Fans using Clustering Neural Networks , 2002 .

[29]  Lingfeng Wang,et al.  Multi-agent control system with intelligent optimization for smart and energy-efficient buildings , 2010, IECON 2010 - 36th Annual Conference on IEEE Industrial Electronics Society.

[30]  M. Zaheer-Uddin,et al.  Optimal, sub-optimal and adaptive control methods for the design of temperature controllers for intelligent buildings , 1993 .

[31]  Amit Patra,et al.  Feedback linearization based control of a variable air volume air conditioning system for cooling applications. , 2008, ISA transactions.

[32]  Luisa F. Cabeza,et al.  Control strategies comparison of a ventilated facade with PCM – energy savings, cost reduction and CO2 mitigation , 2016 .

[33]  Talal Rahwan,et al.  Automatic HVAC Control with Real-time Occupancy Recognition and Simulation-guided Model Predictive Control in Low-cost Embedded System , 2017, ArXiv.

[34]  Brent A. Nelson,et al.  Past visions, current trends, and future context: A review of building energy, carbon, and sustainability , 2018 .

[35]  Manuela M. Veloso,et al.  Multiagent learning using a variable learning rate , 2002, Artif. Intell..

[36]  D. Subbaram Naidu,et al.  Advanced control strategies for HVAC&R systems—An overview: Part II: Soft and fusion control , 2011 .

[37]  Michael L. Littman,et al.  Markov Games as a Framework for Multi-Agent Reinforcement Learning , 1994, ICML.

[38]  Li Xia,et al.  Satisfaction based Q-learning for integrated lighting and blind control , 2016 .

[39]  Yishay Mansour,et al.  Policy Gradient Methods for Reinforcement Learning with Function Approximation , 1999, NIPS.

[40]  Peng Zhao,et al.  An Energy Management System for Building Structures Using a Multi-Agent Decision-Making Control Methodology , 2010, 2010 IEEE Industry Applications Society Annual Meeting.

[41]  Guoqiang Zhang,et al.  Control strategies for integration of thermal energy storage into buildings: State-of-the-art review , 2015 .

[42]  Manuela M. Veloso,et al.  Multiagent Systems: A Survey from a Machine Learning Perspective , 2000, Auton. Robots.

[43]  Lingfeng Wang,et al.  Multi-agent intelligent controller design for smart and sustainable buildings , 2010, 2010 IEEE International Systems Conference.

[44]  Chris Underwood,et al.  Robust control of HVAC plant II: controller design , 2000 .

[45]  Simeng Liu,et al.  Evaluation of Reinforcement Learning for Optimal Control of Building Active and Passive Thermal Storage Inventory , 2005 .

[46]  Luisa F. Cabeza,et al.  Control of a PCM ventilated facade using reinforcement learning techniques , 2015 .

[47]  Tony Roskilly,et al.  A review of building climate and plant controls, and a survey of industry perspectives , 2018 .

[48]  Ruxu Du,et al.  Thermal comfort control based on neural network for HVAC application , 2005, Proceedings of 2005 IEEE Conference on Control Applications, 2005. CCA 2005..

[49]  Alex Graves,et al.  Playing Atari with Deep Reinforcement Learning , 2013, ArXiv.

[50]  Moncef Krarti,et al.  Development of a Predictive Optimal Controller for Thermal Energy Storage Systems , 1997 .

[51]  Simeng Liu,et al.  Experimental analysis of simulated reinforcement learning control for active and passive building thermal storage inventory: Part 1. Theoretical foundation , 2006 .

[52]  L. A. Bollinger,et al.  Multi-agent reinforcement learning for optimizing technology deployment in distributed multi-energy systems , 2016 .

[53]  Luis Pérez-Lombard,et al.  A review on buildings energy consumption information , 2008 .

[54]  Muhd Zaimi Abd Majid,et al.  A global review of energy consumption, CO2 emissions and policy in the residential sector (with an overview of the top ten CO2 emitting countries) , 2015 .

[55]  Lei Yang,et al.  Reinforcement learning for optimal control of low exergy buildings , 2015 .

[56]  Konstantinos Dalamagkidis,et al.  Reinforcement Learning for Building Environmental Control , 2008 .

[57]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[58]  Xiaosong Zhang,et al.  A new adaptive PI controller and its application in HVAC systems , 2007 .

[59]  Jennie Si,et al.  Robust Reinforcement Learning for Heating, Ventilation, and Air Conditioning Control of Buildings , 2004 .

[60]  J. Doyle,et al.  Essentials of Robust Control , 1997 .

[61]  Richard B. Vinter,et al.  Optimal Control , 2000 .

[62]  Madeleine Gibescu,et al.  Unsupervised energy prediction in a Smart Grid context using reinforcement cross-building transfer learning , 2016 .

[63]  Lingfeng Wang,et al.  Multi-objective optimization for decision-making of energy and comfort management in building automation and control , 2012 .

[64]  Anastasios I. Dounis,et al.  Advanced control systems engineering for energy and comfort management in a building environment--A review , 2009 .

[65]  Jonghoon Ahn,et al.  Analysis of energy and control efficiencies of fuzzy logic and artificial neural network technologies in the heating energy supply system responding to the changes of user demands , 2017 .

[66]  Abhijit Gosavi,et al.  Simulation-Based Optimization: Parametric Optimization Techniques and Reinforcement Learning , 2003 .

[67]  A. Isidori.  Nonlinear Control Systems , 1985 .

[68]  Bart De Schutter,et al.  Reinforcement Learning and Dynamic Programming Using Function Approximators , 2010 .

[69]  Carl E. Rasmussen,et al.  PILCO: A Model-Based and Data-Efficient Approach to Policy Search , 2011, ICML.

[70]  Jin Wen,et al.  Review of building energy modeling for control and operation , 2014 .

[71]  Demis Hassabis,et al.  Mastering the game of Go without human knowledge , 2017, Nature.

[72]  Jianhao Chen,et al.  Real-time performance assessment and adaptive control for a water chiller unit in an HVAC system , 2018 .

[73]  Mustafa Inalli,et al.  Artificial neural networks and adaptive neuro-fuzzy inference systems approaches to forecast the meteorological data for HVAC: The case of cities for Turkey , 2018, Energy.

[74]  Giuseppe Tommaso Costanzo,et al.  Experimental analysis of data-driven control for a building heating system , 2015, ArXiv.

[75]  Edwin H.W. Chan,et al.  Success factors of energy performance contracting (EPC) for sustainable building energy efficiency retrofit (BEER) of hotel buildings in China , 2011 .

[76]  Douglas C. Hittle,et al.  Robust reinforcement learning control , 2001, Proceedings of the 2001 American Control Conference. (Cat. No.01CH37148).

[77]  Pierre Geurts,et al.  Tree-Based Batch Mode Reinforcement Learning , 2005, J. Mach. Learn. Res..

[78]  Maria Pia Fanti,et al.  A simulation and control model for building energy management , 2018 .

[79]  William J. Batty,et al.  Fuzzy control strategies to provide cost and energy efficient high quality indoor environments in buildings with high occupant densities , 2003 .

[80]  H. He,et al.  Efficient Reinforcement Learning Using Recursive Least-Squares Methods , 2011, J. Artif. Intell. Res..

[81]  Abbas Javed,et al.  Design and Implementation of a Cloud Enabled Random Neural Network-Based Decentralized Smart Controller With Intelligent Sensor Nodes for HVAC , 2017, IEEE Internet of Things Journal.

[82]  Dirk Vanhoudt,et al.  Model-Free Control of Thermostatically Controlled Loads Connected to a District Heating Network , 2017, ArXiv.

[83]  D. Subbaram Naidu,et al.  Advanced control strategies for heating, ventilation, air-conditioning, and refrigeration systems—An overview: Part I: Hard control , 2011 .

[84]  Bing Dong,et al.  Non-linear optimal controller design for building HVAC systems , 2010, 2010 IEEE International Conference on Control Applications.

[85]  Raj Jain,et al.  A Quantitative Measure Of Fairness And Discrimination For Resource Allocation In Shared Computer Systems , 1998, ArXiv.

[86]  Gerardo Maria Mauro,et al.  A new comprehensive approach for cost-optimal building design integrated with the multi-objective model predictive control of HVAC systems , 2017 .

[87]  Peng Xu,et al.  Demand reduction in building energy systems based on economic model predictive control , 2012 .

[88]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[89]  Zhang Jun,et al.  A Particle Swarm Optimization Approach for Optimal Design of PID Controller for Temperature Control in HVAC , 2011, 2011 Third International Conference on Measuring Technology and Mechatronics Automation.

[90]  Milind Tambe,et al.  Coordinating occupant behavior for building energy and comfort management using multi-agent systems , 2012 .

[91]  W. Cai,et al.  China building energy consumption: Situation, challenges and corresponding measures , 2009 .

[92]  Farrokh Janabi-Sharifi,et al.  Theory and applications of HVAC control systems – A review of model predictive control (MPC) , 2014 .

[93]  Massimiliano Manfren,et al.  Building Automation and Control Systems and performance optimization: A framework for analysis , 2017 .

[94]  Gregor P. Henze,et al.  Evaluation of Reinforcement Learning Control for Thermal Energy Storage Systems , 2003 .

[95]  Kody M. Powell,et al.  Reduced-order residential home modeling for model predictive control , 2014 .

[96]  Johan Driesen,et al.  Valuing knowledge, information and agency in Multi-agent Reinforcement Learning: a case study in smart buildings , 2018, AAMAS.

[97]  Hamed Moradi,et al.  Nonlinear multivariable control and performance analysis of an air-handling unit , 2011 .

[98]  Abbas Javed,et al.  Smart Random Neural Network Controller for HVAC Using Cloud Computing Technology , 2017, IEEE Transactions on Industrial Informatics.

[99]  Craig Boutilier,et al.  The Dynamics of Reinforcement Learning in Cooperative Multiagent Systems , 1998, AAAI/IAAI.

[100]  Simeng Liu,et al.  Experimental analysis of simulated reinforcement learning control for active and passive building thermal storage inventory: Part 2: Results and analysis , 2006 .

[101]  Chris Watkins,et al.  Learning from delayed rewards , 1989 .

[102]  D. Kolokotsa,et al.  Reinforcement learning for energy conservation and comfort in buildings , 2007 .

[103]  Fernando D. Bianchi,et al.  Gain scheduling control of variable-speed wind energy conversion systems using quasi-LPV models , 2005 .

[104]  K. Sycara,et al.  This Is a Publication of the American Association for Artificial Intelligence Multiagent Systems Multiagent System Issues and Challenges Individual Agent Reasoning Task Allocation Multiagent Planning Recognizing and Resolving Conflicts Managing Communication Modeling Other Agents Managing Resources , 2022 .

[105]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[106]  Manfred Morari,et al.  Model predictive control: Theory and practice - A survey , 1989, Autom..

[107]  Yuval Tassa,et al.  Continuous control with deep reinforcement learning , 2015, ICLR.

[108]  Sergey Levine,et al.  Continuous Deep Q-Learning with Model-based Acceleration , 2016, ICML.

[109]  Leslie K. Norford,et al.  Optimal control of HVAC and window systems for natural ventilation through reinforcement learning , 2018, Energy and Buildings.

[110]  Howard M. Schwartz,et al.  Multi-Agent Machine Learning: A Reinforcement Approach , 2014 .