Deep reinforcement learning optimization framework for a power generation plant considering performance and environmental issues

Abstract In the electric power generation sector, striking a balance between maximum power production and acceptable emission limits is a challenging task that requires sophisticated techniques. With traditional methods, this is an extremely complex issue due to the large number of process variables that are involved. In this paper, a deep reinforcement learning optimization framework (DRLOF) is proposed to determine the optimal operating conditions for a commercial circulating fluidized bed (CFB) power plant that strikes a good balance between performance and environmental issues. The DRLOF included the CFB as an environment created from over 1.5 years of plant data with a 1 min sampling time which interacted with an advantage actor-critic (A2C) agent of two architectures named ‘separate-A2CN’ and ‘shared-A2CN’. The framework was optimized by maximizing electric power generation within the constraints of the plant’s capacity and environmental emission standards, taking into consideration the cost of operations. After training, the framework of the separate-A2CN architecture achieved a 1.97% increase in electricity generation and 1.59% emission reduction for NOx at 14.3 times lower computational cost. Furthermore, we demonstrated the framework’s flexibility, adaptability and lower computational burden by carrying out different test scenarios which demonstrated the effectiveness of the DRLOF. The findings of this study are not limited to the CFB power plant but can be extended to other chemical processes and industries. This approach minimizes the need for costly experiments, online optimization challenges and associated customizations.

[1]  M. Brito,et al.  Replacing coal-fired power plants by photovoltaics in the Portuguese electricity system , 2019, Journal of Cleaner Production.

[2]  Hamed Hajebzadeh,et al.  Mathematical modeling and validation of a 320 MW tangentially fired boiler: A case study , 2019 .

[3]  Mingming Gao,et al.  Modeling for the bed temperature 2D-interval prediction of CFB boilers based on long-short term memory network , 2020 .

[4]  Vijay R. Konda,et al.  OnActor-Critic Algorithms , 2003, SIAM J. Control. Optim..

[5]  Afsin Gungor,et al.  Prediction of SO2 and NOx emissions for low-grade Turkish lignites in CFB combustors , 2009 .

[6]  Sedat Keleş,et al.  Energy utilization, environmental pollution and renewable energy sources in Turkey , 2004 .

[7]  Gang Chen,et al.  Dynamic modeling of NOX emission in a 660 MW coal-fired boiler with long short-term memory , 2019, Energy.

[8]  Ravi K Srivastava,et al.  Nitrogen Oxides Emission Control Options for Coal-Fired Electric Utility Boilers , 2005, Journal of the Air & Waste Management Association.

[9]  Weidong Zhang,et al.  ThermalNet: A deep reinforcement learning-based combustion optimization system for coal-fired boiler , 2018, Eng. Appl. Artif. Intell..

[10]  Miguel Angel Gonzalez-Salazar,et al.  Review of the operational flexibility and emissions of gas- and coal-fired power plants in a future with growing renewables , 2018 .

[11]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[12]  Wenqi Zhong,et al.  Combustion optimization of ultra supercritical boiler based on artificial intelligence , 2019, Energy.

[13]  Paweł Madejski,et al.  Numerical study of a large-scale pulverized coal-fired boiler operation using CFD modeling based on the probability density function method , 2018, Applied Thermal Engineering.

[14]  Xia Li,et al.  Combustion optimization of a boiler based on the chaos and Lévy flight vortex search algorithm , 2018, Applied Mathematical Modelling.

[15]  Johannes Urpelainen,et al.  Environmental Justice in India: Incidence of Air Pollution from Coal-Fired Power Plants , 2020 .

[16]  Anna Hnydiuk-Stefan,et al.  Analysis of supercritical coal fired oxy combustion power plant with cryogenic oxygen unit and turbo-compressor , 2017 .

[17]  Kang Li,et al.  A HYBRID GA FOR NOX EMISSION MODELLING IN POWER GENERATION PLANTS , 2002 .

[18]  Wenqi Zhong,et al.  Study on oxy-fuel combustion behaviors in a S-CO2 CFB by 3D CFD simulation , 2020 .

[19]  Jingge Song,et al.  A globally enhanced general regression neural network for on-line multiple emissions prediction of utility boiler , 2017, Knowl. Based Syst..

[20]  Qinhui Wang,et al.  Predictions of NOx/N2O emissions from an ultra-supercritical CFB boiler using a 2-D comprehensive CFD combustion model , 2020 .

[21]  Kang Li,et al.  GA BASED NEURAL NETWORK MODELING OF NOX EMISSION IN A COAL-FIRED POWER GENERATION PLANT , 2002 .

[22]  Zhenhao Tang,et al.  The multi-objective optimization of combustion system operations based on deep data-driven models , 2019, Energy.

[23]  Jiahai Yuan,et al.  A realistic pathway for coal-fired power in China from 2020 to 2030 , 2020 .

[24]  Yasin Tunckaya,et al.  Comparative prediction analysis of 600 MWe coal-fired power plant production rate using statistical and neural-based models , 2015 .

[25]  Kaddour Najim,et al.  Modelling of NO x Emissions Based on a Fuzzy Logic Neural Network , 1996 .

[26]  Wojciech Nowak,et al.  A generalized model of SO2 emissions from large- and small-scale CFB boilers by artificial neural network approach , 2015 .

[27]  Kody M. Powell,et al.  Dynamic modeling and optimization of a coal-fired utility boiler to forecast and minimize NOx and CO emissions simultaneously , 2019, Comput. Chem. Eng..

[28]  Kody M. Powell,et al.  Sustainable NOx emission reduction at a coal-fired power station through the use of online neural network modeling and particle swarm optimization , 2019 .

[29]  Feng Hong,et al.  A dynamic model for the bed temperature prediction of circulating fluidized bed boilers based on least squares support vector machine with real operational data , 2017 .

[30]  Zixiang Li,et al.  Primary air ratio affects coal utilization mode and NOx emission in lignite pulverized boiler , 2019, Energy.

[31]  Yongjie Zhai,et al.  Adaptive LSSVM based iterative prediction method for NOx concentration prediction in coal-fired power plant considering system delay , 2020, Appl. Soft Comput..

[32]  Anders Lyngfelt,et al.  Methods for reducing the emission of nitrous oxide from fluidized bed combustion , 1996 .

[33]  Yang Liu,et al.  Optimizing combustion of coal fired boilers for reducing NOx emission using Gaussian Process , 2018, Energy.

[34]  Zhengtao Ding,et al.  Operation and Bidding Strategies of Power Plants with Carbon Capture , 2017 .

[35]  Qiang Zhang,et al.  Air pollutant emissions from coal-fired power plants in China over the past two decades. , 2020, The Science of the total environment.

[36]  Hao Zhou,et al.  Multi‐objective optimization of the coal combustion performance with artificial neural networks and genetic algorithms , 2005 .

[37]  E. Michaelides,et al.  Substitution of coal power plants with renewable energy sources – Shift of the power demand and energy storage , 2018 .

[38]  Chen Yang,et al.  A hybrid simulation of a 600 MW supercritical circulating fluidized bed boiler system , 2018, Applied Thermal Engineering.

[39]  Qingwei Li,et al.  Improved coal combustion optimization model based on load balance and coal qualities , 2017 .

[40]  Xinli Li,et al.  Prediction of the NO emissions from thermal power plant using long-short term memory neural network , 2020 .

[41]  Derrick Adams,et al.  Prediction of SOx–NOx emission from a coal-fired CFB power plant with machine learning: Plant data learned by deep neural network and least square support vector machine , 2020 .

[42]  Xinxin Zhang,et al.  Multi-objective prediction of coal-fired boiler with a deep hybrid neural networks , 2020 .

[43]  Hao Zhou,et al.  A comparative study of optimization algorithms for low NOx combustion modification at a coal-fired utility boiler , 2009, Expert Syst. Appl..

[44]  Gang Chen,et al.  Modeling and reduction of NOX emissions for a 700 MW coal-fired boiler with the advanced machine learning method , 2016 .

[45]  Johannes Urpelainen,et al.  The future of India's coal-fired power generation capacity , 2019, Journal of Cleaner Production.