Optimal Operation Control of PV-Biomass Gasifier-Diesel-Hybrid Systems Using Reinforcement Learning Techniques

The importance of efficient utilization of biomass as renewable energy in terms of global warming and resource shortages are well known and documented. Biomass gasification is a promising power technology especially for decentralized energy systems. Decisive progress has been made in the gasification technologies development during the last decade. This paper deals with the control and optimization problems for an isolated microgrid combining the renewable energy sources (solar energy and biomass gasification) with a diesel power plant. The control problem of an isolated microgrid is formulated as a Markov decision process and we studied how reinforcement learning can be employed to address this problem to minimize the total system cost. The most economic microgrid configuration was found, and it uses biomass gasification units with an internal combustion engine operating both in single-fuel mode (producer gas) and in dual-fuel mode (diesel fuel and producer gas).

[1]  Siming You,et al.  Optimal design of negative emission hybrid renewable energy systems with biochar production , 2019, Applied Energy.

[2]  Damien Ernst,et al.  Deep Reinforcement Learning Solutions for Energy Microgrids Management , 2016 .

[3]  F. F. Costa,et al.  Particle fragmentation of raw and torrefied biomass during combustion in a drop tube furnace , 2015 .

[4]  C. Lanzerstorfer.  Grate-Fired Biomass Combustion Plants Using Forest Residues as Fuel: Enrichment Factors for Components in the Fly Ash , 2016, Waste and biomass valorization.

[5]  Rangan Banerjee,et al.  Sizing of biomass based distributed hybrid power generation systems in India , 2019, Renewable Energy.

[6]  Vasiliki Vita,et al.  Techno-Economic Assessment of Hybrid Energy Off-Grid System - A Case Study for Masirah Island in Oman , 2017 .

[7]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[8]  R. P. Saini,et al.  A review on planning, configurations, modeling and optimization techniques of hybrid renewable energy systems for off grid applications , 2016 .

[9]  L. V. Zysin,et al.  A study of the joint operation of a diesel engine and a gas generator processing plant biomass , 2002 .

[10]  L. Mazzei,et al.  Reforming of tars and organic sulphur compounds in a plasma-assisted process for waste gasification , 2015 .

[11]  Wojciech Zaremba,et al.  OpenAI Gym , 2016, ArXiv.

[12]  Marian Wysocki,et al.  Experimental and economic study of small-scale CHP installation equipped with downdraft gasifier and internal combustion engine , 2017 .

[13]  Zhehan Yi,et al.  Reinforcement-Learning-Based Optimal Control of Hybrid Energy Storage Systems in Hybrid AC–DC Microgrids , 2019, IEEE Transactions on Industrial Informatics.

[14]  Kailasnath B. Sutar,et al.  Design, development and testing of small downdraft gasifiers for domestic cookstoves , 2017 .

[15]  P. Arun.  Optimum Design of Biomass Gasifier Integrated Hybrid Energy Systems , 2015 .

[16]  Simon M. Lucas,et al.  A Survey of Monte Carlo Tree Search Methods , 2012, IEEE Transactions on Computational Intelligence and AI in Games.

[17]  Aashish Kumar Bohre,et al.  Socio-techno-economic design of hybrid renewable energy system using optimization techniques , 2018 .

[18]  Antonio Liotta,et al.  On-Line Building Energy Optimization Using Deep Reinforcement Learning , 2017, IEEE Transactions on Smart Grid.

[19]  Luu An,et al.  Dynamic Programming for Optimal Energy Management of Hybrid Wind–PV–Diesel–Battery , 2018 .

[20]  Yong He,et al.  Optimal control in microgrid using multi-agent reinforcement learning. , 2012, ISA transactions.

[21]  Gürkan Sin,et al.  Design and preliminary operation of a hybrid syngas/solar PV/battery power system for off-grid applications: A case study in Thailand , 2018 .

[22]  Simo Hostikka,et al.  A thermomechanical explanation for the topology of crack patterns observed on the surface of charred wood and particle fibreboard , 2016, 1604.01249.

[23]  Alex Graves,et al.  Playing Atari with Deep Reinforcement Learning , 2013, ArXiv.

[24]  Carlos Sánchez,et al.  Hybrid biomass-wind power plant for reliable energy generation , 2010 .

[25]  Harwin Saptoadi,et al.  Small-scale downdraft gasifiers for biomass gasification: A review , 2017 .

[26]  R. P. Saini,et al.  A review on Integrated Renewable Energy System based power generation for stand-alone applications: Configurations, storage options, sizing methodologies and control , 2014 .

[27]  O. Sippula,et al.  Ash behaviour and emission formation in a small-scale reciprocating-grate combustion reactor operated with wood chips, reed canary grass and barley straw , 2015 .

[28]  Electo Eduardo Silva Lora,et al.  A new technology for the combined production of charcoal and electricity through cogeneration. , 2014 .

[29]  Onursal Yakaboylu,et al.  Testing the constrained equilibrium method for the modeling of supercritical water gasification of biomass , 2015 .

[30]  Seiichi Shin,et al.  Smart Grid Optimization by Deep Reinforcement Learning over Discrete and Continuous Action Space , 2018, 2018 IEEE 7th World Conference on Photovoltaic Energy Conversion (WCPEC) (A Joint Conference of 45th IEEE PVSC, 28th PVSEC & 34th EU PVSEC).

[31]  Fang Liu,et al.  A Dynamic Analysis of Energy Storage With Renewable and Diesel Generation Using Volterra Equations , 2019, IEEE Transactions on Industrial Informatics.

[32]  Geert Deconinck,et al.  Battery Energy Management in a Microgrid Using Batch Reinforcement Learning , 2017 .

[33]  Mikko Hupa,et al.  Biomass combustion technology development – It is all about chemical details , 2017 .

[34]  Mohamed Cherkaoui,et al.  Study of the different structures of hybrid systems in renewable energies: A review , 2019, Energy Procedia.

[35]  S. Khalilarya,et al.  Exergoeconomic and environmental analysis of a novel configuration of solar-biomass hybrid power generation system , 2018, Energy.

[36]  Eliseu Monteiro,et al.  Co-gasification and recent developments on waste-to-energy conversion: A review , 2018 .

[37]  Sung-Hoon Ahn,et al.  Optimization of hybrid renewable energy power systems: A review , 2015, International Journal of Precision Engineering and Manufacturing-Green Technology.

[38]  Marco J. Castaldi,et al.  Progress and Prospects in the Field of Biomass and Waste to Energy and Added-Value Materials , 2017 .

[39]  A. Kozlov,et al.  A kinetic analysis of the thermochemical conversion of solid fuels (A review) , 2017, Solid Fuel Chemistry.

[40]  Steffen Heidenreich,et al.  New concepts in biomass gasification , 2015 .

[41]  Rangan Banerjee,et al.  Comparison of options for distributed generation in India , 2006 .

[42]  S. Dasappa,et al.  Rural electrification: Optimising the choice between decentralised renewable energy sources and grid extension , 2012 .

[43]  Huaguang Zhang,et al.  Real-Time Energy Management of a Microgrid Using Deep Reinforcement Learning , 2019, Energies.

[44]  Cristian Dinca,et al.  Multi-fuel multi-product operation of IGCC power plants with carbon capture and storage (CCS) , 2015 .

[45]  D. Sidorov,et al.  Toward Zero-Emission Hybrid AC/DC Power Systems with Renewable Energy Sources and Storages: A Case Study from Lake Baikal Region , 2020, Energies.

[46]  Pedro J. Mago,et al.  Analysis and optimization of CCHP systems based on energy, economical, and environmental considerations , 2009 .

[47]  Mohit Sharma,et al.  Performance and emission analysis of a dual fuel variable compression ratio (VCR) CI engine utilizing producer gas derived from walnut shells , 2020 .

[48]  Ting Wang.  Integrated gasification combined cycle (IGCC) technologies , 2016 .

[49]  C. E. Borges,et al.  A critical analysis on hybrid renewable energy modeling tools: An emerging opportunity to include social indicators to optimise systems in small communities , 2020 .

[50]  Dino Musmarra,et al.  Biomass gasification technology： The state of the art overview , 2016 .

[51]  O. Živný,et al.  Numerical Modelling of Wood Gasification in Thermal Plasma Reactor , 2017, Plasma Chemistry and Plasma Processing.

[52]  D. N. Sidorov,et al.  The hybrid model based on Hilbert-Huang Transform and neural networks for forecasting of short-term operation conditions of power system , 2011, 2011 IEEE Trondheim PowerTech.

[53]  Hongqiang Li,et al.  Exergetic and exergoeconomic evaluation of co-firing biomass gas with natural gas in CCHP system integrated with ground source heat pump , 2019, Energy Conversion and Management.

[54]  A. Agarwal,et al.  Coal and Biomass Gasification , 2018 .

[55]  Electo Eduardo Silva Lora,et al.  Syngas production in downdraft biomass gasifiers and its application using internal combustion engines , 2012 .

[56]  Marc A. Rosen,et al.  Recent advances in the development of biomass gasification technology: A comprehensive review , 2017 .

[58]  Yishay Mansour,et al.  Policy Gradient Methods for Reinforcement Learning with Function Approximation , 1999, NIPS.

[59]  Rumi Rajbongshi,et al.  Optimization of PV-biomass-diesel and grid base hybrid energy systems for rural electrification by using HOMER , 2017 .

[60]  Wenhao Zhuo,et al.  Microgrid Energy Management Strategy with Battery Energy Storage System and Approximate Dynamic Programming , 2018, 2018 37th Chinese Control Conference (CCC).

[61]  Anand Singh,et al.  Techno-economic assessment of a solar PV, fuel cell, and biomass gasifier hybrid energy system , 2016 .

[62]  Ibrahim Dincer,et al.  Assessment and Evolutionary Based Multi-Objective Optimization of a Novel Renewable-Based Polygeneration Energy System , 2017 .

[63]  Masoud Aliakbar Golkar,et al.  Optimal design of stand-alone microgrid resources based on proposed Monte-Carlo simulation , 2015, 2015 IEEE Innovative Smart Grid Technologies - Asia (ISGT ASIA).

[64]  A study on raw, torrefied, and steam-exploded wood: Fine grinding, drop-tube reactor combustion tests in N2/O2 and CO2/O2 atmospheres, particle geometry analysis, and numerical kinetics modeling , 2016 .

[65]  Roberto García,et al.  Biomass sources for thermal conversion. Techno-economical overview , 2017 .

[66]  J. Riba,et al.  Optimal Sizing of a Hybrid Grid-Connected Photovoltaic–Wind–Biomass Power System , 2015 .

[67]  Ran Chen,et al.  Stochastic Dispatch of Energy Storage in Microgrids: A Reinforcement Learning Approach Incorporated with MCTS , 2019, ArXiv.

[68]  Alec Radford,et al.  Proximal Policy Optimization Algorithms , 2017, ArXiv.

[69]  E. S. Lora,et al.  Thermodynamic assessment of the integrated gasification-power plant operating in the sawmill industry: An energy and exergy analysis , 2020 .

[70]  A. Thallam Thattai,et al.  Experimental model validation and thermodynamic assessment on high percentage (up to 70%) biomass co-gasification at the 253MWe integrated gasification combined cycle power plant in Buggenum, The Netherlands , 2016 .

[71]  S. R. Kalbande,et al.  EVALUATION OF GASIFIER BASED POWER GENERATION SYSTEM USING DIFFERENT WOODY BIOMASS , 2010 .

[72]  C. Mandl,et al.  Strategies and technologies towards zero emission biomass combustion by primary measures , 2017 .

[73]  Adam Smoliński,et al.  Experimental study on application of high temperature reactor excess heat in the process of coal and biomass co-gasification to hydrogen-rich gas , 2015 .

[74]  Massimiliano Renzi,et al.  Influence of the Syngas Feed on the Combustion Process and Performance of a Micro Gas Turbine with Steam Injection , 2017 .

[75]  Suresh Munuswamy,et al.  Comparing the cost of electricity sourced from a fuel cell-based renewable energy system and the national grid to electrify a rural health centre in India: A case study , 2011 .

[76]  Hongguang Jin,et al.  A review on the utilization of hybrid renewable energy , 2018, Renewable and Sustainable Energy Reviews.