Reinforcement learning with fuzzified reward approach for MPPT control of PV systems

Abstract The array of photovoltaic cells converts solar power into electricity and generate a clean and renewable source of energy. For them to be efficient, they must constantly generate the maximum possible power under various environmental conditions. This is the well-known problem of tracking Maximum Power Point (MPP). The classical methods fall short as they are not able to cope with the changing environmental conditions. This work aims to track MPPT by employing Deep Reinforcement Learning (RL) with a fuzzy reward mechanism introduced for better translation of continuous space into various levels of abstraction. The author simulates a model of a PV array connected to a variable load. The introduction of RL has provided an advantage as employing a Deep-Q learning algorithm makes the structure a model-free and thereby eliminates the various modeling parameters with their effect on the design. The designed setup renders 2.8 W of power under the maximum load conditions and has been able to track the maximum power under the different varying conditions of temperature and irradiations.

[1]  Yize Sun,et al.  An Enhanced MPPT Method Combining Fractional-Order and Fuzzy Logic Control , 2017, IEEE Journal of Photovoltaics.

[2]  Oscar Castillo,et al.  A comprehensive review on type 2 fuzzy logic applications: Past, present and future , 2020, Eng. Appl. Artif. Intell..

[3]  Unal Yilmaz,et al.  PV system fuzzy logic MPPT method and PI control as a charge controller , 2018 .

[4]  Yon-Ping Chen,et al.  Maximum Power Point Tracking of Photovoltaic System Based on Reinforcement Learning , 2019, Sensors.

[5]  Frank L. Lewis,et al.  Corrigendum to 'Reinforcement Learning and Optimal Adaptive Control: An Overview and Implementation Examples' [Annual Reviews in Control 36 (1) (2012) 42-59] , 2012, Annu. Rev. Control..

[6]  Jian-Liung Chen,et al.  Maximum photovoltaic power tracking for the PV array using the fractional-order incremental conductance method , 2011 .

[7]  B N Alajmi,et al.  Fuzzy-Logic-Control Approach of a Modified Hill-Climbing Method for Maximum Power Point in Microgrid Standalone Photovoltaic System , 2011, IEEE Transactions on Power Electronics.

[8]  Jubaer Ahmed,et al.  An Effective Hybrid Maximum Power Point Tracker of Photovoltaic Arrays for Complex Partial Shading Conditions , 2019, IEEE Transactions on Industrial Electronics.

[9]  N. Panwar,et al.  Role of renewable energy sources in environmental protection: A review , 2011 .

[10]  Her-Terng Yau,et al.  Development of a Fractional Order Chaos Synchronization Dynamic Error Detector for Maximum Power Point Tracking of Photovoltaic Power Systems , 2015 .

[11]  Hikmet Esen,et al.  Modelling and experimental performance analysis of solar-assisted ground source heat pump system , 2017, J. Exp. Theor. Artif. Intell..

[12]  George A. Vouros,et al.  Energy management in solar microgrid via reinforcement learning using fuzzy reward , 2018 .

[13]  D. Devaraj,et al.  Development and analysis of adaptive fuzzy controllers for photovoltaic system under varying atmospheric and partial shading condition , 2013, Appl. Soft Comput..

[14]  Jianan Wang,et al.  Simulation and hardware implementation of a hill-climbing modified fuzzy-logic for maximum power point tracking with direct control method using boost converter , 2015 .

[15]  Santi Agatino Rizzo,et al.  ANN based MPPT method for rapidly variable shading conditions , 2015 .

[16]  Bao Chau Phan,et al.  A Deep Reinforcement Learning-Based MPPT Control for PV Systems under Partial Shading Condition , 2020, Sensors.

[17]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[18]  Zhe Zhang,et al.  Reinforcement-Learning-Based Intelligent Maximum Power Point Tracking Control for Wind Energy Conversion Systems , 2015, IEEE Transactions on Industrial Electronics.

[19]  Karim Moharm,et al.  State of the art in big data applications in microgrid: A review , 2019, Adv. Eng. Informatics.

[20]  Massimo Vitelli,et al.  Maximum Power Point Tracking , 2017 .

[21]  Mustafa Inalli,et al.  Predicting performance of a ground-source heat pump system using fuzzy weighted pre-processing-based ANFIS , 2008 .

[22]  Fevrier Valdez,et al.  A review of optimization swarm intelligence-inspired algorithms with type-2 fuzzy logic parameter adaptation , 2020, Soft Comput..

[23]  Ali M. Eltamaly,et al.  Performance of smart maximum power point tracker under partial shading conditions of photovoltaic systems , 2015 .

[24]  Boutaib Dahhou,et al.  Adaptive fuzzy controller based MPPT for photovoltaic systems , 2014 .

[25]  Antonello Rizzi,et al.  Optimization strategies for Microgrid energy management systems by Genetic Algorithms , 2020, Appl. Soft Comput..

[26]  Ibrahim Harbi,et al.  An Adaptive Model-Based MPPT Technique with Drift-Avoidance for Grid-Connected PV Systems , 2020, Energies.

[27]  Frank L. Lewis,et al.  Reinforcement learning and optimal adaptive control: An overview and implementation examples , 2012, Annu. Rev. Control..

[28]  Z. Salam,et al.  An improved perturb and observe (P&O) maximum power point tracking (MPPT) algorithm for higher efficiency , 2015 .

[29]  Abdulkadir Sengür,et al.  Modelling of a new solar air heater through least-squares support vector machines , 2009, Expert Syst. Appl..

[30]  Wen-Yen Chen,et al.  A Reinforcement Learning-Based Maximum Power Point Tracking Method for Photovoltaic Array , 2015 .

[31]  Vijay Kumar,et al.  Performance analysis of optimal hybrid novel interval type-2 fractional order fuzzy logic controllers for fractional order systems , 2018, Expert Syst. Appl..

[32]  Abdulkadir Sengür,et al.  Artificial neural network and wavelet neural network approaches for modelling of a solar air heater , 2009, Expert Syst. Appl..

[33]  Bart De Schutter,et al.  Consistency of fuzzy model-based reinforcement learning , 2008, 2008 IEEE International Conference on Fuzzy Systems (IEEE World Congress on Computational Intelligence).

[34]  Rachid Outbib,et al.  General review and classification of different MPPT Techniques , 2017 .

[35]  Ignacio Carlucho,et al.  Deep reinforcement learning approach for MPPT control of partially shaded PV systems in Smart Grids , 2020, Appl. Soft Comput..

[36]  Maysam F. Abbod,et al.  A novel maximum power point tracking technique based on fuzzy logic for photovoltaic systems , 2018, International Journal of Hydrogen Energy.