Reliable Power Scheduling of an Emission-Free Ship: Multiobjective Deep Reinforcement Learning

Environmental pollutants, as a global concern, have led to a general increase in the utilization of renewable energy resources instead of fossil fuels. Accordingly, the penetration of these resources in all-electric ships, as well as power grids, has increased in recent years. In this article, in order to have a zero-emission and cost-effective energy management in an all-electric ferry boat, a new reliable and optimal power scheduling is presented that uses fuel cell and battery energy storage systems. Furthermore, the real information including load profile and paths is considered for the case study to assess the feasibility and superiority of the proposed approach. In addition to the cost of energy management, to have a reliable combination of the proposed resources, the loss of load expectation (LOLE) as a reliability index is considered in the energy management context and the problem is solved by the deep reinforcement learning in a multiobjective manner. The results of the consideration of two common standards, including DNVGL-ST-0033 and DNVGL-ST-0373, demonstrate that the proposed energy management method is applicable in industrial applications. Finally, the real-time simulation-based hardware-in-the-loop (HIL) is conducted to validate the performance and efficacy of the suggested power scheduling for the emission-free ships.

[1]  Eberhard Waffenschmidt,et al.  A new approach to transform an existing distribution network into a set of micro-grids for enhancing reliability and sustainability , 2017, Appl. Soft Comput..

[2]  Josep M. Guerrero,et al.  Advanced Control Architectures for Intelligent Microgrids—Part I: Decentralized and Hierarchical Control , 2013, IEEE Transactions on Industrial Electronics.

[3]  Majid Nayeripour,et al.  Spectral clustering for designing robust and reliable multi-MG smart distribution systems , 2017 .

[4]  Tomislav Dragicevic,et al.  Robust and Fast Voltage-Source-Converter (VSC) Control for Naval Shipboard Microgrids , 2019, IEEE Transactions on Power Electronics.

[5]  Dongpu Cao,et al.  Reinforcement Learning Optimized Look-Ahead Energy Management of a Parallel Hybrid Electric Vehicle , 2017, IEEE/ASME Transactions on Mechatronics.

[6]  Mohammad Hassan Khooban,et al.  A New Intelligent Hybrid Control Approach for DC–DC Converters in Zero-Emission Ferry Ships , 2020, IEEE Transactions on Power Electronics.

[7]  J S Chalfant,et al.  Analysis of various all-electric-ship electrical distribution system topologies , 2011, 2011 IEEE Electric Ship Technologies Symposium.

[8]  Rudy R. Negenborn,et al.  Ship energy management for hybrid propulsion and power supply with shore charging , 2018, Control Engineering Practice.

[9]  Mohammad Hassan Khooban,et al.  Simultaneous energy management and optimal components sizing of a zero-emission ferry boat , 2020 .

[10]  Hongwen He,et al.  Deep reinforcement learning of energy management with continuous control strategy and traffic information for a series-parallel plug-in hybrid electric bus , 2019, Applied Energy.

[11]  Roy Billinton,et al.  Reliability evaluation of power systems , 1984 .

[12]  Zero-emission casting-off and docking maneuvers for series hybrid excursion ships , 2019 .

[13]  Jingda Wu,et al.  Energy management based on reinforcement learning with double deep Q-learning for a hybrid electric tracked vehicle , 2019, Applied Energy.

[14]  Junwei Cao,et al.  Optimal energy management strategies for energy Internet via deep reinforcement learning approach , 2019, Applied Energy.

[15]  Nassim Rizoug,et al.  Optimal Energy Management for a Li-Ion Battery/Supercapacitor Hybrid Energy Storage System Based on a Particle Swarm Optimization Incorporating Nelder–Mead Simplex Approach , 2017, IEEE Transactions on Intelligent Vehicles.

[16]  Tianhao Tang,et al.  An Energy Management System of a Fuel Cell/Battery Hybrid Boat , 2014 .

[17]  Yuan Shen,et al.  Autonomous Navigation of UAVs in Large-Scale Complex Environments: A Deep Reinforcement Learning Approach , 2019, IEEE Transactions on Vehicular Technology.

[18]  Maryam Imani,et al.  Dynamic multi-objective optimisation using deep reinforcement learning: benchmark, algorithm and an application to identify vulnerable zones based on water quality , 2019, Eng. Appl. Artif. Intell..

[19]  David Gonsoulin,et al.  Predictive Control for Energy Management in Ship Power Systems Under High-Power Ramp Rate Loads , 2017, IEEE Transactions on Energy Conversion.

[20]  M. G. Lauby,et al.  Power system reliability planning practices in North America , 1991 .

[21]  A. Kahrobaeian,et al.  Interactive Distributed Generation Interface for Flexible Micro-Grid Operation in Smart Distribution Systems , 2012, IEEE Transactions on Sustainable Energy.

[22]  Tarlochan S. Sidhu,et al.  Investigations Into the Control and Protection of an Existing Distribution Network to Operate as a Microgrid: A Case Study , 2014, IEEE Transactions on Industrial Electronics.

[23]  Samy Faddel,et al.  Decentralized Control Algorithm for the Hybrid Energy Storage of Shipboard Power System , 2020, IEEE Journal of Emerging and Selected Topics in Power Electronics.

[24]  Ruoli Tang,et al.  A novel optimal energy-management strategy for a maritime hybrid energy system based on large-scale global optimization , 2018, Applied Energy.

[25]  Shimon Whiteson,et al.  Multi-Objective Deep Reinforcement Learning , 2016, ArXiv.

[26]  Sidun Fang,et al.  Two-Step Multi-Objective Management of Hybrid Energy Storage System in All-Electric Ship Microgrids , 2019, IEEE Transactions on Vehicular Technology.

[27]  A. Al-Shaalan Reliability Evaluation of Power Systems , 2019, Reliability and Maintenance - An Overview of Cases.

[28]  Shantha Gamini Jayasinghe,et al.  Power management optimization of hybrid power systems in electric ferries , 2018 .

[29]  Mohammad Hassan Khooban,et al.  A New Adaptive Type-II Fuzzy-Based Deep Reinforcement Learning Control: Fuel Cell Air-Feed Sensors Control , 2019, IEEE Sensors Journal.

[30]  Mohammad Hassan Khooban,et al.  A Comparative Analysis of Optimal Operation Scenarios in Hybrid Emission-Free Ferry Ships , 2020, IEEE Transactions on Transportation Electrification.

[31]  Nooshin Bigdeli,et al.  Optimal management of hybrid PV/fuel cell/battery power system: A comparison of optimal hybrid approaches , 2015 .

[32]  Ritwik Majumder,et al.  Reactive Power Compensation in Single-Phase Operation of Microgrid , 2013, IEEE Transactions on Industrial Electronics.

[33]  Zita Vale,et al.  Coordination between mid-term maintenance outage decisions and short-term security-constrained scheduling in smart distribution systems , 2012 .

[34]  Arif I. Sarwat,et al.  Smart Grid reliability assessment utilizing Boolean Driven Markov Process and variable weather conditions , 2015, 2015 North American Power Symposium (NAPS).

[35]  Suryanarayana Doolla,et al.  Multiagent-Based Distributed-Energy-Resource Management for Intelligent Microgrids , 2013, IEEE Transactions on Industrial Electronics.

[36]  Tom Lenaerts,et al.  Dynamic Weights in Multi-Objective Deep Reinforcement Learning , 2018, ICML.

[37]  Tomislav Dragicevic,et al.  Robust Frequency Regulation in Mobile Microgrids: HIL Implementation , 2019, IEEE Systems Journal.

[38]  Frank L. Lewis,et al.  Mixed Iterative Adaptive Dynamic Programming for Optimal Battery Energy Control in Smart Residential Microgrids , 2017, IEEE Transactions on Industrial Electronics.

[39]  Chenming Li,et al.  Energy Management Strategy for a Hybrid Electric Vehicle Based on Deep Reinforcement Learning , 2018 .

[40]  H. Hofmann,et al.  Control development and performance evaluation for battery/flywheel hybrid energy storage solutions to mitigate load fluctuations in all-electric ship propulsion systems , 2018 .

[41]  Ken Nagasaka,et al.  Multiobjective Intelligent Energy Management for a Microgrid , 2013, IEEE Transactions on Industrial Electronics.

[42]  Asgeir J. Sørensen,et al.  Approaches to Economic Energy Management in Diesel–Electric Marine Vessels , 2017, IEEE Transactions on Transportation Electrification.

[43]  Majid Nayeripour,et al.  Interactive fuzzy binary shuffled frog leaping algorithm for multi-objective reliable economic power distribution system expansion planning , 2015, J. Intell. Fuzzy Syst..

[44]  Karen L. Butler-Purry,et al.  An Integrated Security-Constrained Model-Based Dynamic Power Management Approach for Isolated Microgrids in All-Electric Ships , 2015, IEEE Transactions on Power Systems.

[45]  Jiayi Cao,et al.  Reinforcement learning-based real-time power management for hybrid energy storage system in the plug-in hybrid electric vehicle , 2018 .