Intelligent Energy Management for Plug-in Hybrid Electric Bus with Limited State Space

Tabular Q-learning (QL) can be easily implemented into a controller to realize self-learning energy management control of a plug-in hybrid electric bus (PHEB). However, the “curse of dimensionality” problem is difficult to avoid, as the design space is huge. This paper proposes a QL-PMP algorithm (QL and Pontryagin minimum principle (PMP)) to address the problem. The main novelty is that the difference between the feedback SOC (state of charge) and the reference SOC is exclusively designed as state, and then a limited state space with 50 rows and 25 columns is proposed. The off-line training process shows that the limited state space is reasonable and adequate for the self-learning; the Hardware-in-Loop (HIL) simulation results show that the QL-PMP strategy can be implemented into a controller to realize real-time control, and can on average improve the fuel economy by 20.42%, compared to the charge depleting–charge sustaining (CDCS) strategy.

[1]  Hongwen He,et al.  Continuous reinforcement learning of energy management with deep Q network for a power split hybrid electric bus , 2018, Applied Energy.

[2]  Hongwen He,et al.  Rule-based energy management for dual-source electric buses extracted by wavelet transform , 2018, Journal of Cleaner Production.

[3]  Bin Li,et al.  A review of power management strategies and component sizing methods for hybrid vehicles , 2018, Renewable and Sustainable Energy Reviews.

[4]  Giorgio Rizzoni,et al.  Energy management for plug-in hybrid electric vehicles using equivalent consumption minimisation strategy , 2010 .

[5]  Naehyuck Chang,et al.  Reinforcement learning based power management for hybrid electric vehicles , 2014, 2014 IEEE/ACM International Conference on Computer-Aided Design (ICCAD).

[6]  Simona Onori,et al.  Adaptive Pontryagin’s Minimum Principle supervisory controller design for the plug-in hybrid GM Chevrolet Volt , 2015 .

[7]  Bo Wang,et al.  Online Markov Chain-based energy management for a hybrid tracked vehicle with speedy Q-learning , 2018, Energy.

[8]  Kumeresan A. Danapalasingam,et al.  A review on hybrid electric vehicles architecture and energy management strategies , 2016 .

[9]  Xiaosong Hu,et al.  Pontryagin’s Minimum Principle based model predictive control of energy management for a plug-in hybrid electric bus , 2019, Applied Energy.

[10]  M. Ouyang,et al.  Approximate Pontryagin’s minimum principle applied to the energy management of plug-in hybrid electric vehicles , 2014 .

[11]  Guoyuan Wu,et al.  Deep reinforcement learning enabled self-learning control for energy efficient driving , 2019, Transportation Research Part C: Emerging Technologies.

[12]  Chee Wei Tan,et al.  A review of energy sources and energy management system in electric vehicles , 2013 .

[13]  Chao Yang,et al.  Adaptive real-time optimal energy management strategy based on equivalent factors optimization for plug-in hybrid electric vehicle , 2017 .

[14]  Jiayi Cao,et al.  Reinforcement learning-based real-time power management for hybrid energy storage system in the plug-in hybrid electric vehicle , 2018 .

[15]  Datong Qin,et al.  Dynamic energy management for a novel hybrid electric system based on driving pattern recognition , 2017 .

[16]  Hongwen He,et al.  Battery SOC constraint comparison for predictive energy management of plug-in hybrid electric bus , 2017 .

[17]  Chenming Li,et al.  Energy Management Strategy for a Hybrid Electric Vehicle Based on Deep Reinforcement Learning , 2018 .

[18]  Chunjiang Bao,et al.  Receding horizon control-based energy management for plug-in hybrid electric buses using a predictive model of terminal SOC constraint in consideration of stochastic vehicle mass , 2019 .

[19]  Christian Schroer,et al.  Deep Reinforcement Learning for Advanced Energy Management of Hybrid Electric Vehicles , 2018, ICAART.

[20]  Raymond Chiong,et al.  A Hooke-Jeeves Based Memetic Algorithm for Solving Dynamic Optimisation Problems , 2009, HAIS.

[21]  Paul A. Jennings,et al.  Blended Rule-Based Energy Management for PHEV: System Structure and Strategy , 2016, IEEE Transactions on Vehicular Technology.

[22]  Yang Zhou,et al.  A survey on driving prediction techniques for predictive energy management of plug-in hybrid electric vehicles , 2019, Journal of Power Sources.

[23]  Youngsun Bak,et al.  Development of route information based driving control algorithm for a range-extended electric vehicle , 2016 .