An Online Learning Control Strategy for Hybrid Electric Vehicle Based on Fuzzy Q-Learning

In order to realize the online learning of a hybrid electric vehicle (HEV) control strategy, a fuzzy Q-learning (FQL) method is proposed in this paper. FQL control strategies consists of two parts: The optimal action-value function Q *( x , u ) estimator network (QEN) and the fuzzy parameters tuning (FPT). A back propagation (BP) neural network is applied to estimate Q *( x , u ) as QEN. For the fuzzy controller, we choose a Sugeno-type fuzzy inference system (FIS) and the parameters of the FIS are tuned online based on Q *( x , u ). The action exploration modifier (AEM) is introduced to guarantee all actions are tried. The main advantage of a FQL control strategy is that it does not rely on prior information related to future driving conditions and can self-tune the parameters of the fuzzy controller online. The FQL control strategy has been applied to a HEV and simulation tests have been done. Simulation results indicate that the parameters of the fuzzy controller are tuned online and that a FQL control strategy achieves good performance in fuel economy.

[1]  Youngjin Park,et al.  Optimal adaptation of equivalent factor of equivalent consumption minimization strategy for fuel cell hybrid electric vehicles under active state inequality constraints , 2014 .

[2]  Xia Meng,et al.  The application of fuzzy-neural network on control strategy of hybrid vehicles , 2010 .

[3]  Ben J. A. Kröse,et al.  Learning from delayed rewards , 1995, Robotics Auton. Syst..

[4]  G. Ripaccioli,et al.  Stochastic model predictive control with driver behavior learning for improved powertrain control , 2010, 49th IEEE Conference on Decision and Control (CDC).

[5]  Clive Roberts,et al.  A Power-Management Strategy for Multiple-Unit Railroad Vehicles , 2011, IEEE Transactions on Vehicular Technology.

[6]  Angelika Heinzel,et al.  Power management optimization of fuel cell/battery hybrid vehicles with experimental validation , 2014 .

[7]  Stefano Di Cairano,et al.  MPC-Based Energy Management of a Power-Split Hybrid Electric Vehicle , 2012, IEEE Transactions on Control Systems Technology.

[8]  Huei Peng,et al.  Power management strategy for a parallel hybrid electric truck , 2003, IEEE Trans. Control. Syst. Technol..

[9]  Zheng Chen,et al.  Energy management of a power-split plug-in hybrid electric vehicle based on genetic algorithm and quadratic programming , 2014 .

[10]  Pierluigi Pisu,et al.  A Comparative Study Of Supervisory Control Strategies for Hybrid Electric Vehicles , 2007, IEEE Transactions on Control Systems Technology.

[11]  Yangsheng Xu,et al.  Online learning control for hybrid electric vehicle , 2012 .

[12]  Piyush Bubna,et al.  Prediction-based optimal power management in a fuel cell/battery plug-in hybrid vehicle , 2010 .

[13]  Chen Rongguang,et al.  The application of fuzzy-neural network on control strategy of hybrid vehicles , 2008, 2008 27th Chinese Control Conference.

[14]  Huiyong Chen,et al.  Simulation of hybrid electric vehicle control strategy based on compensation fuzzy neural network , 2008, 2008 7th World Congress on Intelligent Control and Automation.

[15]  Aldo Sorniotti,et al.  Power split strategies for hybrid energy storage systems for vehicular applications , 2014 .

[16]  Chen Zhang,et al.  Route Preview in Energy Management of Plug-in Hybrid Vehicles , 2012, IEEE Transactions on Control Systems Technology.

[17]  Hui Xie,et al.  Model predictive control for series-parallel plug-in hybrid electrical vehicle using GPS system , 2011, 2011 International Conference on Electrical and Control Engineering.

[18]  Guo-Ping Liu,et al.  Optimal fuzzy power control and management of fuel cell/battery hybrid vehicles , 2009 .

[19]  Ali Emadi,et al.  Classification and Review of Control Strategies for Plug-In Hybrid Electric Vehicles , 2011, IEEE Transactions on Vehicular Technology.

[20]  Mohammad J. Mahjoob,et al.  Optimal energy management in a dual-storage fuel-cell hybrid vehicle using multi-dimensional dynamic programming , 2014 .

[21]  Chi-Kwong Li,et al.  An approach to tune fuzzy controllers based on reinforcement learning for autonomous vehicle control , 2005, IEEE Transactions on Intelligent Transportation Systems.

[22]  Xia Meng,et al.  Optimized fuzzy logic control strategy of hybrid vehicles using ADVISOR , 2010, 2010 International Conference on Computer, Mechatronics, Control and Electronic Engineering.

[23]  Miguel A. Ridao,et al.  Model Predictive Control for power management in hybrid fuel cell vehicles , 2010, 2010 IEEE Vehicle Power and Propulsion Conference.

[24]  Chen Zhang,et al.  Role of Terrain Preview in Energy Management of Hybrid Electric Vehicles , 2010, IEEE Transactions on Vehicular Technology.