Powertrain Control for Hybrid-Electric Vehicles Using Supervised Machine Learning

This paper presents a novel framework to enable automatic re-training of the supervisory powertrain control strategy for hybrid electric vehicles using supervised machine learning. The aim of re-training is to customize the control strategy to a user-specific driving behavior without human intervention. The framework is designed to update the control strategy at the end of a driving task. A combination of dynamic programming and supervised machine learning is used to train the control strategy. The trained control strategy denoted as SML is compared to an online-implementable strategy based on the combination of the optimal operation line and Pontryagin’s minimum principle denoted as OOL-PMP, on the basis of fuel consumption. SML consistently performed better than OOL-PMP, evaluated over five standard drive cycles. The EUDC performance was almost identical while on FTP75 the OOL-PMP consumed 14.7% more fuel than SML. Moreover, the deviation from the global benchmark obtained from dynamic programming was between 1.8% and 5.4% for SML and between 5.8% and 16.8% for OOL-PMP. Furthermore, a test-case was conducted to emulate a real-world driving scenario wherein a trained controller is exposed to a new drive cycle. It is found that the performance on the new drive cycle deviates significantly from the optimal policy; however, this performance gap is bridged with a single re-training episode for the respective test-case.

[1]  A. Berthon,et al.  Fuzzy-logic-based control applied to a hybrid electric vehicle with four separate wheel drives , 2004 .

[2]  Erika Ábrahám,et al.  Learning-based control strategies for hybrid electric vehicles , 2015, 2015 IEEE Conference on Control Applications (CCA).

[3]  Suk Won Cha,et al.  Numerical comparison of ECMS and PMP-based optimal control strategy in hybrid vehicles , 2014 .

[4]  Jih-Gau Juang,et al.  Comparison of classical control and intelligent control for a MIMO system , 2008, Appl. Math. Comput..

[5]  Demis Hassabis,et al.  Mastering the game of Go without human knowledge , 2017, Nature.

[6]  Huei Peng,et al.  Optimal Control of Hybrid Electric Vehicles Based on Pontryagin's Minimum Principle , 2011, IEEE Transactions on Control Systems Technology.

[7]  Sean R Eddy,et al.  What is dynamic programming? , 2004, Nature Biotechnology.

[8]  John N. Tsitsiklis,et al.  Parallel and distributed computation , 1989 .

[9]  Qing Wang,et al.  Intelligent Hybrid Vehicle Power Control—Part II: Online Intelligent Energy Management , 2013, IEEE Transactions on Vehicular Technology.

[10]  Daniel F. Opila,et al.  Real-Time Implementation and Hardware Testing of a Hybrid Vehicle Energy Management Controller Based on Stochastic Dynamic Programming , 2013 .

[11]  Simona Onori,et al.  ECMS as a realization of Pontryagin's minimum principle for HEV control , 2009, 2009 American Control Conference.

[12]  M. Steinbuch,et al.  CVT ratio control strategy optimization , 2005, 2005 IEEE Vehicle Power and Propulsion Conference.

[13]  Yeong-Il Park,et al.  Multi-Mode Driving Control of a Parallel Hybrid Electric Vehicle Using Driving Pattern Recognition , 2002 .

[14]  Pierluigi Pisu,et al.  A Comparative Study Of Supervisory Control Strategies for Hybrid Electric Vehicles , 2007, IEEE Transactions on Control Systems Technology.

[15]  Thierry-Marie Guerra,et al.  Equivalent consumption minimization strategy for parallel hybrid powertrains , 2002, Vehicular Technology Conference. IEEE 55th Vehicular Technology Conference. VTC Spring 2002 (Cat. No.02CH37367).

[16]  Hongwen He,et al.  Power Management for a Plug-in Hybrid Electric Vehicle Based on Reinforcement Learning with Continuous State and Action Spaces , 2017 .

[17]  Simona Onori,et al.  A Comparative Analysis of Energy Management Strategies for Hybrid Electric Vehicles , 2011 .

[18]  Lino Guzzella,et al.  Vehicle Propulsion Systems: Introduction to Modeling and Optimization , 2005 .

[19]  Thomas Franke,et al.  Understanding charging behaviour of electric vehicle users , 2013 .

[20]  Teng Liu,et al.  Reinforcement Learning for Hybrid and Plug-In Hybrid Electric Vehicle Energy Management: Recent Advances and Prospects , 2019, IEEE Industrial Electronics Magazine.

[21]  Xiaosong Hu,et al.  Comparison of power-split and parallel hybrid powertrain architectures with a single electric machine: Dynamic programming approach , 2016 .

[22]  Seong-chul Lee,et al.  Fuel Economy Optimization for Parallel Hybrid Vehicles with CVT , 1999 .

[23]  Rolf Pfiffner.  Optimal operation of CVT-based powertrains , 2001 .

[24]  Giorgio Rizzoni,et al.  A-ECMS: An Adaptive Algorithm for Hybrid Electric Vehicle Energy Management , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[25]  Piotr Bielaczyc,et al.  An assessment of regulated emissions and CO2 emissions from a European light-duty CNG-fueled vehicle in the context of Euro 6 emissions regulations , 2014 .

[26]  Eva Ericsson,et al.  Independent driving pattern factors and their influence on fuel-use and exhaust emission factors , 2001 .

[27]  Hyung-Soo Kim,et al.  CVT ratio control with consideration of CVT system loss , 2008 .

[28]  Yanjun Huang,et al.  Model predictive control power management strategies for HEVs: A review , 2017 .

[29]  J.T.B.A. Kessels,et al.  Optimal Control of Hybrid Vehicles , 2013 .

[30]  Alex Graves,et al.  Playing Atari with Deep Reinforcement Learning , 2013, ArXiv.

[31]  Theo Hofman,et al.  RULE-BASED ENERGY MANAGEMENT STRATEGIES FOR HYBRID VEHICLE DRIVETRAINS: A FUNDAMENTAL APPROACH IN REDUCING COMPUTATION TIME , 2006 .

[32]  Jiayi Cao,et al.  Reinforcement learning-based real-time power management for hybrid energy storage system in the plug-in hybrid electric vehicle , 2018 .

[33]  Seung-Ki Sul,et al.  Torque control strategy for a parallel hybrid vehicle using fuzzy logic , 1998, Conference Record of 1998 IEEE Industry Applications Conference. Thirty-Third IAS Annual Meeting (Cat. No.98CH36242).

[34]  Martina Josevski,et al.  Distributed predictive control approach for fuel efficient gear shifting in hybrid electric vehicles , 2016, 2016 European Control Conference (ECC).

[35]  R. Trigui,et al.  Predictive energy management of hybrid vehicle , 2008, 2008 IEEE Vehicle Power and Propulsion Conference.

[36]  Simona Onori,et al.  On Adaptive-ECMS strategies for hybrid electric vehicles , 2011 .

[37]  Yuanwei Jing,et al.  Study and Simulation of Based-fuzzy-logic Parallel Hybrid Electric Vehicles Control Strategy , 2006, Sixth International Conference on Intelligent Systems Design and Applications.

[38]  Sluis Francis van der,et al.  Key Technologies of the Pushbelt CVT - Status and New Developments - , 2013 .