Deep Reinforcement Learning for Advanced Energy Management of Hybrid Electric Vehicles

Machine Learning seizes a substantial role in the development of future low-emission automobiles, as manufacturers are increasingly reaching limits with traditional engineering methods. Apart from autonomous driving, recent advances in reinforcement learning also offer great benefit for solving complex parameterization tasks. In this paper, deep reinforcement learning is used for the derivation of efficient operating strategies for hybrid electric vehicles. There, for achieving fuel efficient solutions, a wide range of potential driving and traffic scenarios have to be anticipated where intelligent and adaptive processes could bring significant improvements. The underlying research proves the ability of a reinforcement learning agent to learn nearlyoptimal operating strategies without any prior route-information and offers great potential for the inclusion of further variables into the optimization process.

[1]  G. Corde,et al.  Towards real-time optimal energy management of HEV powertrains using stochastic dynamic programming , 2012, 2012 IEEE Vehicle Power and Propulsion Conference.

[2]  Huei Peng,et al.  Shortest path stochastic control for hybrid electric vehicles , 2008 .

[3]  Lino Guzzella,et al.  Vehicle Propulsion Systems: Introduction to Modeling and Optimization , 2005 .

[4]  Michael Back,et al.  DETERMINATION OF THE FUEL-OPTIMAL TRAJECTORY FOR A VEHICLE ALONG A KNOWN ROUTE , 2002 .

[5]  Alex Graves,et al.  Playing Atari with Deep Reinforcement Learning , 2013, ArXiv.

[6]  Tony Givargis,et al.  Fine-grained acceleration control for autonomous intersection management using deep reinforcement learning , 2017, 2017 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computed, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI).

[7]  Bernard Bäker,et al.  Generation of Replacement Vehicle Speed Cycles Based on Extensive Customer Data by Means of Markov Models and Threshold Accepting , 2017 .

[8]  Guy Lever,et al.  Deterministic Policy Gradient Algorithms , 2014, ICML.

[9]  Hosam K. Fathy,et al.  Hybrid Electric Vehicles: Energy Management Strategies [Bookshelf] , 2018, IEEE Control Systems.

[10]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[11]  Yuval Tassa,et al.  Continuous control with deep reinforcement learning , 2015, ICLR.

[12]  Sergey Levine,et al.  Trust Region Policy Optimization , 2015, ICML.

[13]  Sergey Levine,et al.  Continuous Deep Q-Learning with Model-based Acceleration , 2016, ICML.

[14]  Chung Choo Chung,et al.  Autonomous braking system via deep reinforcement learning , 2017, 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC).

[15]  Andrei N. Borodin,et al.  Ornstein-Uhlenbeck Process , 1996 .

[16]  Naehyuck Chang,et al.  Reinforcement learning based power management for hybrid electric vehicles , 2014, 2014 IEEE/ACM International Conference on Computer-Aided Design (ICCAD).

[17]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[18]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[19]  Pieter Abbeel,et al.  Benchmarking Deep Reinforcement Learning for Continuous Control , 2016, ICML.

[20]  Chang Liu,et al.  Power management for Plug-in Hybrid Electric Vehicles using Reinforcement Learning with trip information , 2014, 2014 IEEE Transportation Electrification Conference and Expo (ITEC).

[21]  Alexandre Chasse,et al.  Supervisory control of hybrid powertrains: An experimental benchmark of offline optimization and online energy management , 2009 .