Optimizing a Start-Stop Controller Using Policy Search

We applied a policy search algorithm to the problem of optimizing a start-stop controller--a controller used in a car to turn off the vehicle's engine, and thus save energy, when the vehicle comes to a temporary halt. We were able to improve the existing policy by approximately 12% using real driver trace data. We also experimented with using multiple policies, and found that doing so could lead to a further 8% improvement if we could determine which policy to apply at each stop. The driver's behaviors before stopping were found to be uncorrelated with the policy that performed best; however, further experimentation showed that the driver's behavior during the stop may be more useful, suggesting a useful direction for adding complexity to the underlying start-stop policy.

[1]  XiaoYong Wang,et al.  Vehicle System Control for Start-Stop Powertrains with Automatic Transmissions , 2013 .

[2]  Kimiyoshi Nishizawa,et al.  Stop-Start System with Compact Motor Generator and Newly Developed Direct Injection Gasoline Engine , 2012 .

[3]  Claudia V. Goldman,et al.  Learning Driver's Behavior to Improve the Acceptance of Adaptive Cruise Control , 2012, IAAI.

[4]  Danil Prokhorov,et al.  Computational Intelligence in Automotive Applications , 2008, Computational Intelligence in Automotive Applications.

[5]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[6]  Yishay Mansour,et al.  Policy Gradient Methods for Reinforcement Learning with Function Approximation , 1999, NIPS.

[7]  J. Zico Kolter,et al.  Design, analysis, and learning control of a fully actuated micro wind turbine , 2012, 2012 American Control Conference (ACC).

[8]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[9]  R. J. Williams,et al.  Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning , 2004, Machine Learning.

[10]  George Theocharous,et al.  Machine Learning for Adaptive Power Management , 2006 .

[11]  David A. Patterson,et al.  A Case For Adaptive Datacenters To Conserve Energy and Improve Reliability , 2008 .

[12]  M.A. Masrur,et al.  Model-based fault diagnosis in electric drives using machine learning , 2006, IEEE/ASME Transactions on Mechatronics.

[13]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[14]  Ertugrul Taspinar,et al.  An Engine Start/Stop System for Improved Fuel Economy , 2007 .

[15]  Andras Csepinszky.  Operational Results and Conclusions of the FOT Execution Phase of EuroFOT European Large Scale Field Operational Test , 2011 .

[16]  Yi Lu Murphey,et al.  Intelligent Vehicle Power Control Based on Machine Learning of Optimal Control Parameters and Prediction of Road Type and Traffic Congestion , 2009, IEEE Transactions on Vehicular Technology.

[17]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[18]  Chi-Man Vong,et al.  Prediction of automotive engine power and torque using least squares support vector machines and Bayesian inference , 2006, Eng. Appl. Artif. Intell..

[19]  Ilya V. Kolmanovsky,et al.  Predictive energy management of a power-split hybrid electric vehicle , 2009, 2009 American Control Conference.

[20]  Peter Stone,et al.  Machine Learning for Fast Quadrupedal Locomotion , 2004, AAAI.

[21]  Konstantinos Dalamagkidis,et al.  Reinforcement Learning for Building Environmental Control , 2008 .