A Learning Algorithm for Optimal Internal Combustion Engine Calibration in Real Time

Advanced internal combustion engine technologies have increased the number of accessible variables of an engine and our ability to control them. The optimal values of these variables are designated during engine calibration by means of a static correlation between the controllable variables and the corresponding steady-state engine operating points. While the engine is running, these correlations are being interpolated to provide values of the controllable variables for each operating point. These values are controlled by the electronic control unit to achieve desirable engine performance, for example in fuel economy, pollutant emissions, and engine acceleration. The state-of-the-art engine calibration cannot guarantee continuously optimal engine operation for the entire operating domain, especially in transient cases encountered in driving styles of different drivers. This paper presents the theoretical basis and algorithmic implementation for allowing the engine to learn the optimal set values of accessible variables in real time while running a vehicle. Through this new approach, the engine progressively perceives the driver’s driving style and eventually learns to operate in a manner that optimizes specified performance indices. The effectiveness of the approach is demonstrated through simulation of a spark ignition engine, which learns to optimize fuel economy with respect to spark ignition timing, while it is running a vehicle.Copyright © 2007 by ASME

[1]  Jae In Park,et al.  Performance Analysis and Valve Event Optimization for SI Engines Using Fractal Combustion Model , 2006 .

[2]  Richard S. Sutton,et al.  Learning to predict by the methods of temporal differences , 1988, Machine Learning.

[3]  Chris Watkins,et al.  Learning from delayed rewards , 1989 .

[4]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[5]  Sven Meyer,et al.  New Calibration Methods and Control Systems with Artificial Neural Networks , 2002 .

[6]  L. Sennott.  Stochastic Dynamic Programming and the Control of Queueing Systems , 1998 .

[7]  Jim Rutherford,et al.  Practical Experiment Designs for Engineers and Scientists , 2002, Technometrics.

[8]  Christopher J. Rutland,et al.  Improvement of Neural Network Accuracy for Engine Simulations , 2003 .

[9]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[10]  Richard Wheeler,et al.  Decentralized learning in finite Markov chains , 1985, 1985 24th IEEE Conference on Decision and Control.

[11]  Richard O. Lynch,et al.  Introduction to Design and Analysis of Experiments , 1999, Technometrics.

[12]  Eric Rask,et al.  Simulation-Based Engine Calibration: Tools, Techniques, and Applications , 2004 .

[13]  Zoran Filipi,et al.  Cam-Phasing Optimization Using Artificial Neural Networks as Surrogate Models-Maximizing Torque Output , 2005 .

[14]  M. Jankovic,et al.  Fuel economy optimization in automotive engines , 2006, 2006 American Control Conference.

[15]  David Lowe,et al.  Validation of neural networks in automotive engine calibration , 1997 .

[16]  C. Ireland.  Fundamental concepts in the design of experiments , 1964 .

[17]  Panos Y. Papalambros,et al.  Principles of Optimal Design: Author Index , 2000 .

[18]  Panos Y. Papalambros,et al.  Principles of Optimal Design: Modeling and Computation , 1988 .

[19]  John B. Heywood,et al.  Internal combustion engine fundamentals , 1988 .

[20]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[21]  Chris Brace,et al.  Dynamic Behaviour of a High Speed Direct Injection Diesel Engine , 1999 .

[22]  Mohamed Ayeb,et al.  SI Engine Emissions Model Based on Dynamic Neural Networks and D-Optimality , 2005 .

[23]  Henry P. Wynn,et al.  Statistics for engine optimization , 2000 .

[24]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[25]  Christopher M. Atkinson,et al.  Dynamic Model-Based Calibration Optimization: An Introduction and Application to Diesel Engines , 2005 .

[26]  Naohiro Ishii,et al.  Combining Exploitation-Based and Exploration-Based Approach in Reinforcement Learning , 2000, IDEAL.

[27]  R. M. Green,et al.  Measuring the Cylinder-to-Cylinder EGR Distribution in the Intake of a Diesel Engine During Transient Operation , 2000 .

[28]  Chris Nightingale,et al.  Variation of Both Symmetric and Asymmetric Valve Events on a 4-Valve SI Engine and the Effects on Emissions and Fuel Economy , 2000 .

[29]  William J. Diamond.  Practical Experiment Designs for Engineers and Scientists , 1981 .

[30]  Zoran Filipi,et al.  Cam-phasing Optimization Using Artificial Neural Networks as Surrogate Models-Fuel Consumption and NOx Emissions , 2006 .

[31]  Junichiro Yoshimoto,et al.  Control of exploitation-exploration meta-parameter in reinforcement learning , 2002, Neural Networks.

[32]  Thomas Kruse,et al.  Automated Model-Based GDI Engine Calibration Adaptive Online DoE Approach , 2002 .

[33]  Dimitrios C. Rakopoulos,et al.  The Effect of Various Dynamic, Thermodynamic and Design Parameters on the Performance of a Turbocharged Diesel Engine Operating under Transient Load Conditions , 2004 .

[34]  Zoran Filipi,et al.  Transient Diesel Emissions: Analysis of Engine Operation During a Tip-In , 2006 .

[35]  양성봉,et al.  Implementation of the Agent using Universal On-line Q-learning by Balancing Exploration and Exploitation in Reinforcement Learning , 2003 .

[36]  Leonie Kohl,et al.  Fundamental Concepts in the Design of Experiments , 2000 .

[37]  Michael Fischer,et al.  Efficient Layout and Calibration of Variable Valve Trains , 2001 .

[38]  Mark Paul Gravesend Guerrier,et al.  The Development of Model Based Methodologies for Gasoline IC Engine Calibration , 2004 .

[39]  Janet P. Buckingham,et al.  Evaluation of HCCI Engine Potentials in Comparison to Advanced Gasoline and Diesel Engines , 2006 .

[40]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[41]  Sean R Eddy,et al.  What is dynamic programming? , 2004, Nature Biotechnology.

[42]  Cleophas C. Jackson,et al.  Effects of Steady-State and Transient Operation on Exhaust Emissions from Nonroad and Highway Diesel Engines , 1998 .