Deep neural network model with Bayesian hyperparameter optimization for prediction of NOx at transient conditions in a diesel engine

Abstract Owing to increasing interest in the environment, particularly on air quality, regulations in the automobile industry have become stricter. Test cycles have been substituted to simulate real driving conditions, and they offer opportunities for researchers to satisfy regulations and predict emissions using models. The objective of this study is to develop a deep neural network (DNN) model, optimize its hyperparameters using the Bayesian optimization method, and use hidden-node determination logic to predict engine-out NO x emissions by using the worldwide harmonized light vehicles test procedure (WLTP) of diesel engines. A DNN network learns the internal relationships between inputs and target outputs even though they are complicated. However, the hyperparameters of DNNs are typically determined by researchers before training, and they affected the accuracy of the model. In this study, the hyperparameters of the DNN model such as the number of hidden layers, number of nodes in each hidden layer, learning rate, learning rate decay, and batch size are automatically optimized using the Bayesian optimization method. Some logical equations are combined with the number of nodes in the first hidden layer and the number of hidden layers to realize the model’s structure instead of using the number of hidden nodes in each hidden layer. Compared with grid search and random sampling, the Bayesian optimization method is a promising solution to optimize hyperparameters. In addition, a hidden-node determination logic further improved the accuracy of the model. The accuracy of the optimized model is indicated by an R2 value of 0.9675 with 14 input features. The result of cycle prediction shows that the mean absolute errors are approximately 16–17 ppm for four WLTP cycles, which are 1.6% of the maximum NO x value. These results indicate that the accuracy of the model is comparable to that of a physical NO x measurement device whose linearity is 1% of the full scale (5,000 ppm).

[1]  Yoshua Bengio,et al.  Practical Recommendations for Gradient-Based Training of Deep Architectures , 2012, Neural Networks: Tricks of the Trade.

[2]  T. Johnson.  Diesel Emission Control in Review , 2001 .

[3]  Rolf Egnell,et al.  Combustion Diagnostics by Means of Multizone Heat Release Analysis and NO Calculation , 1998 .

[4]  Nando de Freitas,et al.  Taking the Human Out of the Loop: A Review of Bayesian Optimization , 2016, Proceedings of the IEEE.

[5]  Zhen Huang,et al.  Review of state of the art technologies of selective catalytic reduction of NOx from diesel engine exhaust , 2014 .

[6]  Yoshua Bengio,et al.  Random Search for Hyper-Parameter Optimization , 2012, J. Mach. Learn. Res..

[7]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[8]  O. Özener,et al.  Modelıng of NOx Emmıssıons in Internal Combustıon Engıne , 2019, International Journal of Engineering Research and Advanced Technology.

[9]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[10]  Ezio Spessa,et al.  Real-Time Simulation of Torque and Nitrogen Oxide Emissions in an 11.0 L Heavy-Duty Diesel Engine for Model-Based Combustion Control , 2019, Energies.

[11]  E. Özgül,et al.  Use of thermodynamical models with predictive combustion and emission capability in virtual calibration of heavy duty engines , 2020 .

[12]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[13]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[14]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[15]  Lino Guzzella,et al.  A fast and accurate physics-based model for the NOx emissions of Diesel engines , 2013 .

[16]  Gyujin Kim,et al.  Development of a Real-Time Virtual Nitric Oxide Sensor for Light-Duty Diesel Engines , 2017 .

[17]  Carlos Guardiola,et al.  ECU-oriented models for NOx prediction. Part 1: a mean value engine model for NOx prediction , 2015 .

[18]  Nando de Freitas,et al.  A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning , 2010, ArXiv.

[19]  Sepp Hochreiter,et al.  Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs) , 2015, ICLR.

[20]  Cycle-to-Cycle NO and NOx Emissions From a HSDI Diesel Engine , 2018, Journal of Engineering for Gas Turbines and Power.

[21]  J. G. Hawley,et al.  Diesel engine exhaust gas recirculation--a review on advanced and novel concepts , 2004 .

[22]  E. Özgül,et al.  Fast NOx emission prediction methodology via one-dimensional engine performance tools in heavy-duty engines , 2019, Advances in Mechanical Engineering.