论文信息 - Modelling a Penicillin Fermentation Process Using Attention-Based Echo State Networks Optimized by Covariance Matrix Adaption Evolutionary Strategy

Modelling a Penicillin Fermentation Process Using Attention-Based Echo State Networks Optimized by Covariance Matrix Adaption Evolutionary Strategy

Abstract Echo state network (ESN) has emerged as an effective alternative to conventional recurrent neural networks due to its simple training process and good modelling ability for solving a variety of problems, especially time-series modelling tasks. To improve modelling capability and to decrease the reservoir topology complexity, a new attention mechanism based ESN optimised by covariance matrix adaption evolutionary strategy (CMA-ES) is proposed in this paper. CMA-ES is a stochastic and derivative-free algorithm for solving non-linear optimization problems. Attention mechanism is incorporated to guide ESN to focus on regions of interest relevant to the modelling task. The proposed optimised ESN with attention mechanism is used to model a fed-batch penicillin fermentation process and the results are better than those from the standard ESN and ESN with attention mechanism.

K Liu | J Zhang | Kai Liu | Jie Zhang

[1] Petros Koumoutsakos,et al. Reducing the Time Complexity of the Derandomized Evolution Strategy with Covariance Matrix Adaptation (CMA-ES) , 2003, Evolutionary Computation.

[2] Graham Kendall,et al. Optimization of fed-batch fermentation processes using the Backtracking Search Algorithm , 2018, Expert Syst. Appl..

[3] Behzad Moshiri,et al. Optimal control of a nonlinear fed-batch fermentation process using model predictive approach , 2009 .

[4] Barry Lennox,et al. The development of an industrial-scale fed-batch fermentation simulation. , 2015, Journal of biotechnology.

[5] Christopher D. Manning,et al. Effective Approaches to Attention-based Neural Machine Translation , 2015, EMNLP.

[6] Jie Yu,et al. A Bayesian inference based two-stage support vector regression framework for soft sensor development in batch bioprocesses , 2012, Comput. Chem. Eng..

[7] Hendrik Van Brussel,et al. Pruning and regularization in reservoir computing , 2009, Neurocomputing.

[8] Sing Kiong Nguang,et al. Modelling and optimization of fed-batch fermentation processes using dynamic neural networks and genetic algorithms , 2004 .

[9] Herbert Jaeger,et al. Reservoir computing approaches to recurrent neural network training , 2009, Comput. Sci. Rev..