Optimization methodology based on neural networks and self-adaptive differential evolution algorithm applied to an aerobic fermentation process

The determination of the optimal neural network topology is an important aspect when using neural models. Due to the lack of consistent rules, this is a difficult problem, which is solved in this paper using an evolutionary algorithm namely Differential Evolution. An improved, simple, and flexible self-adaptive variant of Differential Evolution algorithm is proposed and tested. The algorithm included two initialization strategies (normal distribution and normal distribution combined with the opposition based principle) and a modified mutation principle. Because the methodology contains new elements, a specific name has been assigned, SADE-NN-1. In order to determine the most influential inputs of the models, a sensitivity analysis was applied. The case study considered in this work refer to the oxygen mass transfer coefficient in stirred bioreactors in the presence of n-dodecane as oxygen vector. The oxygen transfer in the fermentation broths has a significant influence on the growth of cultivated microorganism, the accurate modeling of this process being an important problem that has to be solved in order to optimize the aerobic fermentation process. The neural networks predicted the mass transfer coefficients with high accuracy, which indicates that the proposed methodology had a good performance. The same methodology, with a few modifications, and with the best neural network models, was used for determining the optimal conditions for which the mass transfer coefficient is maximized. A short review of the differential evolution methodology is realized in the first part of this article, presenting the main characteristics and variants, with advantages and disadvantages, and fitting in the modifications proposed within the existing directions of research.

[1]  Dario Floreano,et al.  Neuroevolution: from architectures to learning , 2008, Evol. Intell..

[2]  Marius Turnea,et al.  Comparative study on the effects of n-dodecane addition on oxygen transfer in stirred bioreactors for simulated, bacterial and yeasts broths , 2006 .

[3]  Liang Gao,et al.  A differential evolution algorithm with self-adapting strategy and control parameters , 2011, Comput. Oper. Res..

[4]  Shiliang Sun,et al.  Weighted Data Normalization Based on Eigenvalues for Artificial Neural Network Classification , 2009, ICONIP.

[5]  K. Zielinski,et al.  Stopping Criteria for Differential Evolution in Constrained Single-Objective Optimization , 2008 .

[6]  Vitaliy Feoktistov Differential Evolution: In Search of Solutions , 2006 .

[7]  Silvia Curteanu,et al.  Modeling of oxygen mass transfer in the presence of oxygen-vectors using neural networks developed by differential evolution algorithm , 2011, Eng. Appl. Artif. Intell..

[8]  Daniela Zaharie,et al.  Influence of crossover on the behavior of Differential Evolution Algorithms , 2009, Appl. Soft Comput..

[9]  Ajith Abraham,et al.  A simple adaptive Differential Evolution algorithm , 2009, 2009 World Congress on Nature & Biologically Inspired Computing (NaBIC).

[10]  Ling Wang,et al.  A hybrid differential evolution method for permutation flow-shop scheduling , 2008 .

[11]  R. Storn,et al.  On the usage of differential evolution for function optimization , 1996, Proceedings of North American Fuzzy Information Processing.

[12]  M. M. Ali,et al.  Differential evolution with preferential crossover , 2007, Eur. J. Oper. Res..

[13]  Lorie M. Liebrock,et al.  Empirical sensitivity analysis for computational procedures , 2005, 2005 Richard Tapia Celebration of Diversity in Computing Conference.

[14]  M.A. Mazurowski,et al.  Limitations of sensitivity analysis for neural networks in cases with dependent inputs , 2006, 2006 IEEE International Conference on Computational Cybernetics.

[15]  Stéphane Doncieux,et al.  MENNAG: a modular, regular and hierarchical encoding for neural-networks based on attribute grammars , 2008, Evol. Intell..

[16]  B. V. Babu,et al.  Multiobjective differential evolution (MODE) for optimization of adiabatic styrene reactor , 2005 .

[17]  Janez Brest,et al.  Constrained Real-Parameter Optimization with ε -Self-Adaptive Differential Evolution , 2009 .

[18]  Ashish M. Gujarathi,et al.  Improved Multiobjective Differential Evolution (MODE) Approach for Purified Terephthalic Acid (PTA) Oxidation Process , 2009 .

[19]  Bidyadhar Subudhi,et al.  A differential evolution based neural network approach to nonlinear system identification , 2011, Appl. Soft Comput..

[20]  Shahryar Rahnamayan,et al.  Opposition-Based Differential Evolution Algorithms , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[21]  R. Storn,et al.  Differential Evolution: A Practical Approach to Global Optimization (Natural Computing Series) , 2005 .

[22]  Dezhao Chen,et al.  An improved differential evolution algorithm in training and encoding prior knowledge into feedforward networks with application in chemistry , 2002 .

[23]  Ville Tirronen,et al.  Recent advances in differential evolution: a survey and experimental analysis , 2010, Artificial Intelligence Review.

[24]  X. Yao Evolving Artificial Neural Networks , 1999 .

[25]  Hamid R. Tizhoosh,et al.  Opposition-Based Learning: A New Scheme for Machine Intelligence , 2005, International Conference on Computational Intelligence for Modelling, Control and Automation and International Conference on Intelligent Agents, Web Technologies and Internet Commerce (CIMCA-IAWTIC'06).

[26]  Silvia Curteanu,et al.  Neural networks applied in chemistry. I. Determination of the optimal topology of multilayer perceptron neural networks , 2011 .

[27]  Josef Tvrdík,et al.  Adaptive differential evolution and exponential crossover , 2008, 2008 International Multiconference on Computer Science and Information Technology.

[28]  Joni-Kristian Kämäräinen,et al.  Differential Evolution Training Algorithm for Feed-Forward Neural Networks , 2003, Neural Processing Letters.

[29]  M. Gevrey,et al.  Review and comparison of methods to study the contribution of variables in artificial neural network models , 2003 .

[30]  E. Nicoara,et al.  Mechanisms to Avoid the Premature Convergence of Genetic Algorithms , 2009 .

[31]  H. Abbass The self-adaptive Pareto differential evolution algorithm , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[32]  Carlos A. Coello Coello,et al.  Simple Feasibility Rules and Differential Evolution for Constrained Optimization , 2004, MICAI.

[33]  D. Zaharie A Comparative Analysis of Crossover Variants in Differential Evolution , 2007 .

[34]  Hussein A. Abbass,et al.  A Memetic Pareto Evolutionary Approach to Artificial Neural Networks , 2001, Australian Joint Conference on Artificial Intelligence.

[35]  Rainer Storn,et al.  Differential Evolution Research – Trends and Open Questions , 2008 .

[36]  Jia-Sheng Heh,et al.  A 2-Opt based differential evolution for global optimization , 2010, Appl. Soft Comput..

[37]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[38]  Silvia Curteanu,et al.  Multi-objective optimization of a stacked neural network using an evolutionary hyper-heuristic , 2012, Appl. Soft Comput..

[39]  Bidyadhar Subudhi,et al.  An improved differential evolution trained neural network scheme for nonlinear system identification , 2009, Int. J. Autom. Comput..

[40]  Joseph S. Alford,et al.  Bioprocess control: Advances and challenges , 2006, Comput. Chem. Eng..

[41]  Przemyslaw M. Szecowka,et al.  On reliability of neural network sensitivity analysis applied for sensor array optimization , 2011 .

[42]  Mangesh D. Kapadi,et al.  Optimal control of fed-batch fermentation involving multiple feeds using Differential Evolution , 2004 .

[43]  Marius Turnea,et al.  Enhancement of oxygen mass transfer in stirred bioreactors using oxygen-vectors 2. Propionibacterium shermanii broths , 2005, Bioprocess and biosystems engineering.

[44]  T. Jayabarathi,et al.  Combined Hybrid Differential Particle Swarm Optimization Approach for Economic Dispatch Problems , 2010 .

[45]  Godfrey C. Onwubolu,et al.  Forward Backward Transformation , 2009 .

[46]  Debashisha Jena,et al.  A combined differential evolution and neural network approach to nonlinear system identification , 2008, TENCON 2008 - 2008 IEEE Region 10 Conference.

[47]  R. Storn,et al.  Differential Evolution - A simple and efficient adaptive scheme for global optimization over continuous spaces , 2004 .

[48]  Dimitris K. Tasoulis,et al.  A Review of Major Application Areas of Differential Evolution , 2008 .

[49]  D. Zaharie Statistical Properties of Differential Evolution and Related Random Search Algorithms , 2008 .

[50]  César Hervás-Martínez,et al.  Memetic pareto differential evolutionary artificial neural networks to determine growth multi-classes in predictive microbiology , 2010, Evol. Intell..

[51]  Janez Brest,et al.  Self-Adapting Control Parameters in Differential Evolution: A Comparative Study on Numerical Benchmark Problems , 2006, IEEE Transactions on Evolutionary Computation.

[52]  Xuefeng Yan,et al.  An Immune Self-adaptive Differential Evolution Algorithm with Application to Estimate Kinetic Parameters for Homogeneous Mercury Oxidation , 2009 .

[53]  Xin Yao,et al.  Evolving artificial neural networks , 1999, Proc. IEEE.

[54]  B. V. Babu,et al.  Performance of modified differential evolution for optimal design of complex and non-linear chemical processes , 2006, J. Exp. Theor. Artif. Intell..

[55]  Mustafa E. Abdual-Salam,et al.  Comparative study between Differential Evolution and Particle Swarm Optimization algorithms in training of feed-forward neural network for stock price prediction , 2010, 2010 The 7th International Conference on Informatics and Systems (INFOS).

[56]  Silvia Curteanu,et al.  Comparison between different methods for developing neural network topology applied to a complex polymerization process , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[57]  Jouni Lampinen,et al.  A Trigonometric Mutation Operation to Differential Evolution , 2003, J. Glob. Optim..

[58]  M.M.A. Salama,et al.  Opposition-Based Differential Evolution , 2008, IEEE Transactions on Evolutionary Computation.

[59]  Janez Brest,et al.  Performance comparison of self-adaptive and adaptive differential evolution algorithms , 2007, Soft Comput..

[60]  Yi Liu,et al.  Adaptive Control of Nonlinear Time-Varying Processes Using Selective Recursive Kernel Learning Method , 2011 .

[61]  Uğur Yüzgeç,et al.  Performance comparison of differential evolution techniques on optimization of feeding profile for an industrial scale baker's yeast fermentation process. , 2010, ISA transactions.

[62]  Sigurdur Olafsson,et al.  Chapter 21 Metaheuristics , 2006, Simulation.

[63]  Ray Tsaih,et al.  Sensitivity analysis, neural networks, and the finance , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[64]  Andrew Hunter,et al.  Application of neural networks and sensitivity analysis to improved prediction of trauma survival , 2000, Comput. Methods Programs Biomed..

[65]  Teresa Bernarda Ludermir,et al.  Optimization of Neural Networks Weights and Architecture: A Multimodal Methodology , 2009, 2009 Ninth International Conference on Intelligent Systems Design and Applications.

[66]  D. Himmelblau Accounts of Experiences in the Application of Artificial Neural Networks in Chemical Engineering , 2008 .

[67]  Shahryar Rahnamayan,et al.  Opposition-Based Differential Evolution (ODE) with Variable Jumping Rate , 2007, 2007 IEEE Symposium on Foundations of Computational Intelligence.

[68]  Chaohua Dai,et al.  Dynamic multi-group self-adaptive differential evolution algorithm for reactive power optimization , 2010 .

[69]  Bidyadhar Subudhi,et al.  Differential Evolution and Levenberg Marquardt Trained Neural Network Scheme for Nonlinear System Identification , 2008, Neural Processing Letters.

[70]  George L. Nemhauser,et al.  Handbooks in operations research and management science , 1989 .

[71]  D. Cașcaval,et al.  Prediction of oxygen mass transfer coefficients in stirred bioreactors for bacteria, yeasts and fungus broths , 2004 .

[72]  Godfrey C. Onwubolu,et al.  Differential Evolution: A Handbook for Global Permutation-Based Combinatorial Optimization , 2009 .

[73]  Anyong Qing Differential Evolution: Fundamentals and Applications in Electrical Engineering , 2009 .