Embedding monotonicity and concavity in the training of neural networks by means of genetic algorithms: Application to multiphase flow

Abstract It is well established that including monotonic constraints in neural network (NN) development, when such a priori information is available, increases the confidence in predictions and prevents over fitting. An elegant procedure to guarantee mathematically monotonic NNs with respect to some of their inputs is to force the signs of some of the network’s weights in such a way that the network remains monotonic in the entirety of the input domain. In this work, besides monotonicity, a second-order information, namely, the concavity, is used to guide a genetic algorithm—genetic hill climber optimizer to identify the weights of the neural network. Monotonicity and concavity are key conditions in establishing phenomenological correlations when the available data for training are insufficient to cover in depth the n -dimensional input space. In such instances, classical training procedures fail to unveil the main tendencies in data and may suffer local over fitting problems. In this work, the proof-of-concept of embedding monotonicity and concavity information in the training of NNs by means of genetic algorithms will be illustrated in correlating total liquid holdup in randomly packed bed containing counter-current gas–liquid towers.

[1]  Jacques Periaux,et al.  Genetic Algorithms in Engineering and Computer Science , 1996 .

[2]  Faïçal Larachi,et al.  Gas}liquid interfacial mass transfer in trickle-bed reactors: state-of-the-art correlations , 1999 .

[3]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[4]  L. Darrell Whitley,et al.  The GENITOR Algorithm and Selection Pressure: Why Rank-Based Allocation of Reproductive Trials is Best , 1989, ICGA.

[5]  Shouhong Wang,et al.  LEARNING MONOTONIC‐CONCAVE INTERVAL CONCEPTS USING THE BACK‐PROPAGATION NEURAL NETWORKS , 1996, Comput. Intell..

[6]  Yaser S. Abu-Mostafa,et al.  A Method for Learning From Hints , 1992, NIPS.

[7]  Thiemo Krink,et al.  The LifeCycle Model: Combining Particle Swarm Optimisation, Genetic Algorithms and HillClimbers , 2002, PPSN.

[8]  J. D. Schaffer,et al.  Combinations of genetic algorithms and neural networks: a survey of the state of the art , 1992, [Proceedings] COGANN-92: International Workshop on Combinations of Genetic Algorithms and Neural Networks.

[9]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[10]  L. Ungar,et al.  Deriving Monotonic Function Envelopes from Observations , 2003 .

[11]  Kenneth A. De Jong,et al.  A Cooperative Coevolutionary Approach to Function Optimization , 1994, PPSN.

[12]  Joseph Sill,et al.  Monotonic Networks , 1997, NIPS.

[13]  Faïçal Larachi,et al.  Reinforcing the phenomenological consistency in artificial neural network modeling of multiphase reactors , 2003 .

[14]  Philipp Slusallek,et al.  Introduction to real-time ray tracing , 2005, SIGGRAPH Courses.

[15]  Faïçal Larachi,et al.  Improved Liquid Hold-up Correlation for Randomly Packed Towers , 2001 .

[16]  Mohammad Jamialahmadi,et al.  Study of Bubble Formation Under Constant Flow Conditions , 2001 .

[17]  Bernard P. A. Grandjean,et al.  Integrated Genetic Algorithm−Artificial Neural Network Strategy for Modeling Important Multiphase-Flow Characteristics , 2002 .

[18]  R. Billet,et al.  Prediction of Mass Transfer Columns with Dumped and Arranged Packings , 1999 .

[19]  L. Darrell Whitley,et al.  Genetic Reinforcement Learning with Multilayer Neural Networks , 1991, ICGA.

[20]  W. M. Jenkins,et al.  Genetic Algorithms and Neural Networks , 1999, Neural Networks in the Analysis and Design of Structures.

[21]  L. Ungar,et al.  Estimating Monotonic Functions and Their Bounds , 1999 .

[22]  Faïçal Larachi,et al.  Improving the prediction of irrigated pressure drop in packed absorption towers , 2001 .

[23]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .