Process modeling using stacked neural networks

A new technique for neural-network-based modeling of chemical processes is proposed. Stacked neural networks allow multiple neural networks to be selected and used to model a given process. The idea is that improved predictions can be obtained using multiple networks, instead of simply selecting a single, hopefully optimal network, as is usually done. A methodology for stacking neural networks for plant-process modeling has been developed. This method is inspired by the technique of stacked generalization proposed by Wolpert. The proposed method has been applied and evaluated for three example problems, including the dynamic modeling of a nonlinear chemical process. Results obtained demonstrate the promise of this approach for improved neural-network-based plant-process modeling.

[1]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[2]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[3]  Eric B. Bartlett,et al.  Self determination of input variable importance using neural networks , 1994, Neural Parallel Sci. Comput..

[4]  Donald F. Specht,et al.  A general regression neural network , 1991, IEEE Trans. Neural Networks.

[5]  渡辺 慧,et al.  Knowing and guessing : a quantitative study of inference and information , 1969 .

[6]  Lyle H. Ungar,et al.  A comparison of two nonparametric estimation schemes: MARS and neural networks , 1993 .

[7]  Sholom M. Weiss,et al.  Computer Systems That Learn , 1990 .

[8]  W. Press,et al.  Numerical Recipes: The Art of Scientific Computing , 1987 .

[9]  L. Cooper,et al.  When Networks Disagree: Ensemble Methods for Hybrid Neural Networks , 1992 .

[10]  Alan J. Miller.  Subset Selection in Regression , 1992 .

[11]  N. V. Bhat,et al.  Use of neural nets for dynamic modeling and control of chemical process systems , 1990 .

[12]  Ka-Yiu San,et al.  Process identification using neural networks , 1992 .

[13]  Josiah C. Hoskins,et al.  Artificial neural network models for knowledge representation in chemical engineering , 1990 .

[14]  Mark A. Kramer,et al.  Autoassociative neural networks , 1992 .

[15]  Paul J. Werbos,et al.  The roots of backpropagation , 1994 .

[16]  M. Morari,et al.  Internal model control. VI: Extension to nonlinear systems , 1986 .

[17]  B. Ydstie.  Forecasting and control using adaptive connectionist networks , 1990 .

[18]  D. C. Psichogios,et al.  Direct and indirect model based control using artificial neural networks , 1991 .

[19]  N. V. Bhat,et al.  determining model structure for neural models by network stripping , 1992 .

[20]  Anujit Basu,et al.  Nuclear plant diagnostics using neural networks with dynamic input selection , 1995 .

[21]  Hari K. Iyer,et al.  Model Discrimination for Nonlinear Regression Models , 1990 .

[22]  Robert L. Winkler,et al.  Combining forecasts: A philosophical basis and some current issues , 1989 .

[23]  Thomas E. Marlin,et al.  Multivariate statistical monitoring of process operating performance , 1991 .

[24]  Lyle H. Ungar,et al.  Neural network forecasting of short, noisy time series , 1992 .

[25]  Babu Joseph,et al.  Exploratory data analysis: A comparison of statistical methods with artificial neural networks , 1992 .

[26]  Sherif Hashem,et al.  Optimal Linear Combinations of Neural Networks , 1997, Neural Networks.

[27]  Ferdinand Hergert,et al.  Improving model selection by nonconvergent methods , 1993, Neural Networks.

[28]  Carl G. Looney,et al.  Pattern recognition using neural networks , 1997 .

[29]  J. Freidman,et al.  Multivariate adaptive regression splines , 1991 .

[30]  Robert B. Ash,et al.  Information Theory , 2020, The SAGE International Encyclopedia of Mass Media and Society.

[31]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[32]  Francis X. Diebold,et al.  Forecast combination and encompassing: Reconciling two divergent literatures , 1989 .

[33]  Eric B. Bartlett,et al.  Detecting faults in a nuclear power plant by using dynamic node architecture artificial neural networks , 1994 .

[34]  Lars Kai Hansen,et al.  Neural Network Ensembles , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[35]  Todd K. Leen,et al.  Hebbian feature discovery improves classifier efficiency , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[36]  Lyle H. Ungar,et al.  A NEURAL NETWORK ARCHITECTURE THAT COMPUTES ITS OWN RELIABILITY , 1992 .

[37]  T. McAvoy,et al.  Interpreting biosensor data via backpropagation , 1989, International 1989 Joint Conference on Neural Networks.

[38]  A. Atkinson.  Subset Selection in Regression , 1992 .

[39]  Christian Lebiere,et al.  The Cascade-Correlation Learning Architecture , 1989, NIPS.

[40]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1989, Math. Control. Signals Syst..

[41]  M. Kramer.  Nonlinear principal component analysis using autoassociative neural networks , 1991 .

[42]  Roberto Battiti,et al.  Using mutual information for selecting features in supervised neural net learning , 1994, IEEE Trans. Neural Networks.

[43]  Thomas J. Mc Avoy,et al.  Use of Neural Nets For Dynamic Modeling and Control of Chemical Process Systems , 1989, 1989 American Control Conference.

[44]  R. Clemen.  Combining forecasts: A review and annotated bibliography , 1989 .

[45]  Kumpati S. Narendra,et al.  Identification and control of dynamical systems using neural networks , 1990, IEEE Trans. Neural Networks.

[46]  Martin Fodslette Møller,et al.  A scaled conjugate gradient algorithm for fast supervised learning , 1993, Neural Networks.

[47]  D. M. Titterington,et al.  Neural Networks: A Review from a Statistical Perspective , 1994 .

[48]  Galina L. Rogova,et al.  Combining the results of several neural network classifiers , 1994, Neural Networks.

[49]  J. Stephen Judd,et al.  Neural network design and the complexity of learning , 1990, Neural network modeling and connectionism.

[50]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[51]  Manfred Morari,et al.  PLS/neural networks , 1992 .

[52]  Timothy Masters,et al.  Multilayer Feedforward Networks , 1993 .

[53]  J. M. Bates,et al.  The Combination of Forecasts , 1969 .

[54]  David H. Wolpert,et al.  Stacked generalization , 1992, Neural Networks.

[55]  Guy A. Dumont,et al.  Classification of acoustic emission signals via Hebbian feature extraction , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[56]  J. Mesirov,et al.  Hybrid system for protein secondary structure prediction. , 1992, Journal of molecular biology.

[57]  Eric B. Bartlett,et al.  Dynamic node architecture learning: An information theoretic approach , 1994, Neural Networks.

[58]  Philip D. Wasserman,et al.  Advanced methods in neural computing , 1993, VNR computer library.

[59]  H. White.  Some Asymptotic Results for Learning in Single Hidden-Layer Feedforward Network Models , 1989 .

[60]  R. Lippmann,et al.  An introduction to computing with neural nets , 1987, IEEE ASSP Magazine.

[61]  Keinosuke Fukunaga,et al.  Introduction to Statistical Pattern Recognition , 1972 .

[62]  Venkat Venkatasubramanian,et al.  A neural network methodology for process fault diagnosis , 1989 .