Sequential optimal experiment design for neural networks using multiple linearization

Design of an optimal input signal in system identification using a multi-layer perceptron network is treated. Neural networks of the same structure differing only in parameter values are able to approximate various nonlinear mappings. To ensure high quality of network parameter estimates, it is crucial to find a suitable input signal. It is shown that utilizing the conditional probability density function of parameters for design of the input signal provides better results than currently used procedures based on parameter point estimates only. The conditional probability density function of parameters is unknown and hence it is estimated using the Gaussian sum approach approximating arbitrary probability density function by a sum of normal distributions. This approach is less computationally demanding than the Markov Chain Monte Carlo method and achieves better results in comparison with the commonly used local prediction error methods. The properties of the proposed input signal designs are illustrated in numerical examples.

[1]  Gerhard Paass,et al.  Bayesian Query Construction for Neural Network Models , 1994, NIPS.

[2]  Bo Wahlberg,et al.  ON OPTIMAL INPUT DESIGN IN SYSTEM IDENTIFICATION1 , 2006 .

[3]  Visakan Kadirkamanathan,et al.  Functional Adaptive Control: An Intelligent Systems Approach , 2012 .

[4]  Kenji Fukumizu,et al.  Statistical active learning in multilayer perceptrons , 2000, IEEE Trans. Neural Networks Learn. Syst..

[5]  P. Laycock,et al.  Optimum Experimental Designs , 1995 .

[6]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1989, Math. Control. Signals Syst..

[7]  Anthony C. Atkinson,et al.  Optimum Experimental Designs , 1992 .

[8]  David A. Cohn,et al.  Neural Network Exploration Using Optimal Experiment Design , 1993, NIPS.

[9]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1992, Math. Control. Signals Syst..

[10]  Miroslav Šimandl,et al.  Filtering, Prediction and Smoothing with Gaussian Sum Representation , 2000 .

[11]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[12]  I-Cheng Yeh,et al.  Modeling of strength of high-performance concrete using artificial neural networks , 1998 .

[13]  Inchi Hu,et al.  On sequential designs in nonlinear problems , 1998 .

[14]  David J. C. MacKay,et al.  Information-Based Objective Functions for Active Data Selection , 1992, Neural Computation.

[15]  Visakan Kadirkamanathan,et al.  Functional Adaptive Control , 2001 .

[16]  Håkan Hjalmarsson,et al.  From experiment design to closed-loop control , 2005, Autom..

[17]  Eric Walter,et al.  Identification of Parametric Models: from Experimental Data , 1997 .

[18]  Gérard Dreyfus,et al.  Towards the Optimal Design of Numerical Experiments , 2008, IEEE Transactions on Neural Networks.

[19]  Luc Pronzato,et al.  Optimal experimental design and some related control problems , 2008, Autom..

[20]  Marcin Witczak,et al.  Toward the training of feed-forward neural networks with the D-optimum input sequence , 2006, IEEE Transactions on Neural Networks.

[21]  Miroslav Ŝimandl,et al.  Optimal input and decision in multiple model fault detection , 2005 .

[22]  H. Sorenson,et al.  Recursive bayesian estimation using gaussian sums , 1971 .

[23]  Eric Walter,et al.  Identifiability of parametric models , 1987 .