Selected Problems of Artificial Neural Networks Development

The chapter discusses selected problems of applications of Standard (deterministic) Neural Networks (SNN) but the main attention is focused on Bayesian Neural Networks (BNNs). In Sections 2 and 3 the problems of regression analysis, over-fitting and regularization are discussed basing on two types of network, i.e. Feed-forward Layered Neural Network (FLNN) and Radial Basis Function NN (RBFN). Application of Principal Component Analysis (PCA) is discussed as a method for reduction of input space dimensionality. In Section 4 the application of Kalman filtering to learning of SNNs is presented. Section 5 is devoted to discussion of some basics related to Bayesian inference. Then Maximum Likelihood (ML) and Maximum A Posterior (MAP) methods are presented as a basis for formulation of networks SNN-ML and SNN-MAP. A more general Bayesian framework corresponding to formulation of simple, semi-probabilistic network S-BNN, true probabilistic T-BNN and Gaussian Process GP-BNN is discussed. Section 6 is devoted to the analysis of four study cases, related mostly to the analysis of structural engineering and material mechanics problems.

[1]  S. Haykin.  Kalman Filtering and Neural Networks , 2001 .

[2]  Ronald A. Perez,et al.  A new system identification technique using Kaiman filtering and multilayer neural networks , 1996, Artif. Intell. Eng..

[3]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[4]  D. Mackay,et al.  Introduction to Gaussian processes , 1998 .

[5]  B. Schrefler,et al.  One-dimensional model of cable-in-conduit superconductors under cyclic loading using artificial neural networks , 2002 .

[6]  Zenon Waszczyszyn,et al.  Bayesian neural networks for prediction of response spectra , 2006 .

[7]  Jouko Lampinen,et al.  Bayesian approach for neural networks--review and case studies , 2001, Neural Networks.

[8]  Wray L. Buntine,et al.  Bayesian Back-Propagation , 1991, Complex Syst..

[9]  Artur Dubrawski,et al.  HPC Strength Prediction Using Artificial Neural Network , 1995 .

[10]  Timothy Masters,et al.  Practical neural network recipes in C , 1993 .

[11]  Raúl Rojas,et al.  Neural Networks - A Systematic Introduction , 1996 .

[12]  David J. C. MacKay,et al.  Information Theory, Inference, and Learning Algorithms , 2004, IEEE Transactions on Information Theory.

[13]  Christopher M. Bishop,et al.  Bayesian Regression and Classification , 2003 .

[14]  Zenon Waszczyszyn,et al.  Neural Networks for the Simulation and Identification Analysis of Buildings Subjected to Paraseismic Excitations , 2007 .

[15]  Zenon Waszczyszyn.  Artificial neural networks in civil and structural engineering: Ten years of research in Poland , 2006 .

[16]  Zenon Waszczyszyn,et al.  Kalman filtering for neural prediction of response spectra from mining tremors , 2007 .

[17]  Leszek Rutkowski,et al.  Neural Networks and Soft Computing , 2003 .

[18]  Zenon Waszczyszyn,et al.  Neural Networks in the Analysis and Design of Structures , 1999, CISM International Centre for Mechanical Sciences.

[19]  Duc Truong Pham,et al.  Neural Networks for Identification, Prediction and Control , 1995 .

[20]  Ł. Kaczmarczyk,et al.  Identification of characteristic length of microstructure for second order continuum multiscale model by Bayesian neural networks , 2007 .

[21]  Marek Słoński.  Prediction of concrete fatigue durability using Bayesian neural networks , 2005 .

[22]  Christopher M. Bishop,et al.  Neural networks for pattern recognition , 1995 .

[23]  V Varvara Kouznetsova,et al.  Computational homogenization for the multi-scale analysis of multi-phase materials , 2002 .

[24]  A. Krok.  Simulation of hysteresis loops for a superconductor using neural networks with Kalman filtering , 2006 .

[25]  Marek Słoński.  Bayesian regression approaches on example of concrete fatigue failure prediction , 2006 .

[26]  Marek Słoński.  HPC strength prediction using Bayesian neural networks , 2007 .

[27]  Krystyna Kuźniar.  BP Neural Network Computation of Response Spectra Using a Subpicture Idea , 2003 .

[28]  David J. C. MacKay,et al.  Bayesian Interpolation , 1992, Neural Computation.

[29]  Ian T. Nabney,et al.  Netlab: Algorithms for Pattern Recognition , 2002 .

[30]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[31]  Leonard Ziemiański,et al.  Neural Networks in the Identification Analysis of Structural Mechanics Problems , 2005 .

[32]  Michael E. Tipping.  Sparse Bayesian Learning and the Relevance Vector Machine , 2001, J. Mach. Learn. Res..

[33]  Martin T. Hagan,et al.  Gauss-Newton approximation to Bayesian learning , 1997, Proceedings of International Conference on Neural Networks (ICNN'97).

[34]  Michael E. Tipping.  Bayesian Inference: An Introduction to Principles and Practice in Machine Learning , 2003, Advanced Lectures on Machine Learning.

[35]  Janet M. Twomey,et al.  Validation and Verification , 1997 .

[36]  Georgios E. Stavroulakis,et al.  Parameter identification of materials and structures , 2005 .

[37]  Krystyna Kuźniar,et al.  Computation of natural fundamental periods of vibrations of medium-height prefabricated buildings by neural networks , 2000 .