A Local Linear Wavelet Neural Network Based on a Bayesian Design Method

In general, wavelet neural networks have a problem on the curse of dimensionality, i.e. hidden units to be required are exponentially increased with a high input dimension. To solve the above problem, the wavelet neural network incorporating local linear model is proposed. On the network design, however, the number of hidden units are determined by trial and error. In the present paper, a design method based on Bayesian method is proposed for the local linear wavelet neural network. Through computer simulation, the performance of the proposed method is evaluated.

[1]  Liangyue Cao,et al.  Predicting chaotic time series with wavelet networks , 1995 .

[2]  Yagyensh C. Pati,et al.  Analysis and synthesis of feedforward neural networks using discrete affine wavelet transformations , 1993, IEEE Trans. Neural Networks.

[3]  年男 坂田.  伊庭幸人・種村正美・大森裕浩・和合 肇・佐藤整尚・高橋明彦, 計算統計II, 岩波書店, 2005年 , 2006 .

[4]  Florian Metze,et al.  Generalized radial basis function networks for classification and novelty detection: self-organization of optimal Bayesian decision , 2000, Neural Networks.

[5]  Ting Wang,et al.  A Local Linear Adaptive Wavelet Neural Network , 2002 .

[6]  Jiwen Dong,et al.  Time-series prediction using a local linear wavelet neural network , 2006, Neurocomputing.

[7]  F. Girosi,et al.  Networks for approximation and learning , 1990, Proc. IEEE.

[8]  R. L. Peskin,et al.  Wavelet neural networks and receptive field partitioning , 1993, IEEE International Conference on Neural Networks.

[9]  Hagai Attias,et al.  Inferring Parameters and Structure of Latent Variable Models by Variational Bayes , 1999, UAI.

[10]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[11]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[12]  C. C. Homes,et al.  Bayesian Radial Basis Functions of Variable Dimension , 1998, Neural Computation.

[13]  D. B. Fogel,et al.  Evolving neural networks , 1990, Biological Cybernetics.

[14]  Kwok-Wo Wong,et al.  On-line Successive Synthesis of Wavelet Networks , 1998, Neural Processing Letters.

[15]  Qinghua Zhang,et al.  Wavelet networks , 1992, IEEE Trans. Neural Networks.

[16]  Ken-ichi Funahashi,et al.  On the approximate realization of continuous mappings by neural networks , 1989, Neural Networks.

[17]  Jiwen Dong,et al.  A local linear wavelet neural network , 2004, Fifth World Congress on Intelligent Control and Automation (IEEE Cat. No.04EX788).

[18]  Y. C. Pati,et al.  Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition , 1993, Proceedings of 27th Asilomar Conference on Signals, Systems and Computers.

[19]  Nando de Freitas,et al.  Robust Full Bayesian Learning for Radial Basis Networks , 2001, Neural Computation.

[20]  P. Green.  Reversible jump Markov chain Monte Carlo computation and Bayesian model determination , 1995 .

[21]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[22]  Geoffrey E. Hinton,et al.  Adaptive Mixtures of Local Experts , 1991, Neural Computation.

[23]  Ting Wang,et al.  The Local Linear Adaptive Wavelet Neural Network with Hybrid EP/Gradient Algorithm and Its Application to Nonlinear Dynamic System Identification , 2002 .

[24]  David Allingham,et al.  Wavelet reconstruction of nonlinear dynamics , 1998 .

[25]  David J. C. MacKay,et al.  Information Theory, Inference, and Learning Algorithms , 2004, IEEE Transactions on Information Theory.

[26]  Gaetan Libert,et al.  Combining pattern recognition techniques with Akaike's information criteria for identifying ARMA models , 1994, IEEE Trans. Signal Process..

[27]  S. S. Rao,et al.  Recurrent wavelet networks , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[28]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[29]  Ronald J. Williams,et al.  A Learning Algorithm for Continually Running Fully Recurrent Neural Networks , 1989, Neural Computation.

[30]  B. Irie,et al.  Capabilities of three-layered perceptrons , 1988, IEEE 1988 International Conference on Neural Networks.

[31]  D. D. Bruns,et al.  WaveARX neural network development for system identification using a systematic design synthesis , 1995 .

[32]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .