Recursive Estimation of Dynamic Modular RBF Networks

In this paper, recursive estimation algorithms for dynamic modular networks are developed. The models are based on Gaussian RBF networks and the gating network is considered in two stages: At first, it is simply a time-varying scalar and in the second, it is based on the state, as in the mixture of local experts scheme. The resulting algorithm uses Kalman filter estimation for the model estimation and the gating probability estimation. Both, 'hard' and 'soft' competition based estimation schemes are developed where in the former, the most probable network is adapted and in the latter all networks are adapted by appropriate weighting of the data.

[1]  David J. C. MacKay,et al.  Bayesian Interpolation , 1992, Neural Computation.

[2]  Visakan Kadirkamanathan,et al.  Recursive nonlinear identification using multiple model algorithm , 1995, Proceedings of 1995 IEEE Workshop on Neural Networks for Signal Processing.

[3]  Steven J. Nowlan,et al.  Mixtures of Controllers for Jump Linear and Non-Linear Plants , 1993, NIPS.

[4]  Geoffrey E. Hinton,et al.  Adaptive Mixtures of Local Experts , 1991, Neural Computation.

[5]  Yoshua Bengio,et al.  An Input Output HMM Architecture , 1994, NIPS.

[6]  Y. Bar-Shalom.  Tracking and data association , 1988 .

[7]  S. Srihari.  Mixture Density Networks , 1994 .

[8]  Visakan Kadirkamanathan.  A statistical inference based growth criterion for the RBF network , 1994, Proceedings of IEEE Workshop on Neural Networks for Signal Processing.

[9]  Robert A. Jacobs,et al.  Hierarchical Mixtures of Experts and the EM Algorithm , 1993, Neural Computation.