论文信息 - Combining Estimators Using Non-Constant Weighting Functions

Combining Estimators Using Non-Constant Weighting Functions

This paper discusses the linearly weighted combination of estimators in which the weighting functions are dependent on the input. We show that the weighting functions can be derived either by evaluating the input dependent variance of each estimator or by estimating how likely it is that a given estimator has seen data in the region of the input space close to the input pattern. The latter solution is closely related to the mixture of experts approach and we show how learning rules for the mixture of experts can be derived from the theory about learning with missing features. The presented approaches are modular since the weighting functions can easily be modified (no retraining) if more estimators are added. Furthermore, it is easy to incorporate estimators which were not derived from data such as expert systems or algorithms.

Volker Tresp | Michiaki Taniguchi | Volker Tresp | Michiaki Taniguchi

[1] Ronny Meir,et al. Bias, variance and the combination of estimators; The case of linear least squares , 1995 .

[2] Volker Tresp,et al. Some Solutions to the Missing Feature Problem in Vision , 1992, NIPS.

[3] S. Hanson,et al. Some Solutions to the Missing Feature Problem in Vision , 1993 .

[4] Geoffrey E. Hinton,et al. Adaptive Mixtures of Local Experts , 1991, Neural Computation.

[5] Robert Tibshirani,et al. A Comparison of Some Error Estimates for Neural Network Models , 1996, Neural Computation.

[6] Terrence J. Sejnowski,et al. Filter selection model for motion segmentation and velocity integration , 1994 .

[7] Volker Tresp,et al. Network Structuring and Training Using Rule-Based Knowledge , 1992, NIPS.

[8] Volker Tresp,et al. Training Neural Networks with Deficient Data , 1993, NIPS.

[9] M. Perrone. Improving regression estimation: Averaging methods for variance reduction with extensions to general convex measure optimization , 1993 .