In this paper we describe a divide-and-combine strategy for decomposition of a complex prediction problem into simpler local sub-problems. We rstly show how to perform a soft decomposition via clustering of input data. Such decomposition leads to a partition of the input space into several regions which may overlap. Therefore, to each region is assigned a local predictor (or expert) which is trained only on local data. To construct a solution to the global prediction problem, we combine the local experts using two approaches: weighted averaging where the outputs of local experts are weighted by their prior densities, and nonlinear adaptive combination where the pooling parameters are obtained through minimization of a global error. To illustrate the validity of our approach, we show simulation results for two classiica-tion tasks, vowels and phonemes, using local experts which are Multi-Layer Perceptrons (MLP) and Support Vector Machines (SVM). We compare the results obtained using the two local combination modes with the results obtained using a global predictor and a linear combination of global predictors.
[1]
Martin A. Riedmiller,et al.
A direct adaptive method for faster backpropagation learning: the RPROP algorithm
,
1993,
IEEE International Conference on Neural Networks.
[2]
Duane DeSieno,et al.
Adding a conscience to competitive learning
,
1988,
IEEE 1988 International Conference on Neural Networks.
[3]
L. Cooper,et al.
When Networks Disagree: Ensemble Methods for Hybrid Neural Networks
,
1992
.
[4]
Vladimir Vapnik,et al.
The Nature of Statistical Learning
,
1995
.
[5]
M. Kramer,et al.
Embedding Theoretical Models in Neural Networks
,
1992,
American Control Conference.
[6]
Geoffrey E. Hinton,et al.
An Alternative Model for Mixtures of Experts
,
1994,
NIPS.
[7]
Roderick Murray-Smith,et al.
Multiple Model Approaches to Modelling and Control
,
1997
.
[8]
Vladimir N. Vapnik,et al.
The Nature of Statistical Learning Theory
,
2000,
Statistics for Engineering and Information Science.
[9]
Thorsten Joachims,et al.
Text categorization with support vector machines
,
1999
.