Optimal modularization of learning models in forecasting environmental variables

Data-driven models based on the methods of machine learning have proven to be accurate tools in predicting various natural phenomena. Their accuracy can be however increased if several learning models are combined in an ensemble or a committee. Modular model is a particular type of a committee machine and is comprised of a set of specialized (local) models each of which is responsible for a particular region of the input space, and may be trained on a subset of the training set. This paper presents a number of approaches to building modular models. An issue of including a domain expert into the modelling process is also discussed, and the new algorithms in the class of model trees (piece-wise linear modular regression models) are presented. Comparison of the algorithms based on modular local modelling to the more traditional “global” learning models shows their higher accuracy and the transparency of the resulting models.

[1]  Ian H. Witten,et al.  Induction of model trees for predicting continuous classes , 1996 .

[2]  Paola Campadelli,et al.  A Boosting Algorithm for Regression , 1997, ICANN.

[3]  Geoffrey E. Hinton,et al.  Adaptive Mixtures of Local Experts , 1991, Neural Computation.

[4]  Leo Breiman,et al.  Stacked regressions , 2004, Machine Learning.

[5]  D.P. Solomatine,et al.  AdaBoost.RT: a boosting algorithm for regression problems , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[6]  Dimitri P. Solomatine,et al.  FLEXIBLE AND OPTIMAL M5 MODEL TREES WITH APPLICATIONS TO FLOW PREDICTIONS , 2004 .

[7]  Durga L. Shrestha,et al.  Experiments with AdaBoost.RT, an Improved Boosting Scheme for Regression , 2006, Neural Computation.

[8]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[9]  Ludmila I. Kuncheva,et al.  Combining Pattern Classifiers: Methods and Algorithms , 2004 .

[10]  Christopher J. Merz,et al.  UCI Repository of Machine Learning Databases , 1996 .

[11]  Kristin P. Bennett,et al.  Global Tree Optimization: A Non-greedy Decision Tree Algorithm , 2007 .

[12]  Nikola K. Kasabov,et al.  DENFIS: dynamic evolving neural-fuzzy inference system and its application for time-series prediction , 2002, IEEE Trans. Fuzzy Syst..

[13]  J. R. Quinlan.  Learning With Continuous Classes , 1992 .

[14]  Ian H. Witten,et al.  Interactive machine learning: letting users build classifiers , 2002, Int. J. Hum. Comput. Stud..

[15]  J. Freidman,et al.  Multivariate adaptive regression splines , 1991 .

[16]  D.P. Solomatine,et al.  Semi-optimal hierarchical regression models and ANNs , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[17]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1997, EuroCOLT.

[18]  David H. Wolpert,et al.  Stacked generalization , 1992, Neural Networks.

[19]  Dimitri P. Solomatine,et al.  Modular learning models in forecasting natural phenomena , 2006, Neural Networks.

[20]  Ian Witten,et al.  Data Mining , 2000 .

[21]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .