Hierarchical Estimator

In this paper, a new machine learning solution for function approximation is presented. It combines many simple and relatively inaccurate estimators to achieve high accuracy. It creates - in incremental manner - hierarchical, tree-like structure, adapting it to the specific problem being solved. As most variants use the errors of already constructed parts to direct further construction, it may be viewed as example of boosting - as understood in general sense. The influence of particular constituent estimator on the whole solution's output is not constant, but depends on the feature vector being evaluated. Provided in this paper are: general form of the metaalgorithm, a few specific, detailed solutions, theoretical basis and experimental results with one week power load prediction for country-wide power distribution grid and on simple test datasets.

[1]  Andrzej Bielecki,et al.  Neural Systems for Short-Term Forecasting of Electric Power Load , 2007, ICANNGA.

[2]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[3]  A. Asuncion,et al.  UCI Machine Learning Repository, University of California, Irvine, School of Information and Computer Sciences , 2007 .

[4]  Adam Roman,et al.  A New Notion of Weakness in Classification Theory , 2009, Computer Recognition Systems 3.

[5]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[6]  Leslie G. Valiant,et al.  Cryptographic Limitations on Learning Boolean Formulae and Finite Automata , 1993, Machine Learning: From Theory to Applications.

[7]  Eric T. Bax.  Validation of Voting Committees , 1998, Neural Computation.

[8]  Stelios C. A. Thomopoulos,et al.  Dignet: an unsupervised-learning clustering algorithm for clustering and data fusion , 1995 .

[9]  Marios M. Polycarpou,et al.  Short Term Electric Load Forecasting: A Tutorial , 2007, Trends in Neural Computation.

[10]  James C. Bezdek,et al.  On cluster validity for the fuzzy c-means model , 1995, IEEE Trans. Fuzzy Syst..

[11]  Robert E. Schapire,et al.  The strength of weak learnability , 1990, Mach. Learn..

[12]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[13]  Günther Eibl,et al.  Multiclass Boosting for Weak Classifiers , 2005, J. Mach. Learn. Res..

[14]  Jenq-Neng Hwang,et al.  Handbook of Neural Network Signal Processing , 2000, IEEE Transactions on Neural Networks.

[15]  Josef Kittler,et al.  Strategies for combining classifiers employing shared and distinct pattern representations , 1997, Pattern Recognit. Lett..

[16]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1997, EuroCOLT.

[17]  S. Keerthi,et al.  Improvements to SMO Algorithm for SVM Regression 1 , 1999 .

[18]  Armin Iske,et al.  Hierarchical Nonlinear Approximation for Experimental Design and Statistical Data Fitting , 2007, SIAM J. Sci. Comput..

[19]  J. Friedman.  Greedy function approximation: A gradient boosting machine. , 2001 .

[20]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[21]  J. Ross Quinlan,et al.  Combining Instance-Based and Model-Based Learning , 1993, ICML.

[22]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[23]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[24]  Silja Meyer-Nieberg,et al.  Electric load forecasting methods: Tools for decision making , 2009, Eur. J. Oper. Res..

[25]  Peter L. Bartlett,et al.  AdaBoost is Consistent , 2006, J. Mach. Learn. Res..

[26]  Igor T. Podolak.  Hierarchical classifier with overlapping class groups , 2008, Expert Syst. Appl..

[27]  K. Saito,et al.  A constructive learning algorithm for an HME , 1996, Proceedings of International Conference on Neural Networks (ICNN'96).