Local voting of weak classifiers

Many data mining problems involve an investigation of relationships between features in heterogeneous datasets, where different learning algorithms can be more appropriate for different regions. We propose herein a technique of localized voting of weak classifiers. This technique identifies local regions which have similar characteristics and then uses the votes of each local expert to describe the relationship between the data characteristics and the target class. We performed a comparison with other well known combining methods on standard benchmark datasets and the accuracy of the proposed method was greater.

[1]  Eugene M. Kleinberg.  A Mathematically Rigorous Foundation for Supervised Learning , 2000, Multiple Classifier Systems.

[2]  Pat Langley,et al.  Induction of One-Level Decision Trees , 1992, ML.

[3]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[4]  Louis Vuurpijl,et al.  An overview and comparison of voting methods for pattern recognition , 2002, Proceedings Eighth International Workshop on Frontiers in Handwriting Recognition.

[5]  Andrew W. Moore,et al.  Locally Weighted Learning , 1997, Artificial Intelligence Review.

[6]  Zoran Obradovic,et al.  Adaptive Boosting for Spatial Functions with Unstable Driving Attributes , 2000, PAKDD.

[7]  Sally Jo Cunningham,et al.  MetaData for Database Mining , 1996 .

[8]  Cullen Schaffer,et al.  Selecting a classification method by cross-validation , 1993, Machine Learning.

[9]  Christopher J. Merz,et al.  UCI Repository of Machine Learning Databases , 1996 .

[10]  John G. Cleary,et al.  K*: An Instance-based Learner Using and Entropic Distance Measure , 1995, ICML.

[11]  Yoshua Bengio,et al.  Inference for the Generalization Error , 1999, Machine Learning.

[12]  Léon Bottou,et al.  Local Learning Algorithms , 1992, Neural Computation.

[13]  Tony R. Martinez,et al.  Reduction Techniques for Instance-Based Learning Algorithms , 2000, Machine Learning.

[14]  Pedro M. Domingos,et al.  On the Optimality of the Simple Bayesian Classifier under Zero-One Loss , 1997, Machine Learning.

[15]  Ian H. Witten,et al.  Issues in Stacked Generalization , 2011, J. Artif. Intell. Res..

[16]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[17]  Raymond J. Mooney,et al.  Constructing Diverse Classifier Ensembles using Artificial Training Examples , 2003, IJCAI.

[18]  Andrew W. Moore,et al.  Locally Weighted Learning for Control , 1997, Artificial Intelligence Review.

[19]  David H. Wolpert,et al.  Stacked generalization , 1992, Neural Networks.

[20]  Cullen Schaffer,et al.  Technical Note: Selecting a Classification Method by Cross-Validation , 1993, Machine Learning.

[21]  David W. Aha,et al.  Lazy Learning , 1997, Springer Netherlands.

[22]  Geoffrey I. Webb,et al.  MultiBoosting: A Technique for Combining Boosting and Wagging , 2000, Machine Learning.

[23]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[24]  Nathan Intrator,et al.  Automatic model selection in a hybrid perceptron/radial network , 2002, Inf. Fusion.

[25]  Yoav Freund,et al.  Experiments with a New Boosting Algorithm , 1996, ICML.

[26]  Robert C. Holte,et al.  Very Simple Classification Rules Perform Well on Most Commonly Used Datasets , 1993, Machine Learning.

[27]  Sotiris B. Kotsiantis,et al.  Preventing Student Dropout in Distance Learning Using Machine Learning Techniques , 2003, KES.

[28]  Jiri Matas,et al.  On Combining Classifiers , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  Bernhard Pfahringer,et al.  Locally Weighted Naive Bayes , 2002, UAI.

[30]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[31]  Eric Bauer,et al.  An Empirical Comparison of Voting Classification Algorithms: Bagging, Boosting, and Variants , 1999, Machine Learning.

[32]  Chris Mellish,et al.  Advances in Instance Selection for Instance-Based Learning Algorithms , 2002, Data Mining and Knowledge Discovery.

[33]  Ian Witten,et al.  Data Mining , 2000 .

[34]  Thomas G. Dietterich.  Multiple Classifier Systems , 2000, Lecture Notes in Computer Science.

[35]  Alexander K. Seewald,et al.  How to Make Stacking Better and Faster While Also Taking Care of an Unknown Weakness , 2002, International Conference on Machine Learning.

[36]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques with Java implementations , 2002, SGMD.

[37]  Chuanyi Ji,et al.  Combinations of Weak Classifiers , 1996, NIPS.

[38]  Thomas G. Dietterich,et al.  Solving Multiclass Learning Problems via Error-Correcting Output Codes , 1994, J. Artif. Intell. Res..