Noise detection in the meta-learning level

The presence of noise in real data sets can harm the predictive performance of machine learning algorithms. There are several noise filtering techniques whose goal is to improve the quality of the data in classification tasks. These techniques usually scan the data for noise identification in a preprocessing step. Nonetheless, this is a non-trivial task and some noisy data can remain unidentified, while safe data can also be removed. The bias of each filtering technique influences its performance on a particular data set. Therefore, there is no single technique that can be considered the best for all domains or data distribution and choosing a particular filter is not straightforward. Meta-learning has been largely used in the last years to support the recommendation of the most suitable machine learning algorithm(s) for a new data set. This paper presents a meta-learning recommendation system able to predict the expected performance of noise filters in noisy data identification tasks. For such, a meta-base is created, containing meta-features extracted from several corrupted data sets along with the performance of some noise filters when applied to these data sets. Next, regression models are induced from this meta-base to predict the expected performance of the investigated filters in the identification of noisy data. The experimental results show that meta-learning can provide a good recommendation of the most promising filters to be applied to new classification data sets.

[1]  Anneleen Van Assche,et al.  Ensemble Methods for Noise Elimination in Classification Problems , 2003, Multiple Classifier Systems.

[2]  André Carlos Ponce de Leon Ferreira de Carvalho,et al.  MetaStream: A meta-learning based method for periodic algorithm selection in time-changing data , 2014, Neurocomputing.

[3]  Hilan Bensusan,et al.  Meta-Learning by Landmarking Various Learning Algorithms , 2000, ICML.

[4]  André Carlos Ponce de Leon Ferreira de Carvalho,et al.  A Study on Class Noise Detection and Elimination , 2012, 2012 Brazilian Symposium on Neural Networks.

[5]  André Carlos Ponce de Leon Ferreira de Carvalho,et al.  Empirical Evaluation of Ranking Prediction Methods for Gene Expression Data Classification , 2010, IBERAMIA.

[6]  Matthias Reif A Comprehensive Dataset for Evaluating Approaches of Various Meta-learning Tasks , 2012, ICPRAM.

[7]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[8]  Nada Lavrac,et al.  Advances in Class Noise Detection , 2010, ECAI.

[9]  Jesús Alcalá-Fdez,et al.  KEEL Data-Mining Software Tool: Data Set Repository, Integration of Algorithms and Experimental Analysis Framework , 2011, J. Multiple Valued Log. Soft Comput..

[10]  Ricardo Vilalta,et al.  Introduction to the Special Issue on Meta-Learning , 2004, Machine Learning.

[11]  Ronaldo C. Prati,et al.  Complex Network Measures for Data Set Characterization , 2013, 2013 Brazilian Conference on Intelligent Systems.

[12]  Felix Naumann,et al.  Data fusion , 2009, CSUR.

[13]  WuXindong,et al.  Class noise vs. attribute noise , 2004 .

[14]  M. Verleysen,et al.  Classification in the Presence of Label Noise: A Survey , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[15]  J. Ross Quinlan,et al.  Induction of Decision Trees , 1986, Machine Learning.

[16]  Kate Smith-Miles,et al.  Cross-disciplinary perspectives on meta-learning for algorithm selection , 2009, CSUR.

[17]  Xingquan Zhu,et al.  Class Noise vs. Attribute Noise: A Quantitative Study , 2003, Artificial Intelligence Review.

[18]  Peter A. Flach,et al.  Improved Dataset Characterisation for Meta-learning , 2002, Discovery Science.

[19]  Hilan Bensusan,et al.  A Higher-order Approach to Meta-learning , 2000, ILP Work-in-progress reports.

[20]  Tony R. Martinez,et al.  Reduction Techniques for Instance-Based Learning Algorithms , 2000, Machine Learning.

[21]  Dennis L. Wilson,et al.  Asymptotic Properties of Nearest Neighbor Rules Using Edited Data , 1972, IEEE Trans. Syst. Man Cybern..

[22]  Ethem Alpaydin,et al.  Introduction to machine learning , 2004, Adaptive computation and machine learning.

[23]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[24]  André Carlos Ponce de Leon Ferreira de Carvalho,et al.  Meta-Learning for Periodic Algorithm Selection in Time-Changing Data , 2012, 2012 Brazilian Symposium on Neural Networks.

[25]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[26]  David J. Spiegelhalter,et al.  Machine Learning, Neural and Statistical Classification , 2009 .

[27]  Guillermo Ricardo Simari,et al.  Advances in Artificial Intelligence – IBERAMIA 2010 , 2010, Lecture Notes in Computer Science.

[28]  Nada Lavrac,et al.  Ensemble-based noise detection: noise ranking and visual performance evaluation , 2012, Data Mining and Knowledge Discovery.

[29]  Ron Kohavi,et al.  Wrappers for Feature Subset Selection , 1997, Artif. Intell..

[30]  Francisco Herrera,et al.  Predicting noise filtering efficacy with data complexity measures for nearest neighbor classification , 2013, Pattern Recognit..

[31]  Carlos Soares,et al.  Sampling-Based Relative Landmarks: Systematically Test-Driving Algorithms Before Choosing , 2001, EPIA.

[32]  Giovanna Castellano,et al.  Meta-data: Characterization of Input Features for Meta-learning , 2005, MDAI.

[33]  Tin Kam Ho,et al.  Complexity Measures of Supervised Classification Problems , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[34]  Ricardo Vilalta,et al.  Metalearning - Applications to Data Mining , 2008, Cognitive Technologies.

[35]  Choh-Man Teng,et al.  Correcting Noisy Data , 1999, ICML.

[36]  André Carlos Ponce de Leon Ferreira de Carvalho,et al.  Selection of algorithms to solve traveling salesman problems using meta-learning , 2011, Int. J. Hybrid Intell. Syst..

[37]  Xindong Wu,et al.  Eliminating Class Noise in Large Datasets , 2003, ICML.

[38]  Carla E. Brodley,et al.  Identifying and Eliminating Mislabeled Training Instances , 1996, AAAI/IAAI, Vol. 1.

[39]  Carlos Soares,et al.  A Meta-Learning Method to Select the Kernel Width in Support Vector Regression , 2004, Machine Learning.

[40]  Nada Lavrac,et al.  Experiments with Noise Filtering in a Medical Domain , 1999, ICML.