Selecting Machine Learning Algorithms Using the Ranking Meta-Learning Approach

In this work, we present the use of Ranking Meta-Learning approaches to ranking and selecting algorithms for problems of time series forecasting and clustering of gene expression data. Given a problem (forecasting or clustering), the Meta-Learning approach provides a ranking of the candidate algorithms, according to the characteristics of the problem’s dataset. The best ranked algorithm can be returned as the selected one. In order to evaluate the Ranking Meta-Learning proposal, prototypes were implemented to rank artificial neural networks models for forecasting financial and economic time series and to rank clustering algorithms in the context of cancer gene expression microarray datasets. The case studies regard experiments to measure the correlation between the suggested rankings of algorithms and the ideal rankings. The results revealed that Meta-Learning was able to suggest more adequate rankings in both domains of application considered.

[1]  Carlos Soares,et al.  Zoomed Ranking: Selection of Classification Algorithms Based on Relevant Performance Information , 2000, PKDD.

[2]  Pat Langley,et al.  Editorial: On Machine Learning , 1986, Machine Learning.

[3]  David W. Aha,et al.  Generalizing from Case studies: A Case Study , 1992, ML.

[4]  Felix Naumann,et al.  Data fusion , 2009, CSUR.

[5]  John Quackenbush,et al.  Computational genetics: Computational analysis of microarray data , 2001, Nature Reviews Genetics.

[6]  André Carlos Ponce de Leon Ferreira de Carvalho,et al.  Meta-learning approach to gene expression data classification , 2009, Int. J. Intell. Comput. Cybern..

[7]  João Gama,et al.  On Data and Algorithms: Understanding Inductive Performance , 2004, Machine Learning.

[8]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[9]  Jill P. Mesirov,et al.  Consensus Clustering: A Resampling-Based Method for Class Discovery and Visualization of Gene Expression Microarray Data , 2003, Machine Learning.

[10]  Fred Collopy,et al.  Automatic Identification of Time Series Features for Rule-Based Forecasting , 2001 .

[11]  J. Z. Szymarkiewicz.  Forecasting and Time Series Analysis , 1978 .

[12]  Bay Arinze,et al.  Selecting appropriate forecasting models using rule induction , 1994 .

[13]  Teresa Bernarda Ludermir,et al.  Selection of Models for Time Series Prediction via Meta-Learning , 2002, HIS.

[14]  Francisco de A. T. de Carvalho,et al.  A Modal Symbolic Classifier for selecting time series models , 2004, Pattern Recognit. Lett..

[15]  Alexandros Kalousis,et al.  NOEMON: Design, implementation and performance results of an intelligent assistant for classifier selection , 1999, Intell. Data Anal..

[16]  Alexander Schliep,et al.  Ranking and selecting clustering algorithms using a meta-learning approach , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[17]  Tao Xiong,et al.  A combined SVM and LDA approach for classification , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[18]  Jill P. Mesirov,et al.  A resampling-based method for class discovery and visualization of gene expression microarray data , 2003 .

[19]  D. Slonim.  From patterns to pathways: gene expression data analysis comes of age , 2002, Nature Genetics.

[20]  Melanie Hilario,et al.  Representational Issues in Meta-Learning , 2003, ICML.

[21]  Melanie Hilario,et al.  Feature Selection for Meta-learning , 2001, PAKDD.

[22]  Ricardo Vilalta,et al.  A Perspective View and Survey of Meta-Learning , 2002, Artificial Intelligence Review.

[23]  Grigorios Tsoumakas,et al.  Lazy Adaptive Multicriteria Planning , 2004, ECAI.

[24]  Ivan G. Costa,et al.  Mining Rules for the Automatic Selection Process of Clustering Methods Applied to Cancer Gene Expression Data , 2009, ICANN.

[25]  Teresa Bernarda Ludermir,et al.  Meta-learning approaches to selecting time series models , 2004, Neurocomputing.

[26]  Pavel Brazdil,et al.  Predicting relative performance of classifiers from samples , 2005, ICML '05.

[27]  Hilan Bensusan,et al.  Estimating the Predictive Accuracy of a Classifier , 2001, ECML.

[28]  Leonard J. Tashman,et al.  Out-of-sample tests of forecasting accuracy: an analysis and review , 2000 .

[29]  Wlodzislaw Duch,et al.  What Is Computational Intelligence and Where Is It Going? , 2007, Challenges for Computational Intelligence.

[30]  Rui Xu,et al.  Survey of clustering algorithms , 2005, IEEE Transactions on Neural Networks.

[31]  Robert Engels,et al.  Using a Data Metric for Preprocessing Advice for Data Mining Applications , 1998, ECAI.

[32]  Ricardo Vilalta,et al.  Metalearning - Applications to Data Mining , 2008, Cognitive Technologies.

[33]  Christian Rudolf Köpf.  Meta-learning: strategies, implementations, and evaluations for algorithm selection , 2004 .

[34]  David J. Spiegelhalter,et al.  Machine Learning, Neural and Statistical Classification , 2009 .

[35]  Teresa Bernarda Ludermir,et al.  A Machine Learning Approach to Define Weights for Linear Combination of Forecasts , 2006, ICANN.

[36]  Aurora Trinidad Ramirez Pozo,et al.  Selecting software reliability models with a neural network meta classifier , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[37]  Kenneth Levenberg.  A METHOD FOR THE SOLUTION OF CERTAIN NON – LINEAR PROBLEMS IN LEAST SQUARES , 1944 .

[38]  Hilan Bensusan,et al.  Meta-Learning by Landmarking Various Learning Algorithms , 2000, ICML.

[39]  Jan Komorowski,et al.  Principles of Data Mining and Knowledge Discovery , 2001, Lecture Notes in Computer Science.

[40]  Włodzisław Duch,et al.  Challenges for Computational Intelligence , 2007, Studies in Computational Intelligence.

[41]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[42]  Anil K. Jain,et al.  Algorithms for Clustering Data , 1988 .

[43]  José Carlos Príncipe,et al.  Principles and networks for self-organization in space-time , 2002, Neural Networks.

[44]  Kate Smith-Miles,et al.  Towards insightful algorithm selection for optimisation using meta-learning concepts , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[45]  Carlos Soares,et al.  Ranking Learning Algorithms: Using IBL and Meta-Learning on Accuracy and Time Results , 2003, Machine Learning.

[46]  Carlos Soares.  UCI++: Improved Support for Algorithm Selection Using Datasetoids , 2009, PAKDD.

[47]  Levent Ertoz,et al.  A New Shared Nearest Neighbor Clustering Algorithm and its Applications , 2002 .

[48]  Patrik D'haeseleer,et al.  How does gene expression clustering work? , 2005, Nature Biotechnology.

[49]  Teresa Bernarda Ludermir,et al.  Selection of time series forecasting models based on performance information , 2004, Fourth International Conference on Hybrid Intelligent Systems (HIS'04).

[50]  Kevin J. Lang.  A time delay neural network architecture for speech recognition , 1989 .

[51]  Norbert Jankowski,et al.  Building meta-learning algorithms basing on search controlled by machine complexity , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[52]  Aapo Hyvärinen,et al.  Learning Features by Contrasting Natural Images with Noise , 2009, ICANN.

[53]  Richard A. Johnson,et al.  Applied Multivariate Statistical Analysis , 1983 .

[54]  Ricardo Vilalta,et al.  Introduction to the Special Issue on Meta-Learning , 2004, Machine Learning.

[55]  G. W. Milligan,et al.  A study of standardization of variables in cluster analysis , 1988 .

[56]  Alexander Schliep,et al.  Clustering cancer gene expression data: a comparative study , 2008, BMC Bioinformatics.

[57]  Kate Smith-Miles,et al.  Cross-disciplinary perspectives on meta-learning for algorithm selection , 2009, CSUR.