Variable selection from random forests: application to gene expression data

AbstractRandom forest is a classification algorithm well suited for microarray data: it shows excellent performanceeven when most predictive variables are noise, can be used when the number of variables is much largerthan the number of observations, and returns measures of variable importance. Thus, it is important tounderstand the performance of random forest with microarray data and its use for gene selection.We first show the effects of changes in parameters of random forest on the prediction error. Then wepresent an approach for gene selection that uses measures of variable importance and error rate, and istargeted towards the selection of small sets of genes. Using simulated and real microarray data, we showthat the gene selection procedure yields small sets of genes while preserving predictive accuracy.We first show the effects of changes in parameters of random forest on the prediction error rate withmicroarray data. Then we present two approaches for gene selection with random forest: 1) comparingvariable importance plots of variable importance from original and permuted data sets; 2) using backwardsvariable elimination. Using simulated and real microarray data, we show: 1) variable importance plots canbe used to recover the full set of genes related to the outcome of interest, without being adversely affected bycollinearities; 2) backwards variable elimination yields small sets of genes while preserving predictive accuracy(compared to several state-of-the art algorithms). Thus, both methods are useful for gene selection.All code is available as an Rpackage, varSelRF,from CRANhttp://cran.r-project.org/src/contrib/PACKAGES.htmlor from the supplementary material page.Supplementary information: http://ligarto.org/rdiaz/Papers/rfVS/randomForestVarSel.html