Advances in random forests with application to classification

Advances in Random Forests with Application to

[1]  Bogdan E. Popescu,et al.  PREDICTIVE LEARNING VIA RULE ENSEMBLES , 2008, 0811.1679.

[2]  Jianjun Li.  A two-step rejection procedure for testing multiple hypotheses , 2008 .

[3]  Hua Wang,et al.  A maximally diversified multiple decision tree algorithm for microarray data classification , 2006 .

[4]  B. Holland,et al.  An Improved Sequentially Rejective Bonferroni Test Procedure , 1987 .

[5]  R. Tibshirani,et al.  Diagnosis of multiple cancer types by shrunken centroids of gene expression , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[6]  J. L. Hodges,et al.  Rank Methods for Combination of Independent Experiments in Analysis of Variance , 1962 .

[7]  Juan José Rodríguez Diez,et al.  Rotation Forest: A New Classifier Ensemble Method , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  O. J. Dunn.  Estimation of the Medians for Dependent Variables , 1959 .

[9]  Mohamed Medhat Gaber,et al.  GARF: Towards Self-optimised Random Forests , 2012, ICONIP.

[10]  J. Friedman.  Greedy function approximation: A gradient boosting machine. , 2001 .

[11]  Rich Caruana,et al.  An empirical comparison of supervised learning algorithms , 2006, ICML.

[12]  Renato De Mori,et al.  The Application of Semantic Classification Trees to Natural Language Understanding , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  R. Peng.  Reproducible Research in Computational Science , 2011, Science.

[14]  Simon Kasif,et al.  Induction of Oblique Decision Trees , 1993, IJCAI.

[15]  Gareth James,et al.  Variance and Bias for General Loss Functions , 2003, Machine Learning.

[16]  Terry R. Payne,et al.  Implicit Feature Selection with the Value Difference Metric , 1998, ECAI.

[17]  Jerome H. Friedman,et al.  On Bias, Variance, 0/1—Loss, and the Curse-of-Dimensionality , 2004, Data Mining and Knowledge Discovery.

[18]  Tin Kam Ho,et al.  The Random Subspace Method for Constructing Decision Forests , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Horst Bischof,et al.  On-line Random Forests , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[20]  Trevor F. Cox,et al.  Metric multidimensional scaling , 2000 .

[21]  Leo Breiman,et al.  Classification and Regression Trees , 1984 .

[22]  B. Efron.  Estimating the Error Rate of a Prediction Rule: Improvement on Cross-Validation , 1983 .

[23]  George C. Runger,et al.  Gene selection with guided regularized random forest , 2012, Pattern Recognit..

[24]  David J Hand,et al.  Evaluating diagnostic tests: The area under the ROC curve and the balance of errors , 2010, Statistics in medicine.

[25]  Y. Hochberg.  A sharper Bonferroni procedure for multiple tests of significance , 1988 .

[26]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[27]  Patrick van der Smagt,et al.  Introduction to neural networks , 1995, The Lancet.

[28]  Jae Won Lee,et al.  An extensive comparison of recent classification tools applied to microarray data , 2004, Comput. Stat. Data Anal..

[29]  Gerald J August,et al.  Diagnosing attention deficit disorders with the Behavioral Assessment System for Children and the Child Behavior Checklist: test and construct validity analyses using optimal discriminant classification trees. , 1998, Journal of consulting and clinical psychology.

[30]  Hongshik Ahn,et al.  A weight-adjusted voting algorithm for ensembles of classifiers , 2011 .

[31]  Yoram Singer,et al.  Reducing Multiclass to Binary: A Unifying Approach for Margin Classifiers , 2000, J. Mach. Learn. Res..

[32]  Elie Bienenstock,et al.  Neural Networks and the Bias/Variance Dilemma , 1992, Neural Computation.

[33]  J. Morgan,et al.  Problems in the Analysis of Survey Data, and a Proposal , 1963 .

[34]  David H. Wolpert,et al.  On Bias Plus Variance , 1997, Neural Computation.

[35]  Andrew P. Bradley,et al.  The use of the area under the ROC curve in the evaluation of machine learning algorithms , 1997, Pattern Recognit..

[36]  E. Tuzcu,et al.  Coronary Plaque Classification With Intravascular Ultrasound Radiofrequency Data Analysis , 2002, Circulation.

[37]  R. Iman,et al.  Approximations of the critical region of the fbietkan statistic , 1980 .

[38]  Johannes Welbl,et al.  Casting Random Forests as Artificial Neural Networks (and Profiting from It) , 2014, GCPR.

[39]  Yunming Ye,et al.  Classifying Very High-Dimensional Data with Random Forests Built from Small Subspaces , 2012, Int. J. Data Warehous. Min..

[40]  Fatin Zaklouta,et al.  Traffic sign classification using K-d trees and Random Forests , 2011, The 2011 International Joint Conference on Neural Networks.

[41]  Piotr Fryzlewicz,et al.  Random Rotation Ensembles , 2016, J. Mach. Learn. Res..

[42]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[43]  Mohamed Medhat Gaber,et al.  On Extreme Pruning of Random Forest Ensembles for Real-time Predictive Applications , 2015, ArXiv.

[44]  Iñaki Inza,et al.  Dealing with the evaluation of supervised classification algorithms , 2015, Artificial Intelligence Review.

[45]  R. A. Leibler,et al.  On Information and Sufficiency , 1951 .

[46]  Yali Amit,et al.  Shape Quantization and Recognition with Randomized Trees , 1997, Neural Computation.

[47]  G. Hommel.  A stagewise rejective multiple test procedure based on a modified Bonferroni test , 1988 .

[48]  K. Hornik,et al.  Unbiased Recursive Partitioning: A Conditional Inference Framework , 2006 .

[49]  Peter Buhlmann,et al.  BOOSTING ALGORITHMS: REGULARIZATION, PREDICTION AND MODEL FITTING , 2007, 0804.2752.

[50]  Victoria Stodden,et al.  Reproducible Research , 2019, The New Statistics with R.

[51]  Achim Zeileis,et al.  BMC Bioinformatics BioMed Central Methodology article Conditional variable importance for random forests , 2008 .

[52]  R. E. Lee,et al.  Distribution-free multiple comparisons between successive treatments , 1995 .

[53]  B. Efron.  The jackknife, the bootstrap, and other resampling plans , 1987 .

[54]  Tadayoshi Fushiki,et al.  Estimation of prediction error by using K-fold cross-validation , 2011, Stat. Comput..

[55]  George C. Runger,et al.  Feature selection via regularized trees , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[56]  Ron Kohavi,et al.  Bias Plus Variance Decomposition for Zero-One Loss Functions , 1996, ICML.

[57]  R. Tibshirani,et al.  Improvements on Cross-Validation: The 632+ Bootstrap Method , 1997 .

[58]  Ron Kohavi,et al.  A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection , 1995, IJCAI.

[59]  Robert Tibshirani,et al.  Bias, Variance and Prediction Error for Classification Rules , 1996 .

[60]  David L. Dowe,et al.  Decision Forests with Oblique Decision Trees , 2006, MICAI.

[61]  Laurent Heutte,et al.  On the selection of decision trees in Random Forests , 2009, 2009 International Joint Conference on Neural Networks.

[62]  Ramón Díaz-Uriarte,et al.  Gene selection and classification of microarray data using random forest , 2006, BMC Bioinformatics.

[63]  Trevor Hastie,et al.  An Introduction to Statistical Learning , 2013, Springer Texts in Statistics.

[64]  Anne-Laure Boulesteix,et al.  Overview of random forest methodology and practical guidance with emphasis on computational biology and bioinformatics , 2012, WIREs Data Mining Knowl. Discov..

[65]  Tony R. Martinez,et al.  Improved Heterogeneous Distance Functions , 1996, J. Artif. Intell. Res..

[66]  Chris Carter,et al.  Multiple decision trees , 2013, UAI.

[67]  M. Friedman.  The Use of Ranks to Avoid the Assumption of Normality Implicit in the Analysis of Variance , 1937 .

[68]  Pierre Geurts,et al.  Contributions to decision tree induction: bias/variance tradeoff and time series classification , 2002 .

[69]  Olivier Debeir,et al.  Limiting the Number of Trees in Random Forests , 2001, Multiple Classifier Systems.

[70]  Dirk Van den Poel,et al.  Predicting customer retention and profitability by using random forests and regression forests techniques , 2005, Expert Syst. Appl..

[71]  David S. Siroky.  Navigating Random Forests and related advances in algorithmic modeling , 2009 .

[72]  Ulisses Braga-Neto,et al.  Bolstered error estimation , 2004, Pattern Recognit..

[73]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[74]  S. García,et al.  An Extension on "Statistical Comparisons of Classifiers over Multiple Data Sets" for all Pairwise Comparisons , 2008 .

[75]  John D. Storey,et al.  Statistical significance for genomewide studies , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[76]  Griselda Saldaña-González,et al.  Investigation of Random Forest Performance with Cancer Microarray Data , 2008, CATA.

[77]  Yi Lin,et al.  Random Forests and Adaptive Nearest Neighbors , 2006 .

[78]  Andreas Ziegler,et al.  Mining data with random forests: current options for real‐world applications , 2014, WIREs Data Mining Knowl. Discov..

[79]  Houtao Deng,et al.  Guided Random Forest in the RRF Package , 2013, ArXiv.

[80]  Tolga Tasdizen,et al.  Disjunctive normal random forests , 2015, Pattern Recognit..

[81]  Houtao Deng,et al.  Interpreting tree ensembles with inTrees , 2018, International Journal of Data Science and Analytics.

[82]  Marko Robnik-Sikonja,et al.  Improving Random Forests , 2004, ECML.

[83]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[84]  Chun-Xia Zhang,et al.  RotBoost: A technique for combining Rotation Forest and AdaBoost , 2008, Pattern Recognit. Lett..

[85]  J. Ross Quinlan,et al.  Induction of Decision Trees , 1986, Machine Learning.

[86]  Christoforos Anagnostopoulos,et al.  A better Beta for the H measure of classification performance , 2012, Pattern Recognit. Lett..

[87]  R. Fisher,et al.  STATISTICAL METHODS AND SCIENTIFIC INDUCTION , 1955 .

[88]  G. V. Kass.  An Exploratory Technique for Investigating Large Quantities of Categorical Data , 1980 .

[89]  Mohamed Abdel-Aty,et al.  Using conditional inference forests to identify the factors affecting crash severity on arterial corridors. , 2009, Journal of safety research.

[90]  Leo Breiman,et al.  Statistical Modeling: The Two Cultures (with comments and a rejoinder by the author) , 2001 .

[91]  David H. Wolpert,et al.  An Efficient Method To Estimate Bagging's Generalization Error , 1999, Machine Learning.

[92]  R. Plant,et al.  Classification trees: An alternative non‐parametric approach for predicting species distributions , 2000 .

[93]  Adele Cutler,et al.  PERT – Perfect Random Tree Ensembles , 2001 .

[94]  Ron Kohavi,et al.  The Case against Accuracy Estimation for Comparing Induction Algorithms , 1998, ICML.

[95]  David Mease,et al.  Evidence Contrary to the Statistical View of Boosting , 2008, J. Mach. Learn. Res..

[96]  Gareth M. James,et al.  Generalizations of the Bias/Variance Decomposition for Prediction Error , 1997 .

[97]  Mykola Pechenizkiy,et al.  Dynamic Integration with Random Forests , 2006, ECML.

[98]  Ponnuthurai N. Suganthan,et al.  Random Forests with ensemble of feature spaces , 2014, Pattern Recognit..

[99]  Jean-Michel Poggi,et al.  Variable selection using random forests , 2010, Pattern Recognit. Lett..

[100]  Borja Calvo,et al.  scmamp: Statistical Comparison of Multiple Algorithms in Multiple Problems , 2016, R J..

[101]  Tom Heskes,et al.  Bias/Variance Decompositions for Likelihood-Based Estimators , 1998, Neural Computation.

[102]  Yunming Ye,et al.  Stratified sampling for feature subspace selection in random forests for high dimensional data , 2013, Pattern Recognit..

[103]  D. Rom.  A sequentially rejective test procedure based on a modified Bonferroni inequality , 1990 .

[104]  P. Burman.  A comparative study of ordinary cross-validation, v-fold cross-validation and the repeated learning-testing methods , 1989 .

[105]  G. Hommel,et al.  Improvements of General Multiple Test Procedures for Redundant Systems of Hypotheses , 1988 .

[106]  Yong-Heng Zhao,et al.  Random forest algorithm for classification of multiwavelength data , 2009 .

[107]  D. Quade.  Using Weighted Rankings in the Analysis of Complete Blocks with Additive Block Effects , 1979 .

[108]  Pierre Geurts,et al.  Extremely randomized trees , 2006, Machine Learning.

[109]  Joshua Zhexue Huang,et al.  Unbiased Feature Selection in Learning Random Forests for High-Dimensional Data , 2015, TheScientificWorldJournal.

[110]  Dimitrios I. Fotiadis,et al.  Modifications of the construction and voting mechanisms of the Random Forests Algorithm , 2013, Data Knowl. Eng..

[111]  S. Holm.  A Simple Sequentially Rejective Multiple Test Procedure , 1979 .

[112]  L. Breiman.  OUT-OF-BAG ESTIMATION , 1996 .

[113]  H. Finner.  On a Monotonicity Problem in Step-Down Multiple Test Procedures , 1993 .

[114]  Leo Breiman,et al.  Randomizing Outputs to Increase Prediction Accuracy , 2000, Machine Learning.

[115]  Gian Luca Foresti,et al.  Meta Random Forests , 2006 .

[116]  Ullrich Köthe,et al.  On Oblique Random Forests , 2011, ECML/PKDD.

[117]  William G. Hanley,et al.  Discriminant Random Forests , 2008, DMIN.

[118]  J. Shaffer.  Modified Sequentially Rejective Multiple Test Procedures , 1986 .

[119]  Christoforos Anagnostopoulos,et al.  When is the area under the receiver operating characteristic curve an appropriate measure of classifier performance? , 2013, Pattern Recognit. Lett..

[120]  Thomas G. Dietterich.  Approximate Statistical Tests for Comparing Supervised Classification Learning Algorithms , 1998, Neural Computation.

[121]  Mark R. Segal,et al.  Machine Learning Benchmarks and Random Forest Regression , 2004 .

[122]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[123]  Geraldo Zimbrão,et al.  Transforming collaborative filtering into supervised learning , 2015, Expert Syst. Appl..

[124]  Francisco Herrera,et al.  Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: Experimental analysis of power , 2010, Inf. Sci..

[125]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[126]  Yung-Seop Lee,et al.  Enriched random forests , 2008, Bioinform..

[127]  A. Isaksson,et al.  Cross-validation and bootstrapping are unreliable in small sample classification , 2008, Pattern Recognit. Lett..

[128]  Ryan M. Rifkin,et al.  In Defense of One-Vs-All Classification , 2004, J. Mach. Learn. Res..

[129]  David J. Hand,et al.  Measuring classifier performance: a coherent alternative to the area under the ROC curve , 2009, Machine Learning.

[130]  Thomas G. Dietterich,et al.  Machine Learning Bias, Statistical Bias, and Statistical Variance of Decision Tree Algorithms , 2008 .

[131]  Pedro M. Domingos.  A Unified Bias-Variance Decomposition , 2022 .

[132]  D. R. Cutler,et al.  Utah State University From the SelectedWorks of , 2017 .

[133]  Thomas G. Dietterich,et al.  Error-Correcting Output Coding Corrects Bias and Variance , 1995, ICML.

[134]  Sara J. Iverson,et al.  Fatty acid signatures and classification trees: new tools for investigating the foraging ecology of seals , 1997 .

[135]  Rich Caruana,et al.  An empirical evaluation of supervised learning in high dimensions , 2008, ICML '08.

[136]  Henrik Boström.  Estimating class probabilities in random forests , 2007, ICMLA 2007.