Confident Interpretation of Bayesian Decision Tree Ensembles for Clinical Applications

Bayesian averaging (BA) over ensembles of decision models allows evaluation of the uncertainty of decisions that is of crucial importance for safety-critical applications such as medical diagnostics. The interpretability of the ensemble can also give useful information for experts responsible for making reliable decisions. For this reason, decision trees (DTs) are attractive decision models for experts. However, BA over such models makes an ensemble of DTs uninterpretable. In this paper, we present a new approach to probabilistic interpretation of Bayesian DT ensembles. This approach is based on the quantitative evaluation of uncertainty of the DTs, and allows experts to find a DT that provides a high predictive accuracy and confident outcomes. To make the BA over DTs feasible in our experiments, we use a Markov Chain Monte Carlo technique with a reversible jump extension. The results obtained from clinical data show that in terms of predictive accuracy, the proposed method outperforms the maximum a posteriori (MAP) method that has been suggested for interpretation of DT ensembles

[1]  Refik Soyer,et al.  Bayesian Methods for Nonlinear Classification and Regression , 2004, Technometrics.

[2]  W. Copes,et al.  Evaluating trauma care: the TRISS method. Trauma Score and the Injury Severity Score. , 1987, The Journal of trauma.

[3]  Wray L. Buntine,et al.  Learning classification trees , 1992 .

[4]  Alberto Maria Segre,et al.  Programs for Machine Learning , 1994 .

[5]  Subhash C. Bagui,et al.  Combining Pattern Classifiers: Methods and Algorithms , 2005, Technometrics.

[6]  Jonathan E. Fieldsend,et al.  The Bayesian Decision Tree Technique with a Sweeping Strategy , 2005, ArXiv.

[7]  David G. Stork,et al.  Pattern Classification (2nd ed.) , 1999 .

[8]  E. George,et al.  Making sense of a forest of treesH , 2007 .

[9]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[10]  Jonathan E. Fieldsend,et al.  Bayesian inductively learned modules for safety critical systems , 2003 .

[11]  Pedro M. Domingos.  Knowledge Discovery Via Multiple Models , 1998, Intell. Data Anal..

[12]  Ludmila I. Kuncheva,et al.  Combining Pattern Classifiers: Methods and Algorithms , 2004 .

[13]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[14]  H. Chipman,et al.  Bayesian CART Model Search , 1998 .

[15]  Kathryn Fraughnaugh,et al.  Introduction to graph theory , 1973, Mathematical Gazette.

[16]  P. Green.  Reversible jump Markov chain Monte Carlo computation and Bayesian model determination , 1995 .

[17]  Christopher J. Merz,et al.  UCI Repository of Machine Learning Databases , 1996 .