Applying Bayesian Approach to Decision Tree

Applying Bayesian approach to decision tree (DT) model, and then a Bayesian-inference-based decision tree (BDT) model is proposed. For BDT we assign prior to the model parameters. Together with observed samples, prior are converted to posterior through Bayesian inference. When making inference we resort to simulation methods using reversible jump Markov chain Monte Carlo (RJMCMC) since the dimension of posterior distribution is varying. Compared with DT, BDT enjoys the following three advantages. Firstly, the model's learning procedure is implemented with sampling instead of a series of splitting and pruning operations. Secondly, the model provides output that gives insight into different tree structures and recursive partition of the decision space, resulting in better classification accuracy. And thirdly, the model can indicate confidence that the sample belongs to a particular class in classification. The experiments on music style classification demonstrate the efficiency of BDT.