Building Interpretable Classifiers with Rules using Bayesian Analysis

We aim to produce predictive models that are not only accurate, but are also interpretable to human experts. Our models are decision lists, which consist of a series of if...then... statements (for example, if high blood pressure, then stroke) that discretize a high-dimensional, multivariate feature space into a series of simple, readily interpretable decision statements. We introduce a generative model called the Bayesian List Machine (BLM), which yields a posterior distribution over possible decision lists. It employs a novel prior structure to encourage sparsity. In terms of predictive accuracy, our experiments show that the Bayesian List Machine is on par with the current top algorithms for prediction in machine learning. Our method is motivated by recent developments in personalized medicine, and can be used to produce highly accurate and interpretable medical scoring systems. We demonstrate this by producing an alternative to the CHADS2 score, actively used in clinical practice for estimating the risk of stroke in patients that have atrial fibrillation. Our model is as interpretable as CHADS2, but more accurate. if total cholesterol ≥160 and smoke then 10 year CHD risk ≥ 5% else if smoke and systolic blood pressure≥140 then 10 year CHD risk ≥ 5% else 10 year CHD risk < 5% Figure 1: Example decision list created using the NHBLI Framingham Heart Study Coronary Heart Disease (CHD) inventory for a 45 year old male.