A Bayesian Approach to Learning Scoring Systems

We present a Bayesian method for building scoring systems, which are linear models with coefficients that have very few significant digits. Usually the construction of scoring systems involve manual effort-humans invent the full scoring system without using data, or they choose how logistic regression coefficients should be scaled and rounded to produce a scoring system. These kinds of heuristics lead to suboptimal solutions. Our approach is different in that humans need only specify the prior over what the coefficients should look like, and the scoring system is learned from data. For this approach, we provide a Metropolis-Hastings sampler that tends to pull the coefficient values toward their "natural scale." Empirically, the proposed method achieves a high degree of interpretability of the models while maintaining competitive generalization performances.

[1]  D. E. Lawrence,et al.  APACHE—acute physiology and chronic health evaluation: a physiologically based classification system , 1981, Critical care medicine.

[2]  Gregory Y H Lip,et al.  Refining clinical risk stratification for predicting stroke and thromboembolism in atrial fibrillation using a novel risk factor-based approach: the euro heart survey on atrial fibrillation. , 2010, Chest.

[3]  S. Lemeshow,et al.  A new Simplified Acute Physiology Score (SAPS II) based on a European/North American multicenter study , 1993 .

[4]  Ivan Bratko,et al.  Machine Learning: Between Accuracy and Interpretability , 1997 .

[5]  Bart Baesens,et al.  Decompositional Rule Extraction from Support Vector Machines by Active Learning , 2009, IEEE Transactions on Knowledge and Data Engineering.

[6]  Bart Baesens,et al.  Comprehensible Credit Scoring Models Using Rule Extraction from Support Vector Machines , 2007, Eur. J. Oper. Res..

[7]  Erik Strumbelj,et al.  An Efficient Explanation of Individual Classifications using Game Theory , 2010, J. Mach. Learn. Res..

[8]  Cynthia Rudin,et al.  Supersparse linear integer models for optimized medical scoring systems , 2015, Machine Learning.

[9]  J. Ross Quinlan,et al.  Induction of Decision Trees , 1986, Machine Learning.

[10]  Cynthia Rudin,et al.  Supersparse Linear Integer Models for Interpretable Classification , 2013, 1306.6677.

[11]  Emilio Carrizosa,et al.  Strongly agree or strongly disagree?: Rating features in Support Vector Machines , 2016, Inf. Sci..

[12]  R. Rivest.  Learning Decision Lists , 1987, Machine Learning.

[13]  Martin T. Hagan,et al.  Neural network design , 1995 .

[14]  Peter Bauer,et al.  SAPS 3—From evaluation of the patient to evaluation of the intensive care unit. Part 2: Development of a prognostic model for hospital mortality at ICU admission , 2005, Intensive Care Medicine.

[15]  R. Tibshirani,et al.  Prototype selection for interpretable classification , 2011, 1202.5933.

[16]  Christophe Giraud-Carrier.  Beyond predictive accuracy : what? , 1998 .

[17]  Glenn Fung,et al.  Rule extraction from linear support vector machines , 2005, KDD '05.

[18]  Motoaki Kawanabe,et al.  How to Explain Individual Classification Decisions , 2009, J. Mach. Learn. Res..

[19]  Martha J. Radford,et al.  Validation of Clinical Classification Schemes for Predicting Stroke: Results From the National Registry of Atrial Fibrillation , 2001 .

[20]  W. C. Ball,et al.  Pleural effusions: the diagnostic separation of transudates and exudates. , 1972, Annals of internal medicine.

[21]  Janis Bormanis,et al.  Value of assessment of pretest probability of deep-vein thrombosis in clinical management , 1997, The Lancet.

[22]  N. Meinshausen.  Node harvest: simple and interpretable regression and classication , 2009, 0910.2145.

[23]  Brian A. Worley.  KNOWLEDGE DISCOVERY FROM DYNAMIC, DISPARATE DATA , 2012 .

[24]  Yann Chevaleyre,et al.  Rounding Methods for Discrete Linear Classification , 2013, ICML.

[25]  W. Knaus,et al.  The APACHE III prognostic system. Risk prediction of hospital mortality for critically ill hospitalized adults. , 1991, Chest.

[26]  Naeem Siddiqi,et al.  Credit Risk Scorecards: Developing and Implementing Intelligent Credit Scoring , 2005 .

[27]  Ron Kohavi,et al.  The Power of Decision Tables , 1995, ECML.

[28]  Erik Strumbelj,et al.  Explaining instance classifications with interactions of subsets of feature values , 2009, Data Knowl. Eng..

[29]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[30]  E. Antman,et al.  The TIMI risk score for unstable angina/non-ST elevation MI: A method for prognostication and therapeutic decision making. , 2000, JAMA.

[31]  Glenn Zorpette,et al.  The pitfalls of prediction [Spectral Lines] , 2014 .

[32]  J. L. Gall,et al.  A simplified acute physiology score for ICU patients , 1984, Critical care medicine.

[33]  Joel T. Andrade.  Handbook of Violence Risk Assessment and Treatment: New Approaches for Mental Health Professionals , 2009 .

[34]  Hendrik Blockeel,et al.  Seeing the Forest Through the Trees: Learning a Comprehensible Model from an Ensemble , 2007, ECML.

[35]  J. Ranson,et al.  Prognostic signs and the role of operative management in acute pancreatitis. , 1974, Surgery, gynecology & obstetrics.

[36]  Bart Baesens,et al.  Building comprehensible customer churn prediction models with advanced rule induction techniques , 2011, Expert Syst. Appl..

[37]  M Gent,et al.  Derivation of a Simple Clinical Model to Categorize Patients Probability of Pulmonary Embolism: Increasing the Models Utility with the SimpliRED D-dimer , 2000, Thrombosis and Haemostasis.

[38]  Cynthia Rudin,et al.  Supersparse Linear Integer Models for Predictive Scoring Systems , 2013, AAAI.

[39]  Bart Baesens,et al.  Domain knowledge integration in data mining using decision tables: case studies in churn prediction , 2009, J. Oper. Res. Soc..

[40]  J. E. Carceller.  American College of Chest Physicians/Society of Critical Care Medicine Consensus Conference: Definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis , 1992, Critical care medicine.

[41]  Stefan Rüping,et al.  Learning interpretable models , 2006 .

[42]  S. Lemeshow,et al.  A new Simplified Acute Physiology Score (SAPS II) based on a European/North American multicenter study. , 1993, JAMA.

[43]  T. Ong,et al.  A new, precise, and rapid technique of intestinal resection and anastomosis with staples. , 1974, Surgery, gynecology & obstetrics.

[44]  Michael J. Pazzani,et al.  Knowledge discovery from data? , 2000, IEEE Intell. Syst..

[45]  Alex Alves Freitas,et al.  Comprehensible classification models: a position paper , 2014, SKDD.

[46]  R. Apweiler,et al.  On the Importance of Comprehensible Classification Models for Protein Function Prediction , 2010, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[47]  E. Antman,et al.  TIMI Risk Score for ST-Elevation Myocardial Infarction: A Convenient, Bedside, Clinical Score for Risk Assessment at Presentation: An Intravenous nPA for Treatment of Infarcting Myocardium Early II Trial Substudy , 2000, Circulation.

[48]  W. Knaus,et al.  APACHE II: a severity of disease classification system. , 1985 .