Framework for integration of domain knowledge into logistic regression

Traditionally, machine learning extracts knowledge solely based on data. However, huge volume of knowledge is available in other sources which can be included into machine learning models. Still, domain knowledge is rarely used in machine learning. We propose a framework that integrates domain knowledge in form of hierarchies into machine learning models, namely logistic regression. Integration of the hierarchies is done by using stacking (stacked generalization). We show that the proposed framework yields better results compared to standard logistic regression model. The framework is tested on the binary classification problem for predicting 30-days hospital readmission. Results suggest that the proposed framework improves AUC (area under the curve) compared to logistic regression models unaware of domain knowledge by 9% on average.

[1]  Saso Dzeroski,et al.  An extensive experimental comparison of methods for multi-label learning , 2012, Pattern Recognit..

[2]  Eibe Frank,et al.  Logistic Model Trees , 2003, Machine Learning.

[3]  Marko Bohanec,et al.  DEX: An Expert System Shell for Decision Support • , 1990 .

[4]  Svetha Venkatesh,et al.  Stable feature selection for clinical prediction: Exploiting ICD tree structure using Tree-Lasso , 2015, J. Biomed. Informatics.

[5]  John Langford,et al.  Cost-sensitive learning by cost-proportionate example weighting , 2003, Third IEEE International Conference on Data Mining.

[6]  Zoran Obradovic,et al.  Improving Hospital Readmission Prediction Using Domain Knowledge Based Virtual Examples , 2015, KMO.

[7]  Baria Hafeez,et al.  Predicting frequent emergency department use among children with epilepsy: A retrospective cohort study using electronic health data from 2 centers , 2018, Epilepsia.

[8]  Nada Lavrač,et al.  Analysis of Example Weighting in Subgroup Discovery by Comparison of Three Algorithms on a Real-life Data Set , 2004 .

[9]  Edward J. Tanner,et al.  Under-utilization of minimally invasive surgery in the management of endometrial cancer: A Healthcare Cost and Utilization Project-National Inpatient Sample study (HCUP-NIS) , 2015 .

[10]  Pedro M. Domingos.  MetaCost: a general method for making classifiers cost-sensitive , 1999, KDD '99.

[11]  Guillermo Ricardo Simari,et al.  The Added Value of Argumentation , 2013 .

[12]  Eric P. Xing,et al.  Tree-Guided Group Lasso for Multi-Task Regression with Structured Sparsity , 2009, ICML.

[13]  N. Meinshausen,et al.  Stability selection , 2008, 0809.2932.

[14]  Zoran Obradovic,et al.  Domain knowledge Based Hierarchical Feature Selection for 30-Day Hospital Readmission Prediction , 2015, AIME.

[15]  Lei Wang,et al.  Retrieval with knowledge-driven kernel design: an approach to improving SVM-based CBIR with relevance feedback , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[16]  Fei Wang,et al.  FeaFiner: biomarker identification from medical data through feature generalization and selection , 2013, KDD.

[17]  Diego Calvanese,et al.  The Description Logic Handbook: Theory, Implementation, and Applications , 2003, Description Logic Handbook.

[18]  Ji Zhu,et al.  Kernel Logistic Regression and the Import Vector Machine , 2001, NIPS.

[19]  Geoffrey E. Hinton,et al.  An Alternative Model for Mixtures of Experts , 1994, NIPS.

[20]  Peter A. Flach,et al.  Propositionalization approaches to relational data mining , 2001 .

[21]  Geoffrey I. Webb,et al.  An Experimental Evaluation of Integrating Machine Learning with Knowledge Acquisition , 1999, Machine Learning.

[22]  Grigorios Tsoumakas,et al.  Effective and Efficient Multilabel Classification in Domains with Large Number of Labels , 2008 .

[23]  Heiko Paulheim,et al.  Feature Selection in Hierarchical Feature Spaces , 2014, Discovery Science.

[24]  J. Ross Quinlan,et al.  Learning logical definitions from relations , 1990, Machine Learning.

[25]  Andreas Holzinger,et al.  Interactive machine learning for health informatics: when do we need the human-in-the-loop? , 2016, Brain Informatics.

[26]  Marko Bohanec,et al.  Integrating knowledge from DEX hierarchies into a logistic regression stacking model for predicting ski injuries , 2018, J. Decis. Syst..

[27]  Boris Delibasic,et al.  Building interpretable predictive models for pediatric hospital readmission using Tree-Lasso logistic regression , 2016, Artif. Intell. Medicine.

[28]  Fei Wang,et al.  Comprehensible Predictive Modeling Using Regularized Logistic Regression and Comorbidity Based Features , 2015, PloS one.

[29]  Júlio C. Nievola,et al.  Hierarchical Classification of Gene Ontology with Learning Classifier Systems , 2012, IBERAMIA.

[30]  อนิรุธ สืบสิงห์,et al.  Data Mining Practical Machine Learning Tools and Techniques , 2014 .

[31]  Saso Dzeroski,et al.  Simultaneous Prediction of Mulriple Chemical Parameters of River Water Quality with TILDE , 1999, PKDD.

[32]  Tony Jan,et al.  VQSVM: A case study for incorporating prior domain knowledge into inductive machine learning , 2010, Neurocomputing.

[33]  Trevor Hastie,et al.  An Introduction to Statistical Learning , 2013, Springer Texts in Statistics.

[34]  Tso-Jung Yen,et al.  Discussion on "Stability Selection" by Meinshausen and Buhlmann , 2010 .

[35]  Zoran Obradovic,et al.  A Data and Knowledge Driven Randomization Technique for Privacy-Preserving Data Enrichment in Hospital Readmission Prediction , 2016 .

[36]  Marko Bohanec,et al.  Data-Mining and Expert Models for Predicting Injury Risk in Ski Resorts , 2015, ICDSST.

[37]  Vítor Santos Costa,et al.  Inductive Logic Programming , 2013, Lecture Notes in Computer Science.

[38]  Alan M Zaslavsky,et al.  Pediatric readmission prevalence and variability across hospitals. , 2013, JAMA.

[39]  Ivan Bratko,et al.  Elicitation of neurological knowledge with argument-based machine learning , 2013, Artif. Intell. Medicine.

[40]  Stephen Muggleton,et al.  Inverse entailment and progol , 1995, New Generation Computing.