Predicting Student Success in Communication Skills Learning Scenarios with Virtual Humans

Virtual humans are frequently used to help medical students practice communication skills. Here, we show that communication skills features drawn from the literature on best practices for doctor-patient communication can be used to predict student interviewers' success in a given domain skill. We also demonstrate the viability of Bayesian Rule Lists, an interpretable machine learning model, for this use case. Bayesian Rule Lists' predictive performance is comparable to that of other other commonly used algorithms, including decision trees. This suggests that Bayesian Rule Lists, which produce simple, human-readable trained binary classifiers, may be suitable for providing feedback for educational purposes.

[1]  Nitesh V. Chawla,et al.  MOOC Dropout Prediction: Lessons Learned from Making Pipelines Interpretable , 2017, WWW.

[2]  Ruksana Osman,et al.  Massification in higher education: large classes and student learning , 2014 .

[3]  Cynthia Rudin,et al.  Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction model , 2015, ArXiv.

[4]  Alex Alves Freitas,et al.  On the Importance of Comprehensible Classification Models for Protein Function Prediction , 2010, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[5]  Carlos Guestrin,et al.  "Why Should I Trust You?": Explaining the Predictions of Any Classifier , 2016, ArXiv.

[6]  Brent Rossen,et al.  A crowdsourcing method to develop virtual human conversational agents , 2012, Int. J. Hum. Comput. Stud..

[7]  M. Lipkin,et al.  A randomized trial of teaching clinical skills using virtual and live standardized patients , 2006, Journal of General Internal Medicine.

[8]  Brent Rossen,et al.  High Score! - Motivation Strategies for User Participation in Virtual Human Development , 2010, IVA.

[9]  Susanne P. Lajoie,et al.  Using Learning Analytics to Identify Medical Student Misconceptions in an Online Virtual Patient Environment. , 2015 .

[10]  Xavier Ochoa,et al.  The RAP system: automatic feedback of oral presentation skills using multimodal analysis and low-cost sensors , 2018, LAK.

[11]  M. Stewart,et al.  Communication patterns of primary care physicians. , 1997, JAMA.

[12]  Wei-Yin Loh,et al.  Classification and regression trees , 2011, WIREs Data Mining Knowl. Discov..

[13]  Brent Rossen,et al.  Constructionism of virtual humans to improve perceptions of conversational partners , 2012, CHI EA '12.

[14]  Brent Rossen,et al.  Human-Centered Distributed Conversational Modeling: Efficient Modeling of Robust Virtual Human Conversations , 2009, IVA.

[15]  R C Smith,et al.  A strategy for improving patient satisfaction by the intensive training of residents in psychosocial medicine: a controlled, randomized study , 1995, Academic medicine : journal of the Association of American Medical Colleges.

[16]  Bart Baesens,et al.  An empirical evaluation of the comprehensibility of decision table, tree and rule based predictive models , 2011, Decis. Support Syst..

[17]  L. Callister,et al.  Patient‐centered care and adherence: Definitions and applications to improve outcomes , 2008, Journal of the American Academy of Nurse Practitioners.

[18]  Jeffrey T. Hancock,et al.  Language Style Matching as a Predictor of Social Dynamics in Small Groups , 2010, Commun. Res..

[19]  Ulrike Cress,et al.  The impact of physicians' communication styles on evaluation of physicians and information processing: A randomized study with simulated video consultations on contraception with an intrauterine device , 2016, Health expectations : an international journal of public participation in health care and health policy.

[20]  Amy Ogan,et al.  Alex: A Virtual Peer that Identifies Student Dialect , 2013 .

[21]  Ryan L. Boyd,et al.  The Development and Psychometric Properties of LIWC2015 , 2015 .

[22]  Kate Sweeny,et al.  Characteristics and Correlates of Word Use in Physician-Patient Communication , 2016, Annals of behavioral medicine : a publication of the Society of Behavioral Medicine.

[23]  Benjamin Lok,et al.  Understanding empathy training with virtual patients , 2015, Comput. Hum. Behav..

[24]  J. Myers,et al.  Doctor–Patient Communication, Health-Related Beliefs, and Adherence in Glaucoma: Results from the Glaucoma Adherence and Persistency Study , 2009 .

[25]  Richard M. Frankel,et al.  The Medical Interview , 1995, Frontiers of Primary Care.

[26]  James W. Pennebaker,et al.  Linguistic Inquiry and Word Count (LIWC2007) , 2007 .

[27]  Sandro Sperandei,et al.  Understanding logistic regression analysis , 2014, Biochemia medica.

[28]  Amy O. Stevens,et al.  The use of virtual patients to teach medical students history taking and communication skills. , 2006, American journal of surgery.

[29]  Ruth B Hoppe,et al.  "Best practice" for patient-centered communication: a narrative review. , 2013, Journal of graduate medical education.

[30]  Rui Guo,et al.  Participation-based student final performance prediction model through interpretable Genetic Programming: Integrating learning analytics, educational data mining and theory , 2015, Comput. Hum. Behav..

[31]  Cristina Conati,et al.  AI in Education needs interpretable machine learning: Lessons from Open Learner Modelling , 2018, ArXiv.

[32]  Benjamin Lok,et al.  Adapting Virtual Patient Interviews for Interviewing Skills Training of Novice Healthcare Students , 2015, IVA.