Language Features for Automated Evaluation of Cognitive Behavior Psychotherapy Sessions

Cognitive Behavior Therapy (CBT) is a psychotherapy treatment that uses cognitive change strategies to address mental health problems. Quality assessment of a CBT session is traditionally addressed by human raters who evaluate recorded sessions along specific behavioral codes, a cost prohibitive and time consuming method. In this work we examine how linguistic features can be effectively used to develop an automatic competency rating tool for CBT. We explore both standard, widelyused lexical features and domain-specific ones, adapting methods which have been successfully used in similar psychotherapy session coding tasks. Experiments are conducted on manual transcripts of CBT sessions and on automatically derived ones, thus introducing an end-to-end approach. Our results suggest that a real-world system could be developed to automatically evaluate CBT sessions to assist training, supervision, or quality assurance of services.

[1]  Shrikanth S. Narayanan,et al.  Predicting therapist empathy in motivational interviews using language features inspired by psycholinguistic norms , 2015, INTERSPEECH.

[2]  D. Ph.,et al.  COGNITIVE THERAPY SCALE RATING MANUAL , 2004 .

[3]  Panayiotis G. Georgiou,et al.  Analyzing speech rate entrainment and its relation to therapist empathy in drug addiction counseling , 2015, INTERSPEECH.

[4]  Panayiotis G. Georgiou,et al.  A Deep Learning Approach to Modeling Empathy in Addiction Counseling , 2016, INTERSPEECH.

[5]  Shrikanth S. Narayanan,et al.  Therapy language analysis using automatically generated psycholinguistic norms , 2015, INTERSPEECH.

[6]  David C. Atkins,et al.  Scaling up the evaluation of psychotherapy: evaluating motivational interviewing fidelity via statistical text classification , 2014, Implementation Science.

[7]  Denise Ernst,et al.  The Motivational Interviewing Treatment Integrity Code (MITI 4): Rationale, Preliminary Reliability and Validity. , 2016, Journal of substance abuse treatment.

[8]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[9]  Shrikanth S. Narayanan,et al.  A dialog act tagging approach to behavioral coding: a case study of addiction counseling conversations , 2015, INTERSPEECH.

[10]  Panayiotis G. Georgiou,et al.  Analyzing the language of therapist empathy in Motivational Interview based psychotherapy , 2012, Proceedings of The 2012 Asia Pacific Signal and Information Processing Association Annual Summit and Conference.

[11]  Panayiotis G. Georgiou,et al.  Attention Networks for Modeling Behaviors in Addiction Counseling , 2017, INTERSPEECH.

[12]  Panayiotis G. Georgiou,et al.  Modeling therapist empathy through prosody in drug addiction counseling , 2014, INTERSPEECH.

[13]  Panayiotis G. Georgiou,et al.  Behavioral Coding of Therapist Language in Addiction Counseling Using Recurrent Neural Networks , 2016, INTERSPEECH.

[14]  S. Hollon,et al.  6 – Cognitive Therapy of Depression1 , 1979 .

[15]  Mauro Cettolo,et al.  IRSTLM: an open source toolkit for handling large scale language models , 2008, INTERSPEECH.

[16]  Padhraic Smyth,et al.  Recursive Neural Networks for Coding Therapist and Patient Behavior in Motivational Interviewing , 2015, CLPsych@HLT-NAACL.

[17]  Ryan L. Boyd,et al.  The Development and Psychometric Properties of LIWC2015 , 2015 .

[18]  Che-Wei Huang,et al.  Distributed under Creative Commons Cc-by 4.0 a Technology Prototype System for Rating Therapist Empathy from Audio Recordings in Addiction Counseling , 2022 .

[19]  Fabrizio Sebastiani,et al.  Machine learning in automated text categorization , 2001, CSUR.

[20]  Michael McGill,et al.  Introduction to Modern Information Retrieval , 1983 .

[21]  David C. Atkins,et al.  More than reflections: empathy in motivational interviewing includes language style synchrony between therapist and client. , 2015, Behavior therapy.

[22]  Rachel Sherrill,et al.  Implementation of transdiagnostic cognitive therapy in community behavioral health: The Beck Community Initiative. , 2016, Journal of consulting and clinical psychology.

[23]  A. Beck,et al.  Cognitive behavior therapy : basics and beyond , 2011 .

[24]  K. Dobson,et al.  The Cognitive Therapy Scale: psychometric properties. , 1986, Journal of consulting and clinical psychology.

[25]  Shrikanth Narayanan,et al.  Signal Processing and Machine Learning for Mental Health Research and Clinical Applications [Perspectives] , 2017, IEEE Signal Processing Magazine.

[26]  Alicia C. Bunger,et al.  Outcomes for Implementation Research: Conceptual Distinctions, Measurement Challenges, and Research Agenda , 2010, Administration and Policy in Mental Health and Mental Health Services Research.