QART: A System for Real-Time Holistic Quality Assurance for Contact Center Dialogues

Quality assurance (QA) and customer satisfaction (C-Sat) analysis are two commonly used practices to measure goodness of dialogues between agents and customers in contact centers. The practices however have a few shortcomings. QA puts sole emphasis on agents' organizational compliance aspect whereas C-Sat attempts to measure customers' satisfaction only based on post-dialogue surveys. As a result, outcome of independent QA and C-Sat analysis may not always be in correspondence. Secondly, both processes are retrospective in nature and hence, evidences of bad past dialogues (and consequently bad customer experiences) can only be found after hours or days or weeks depending on their periodicity. Finally, human intensive nature of these practices lead to time and cost overhead while being able to analyze only a small fraction of dialogues. In this paper, we introduce an automatic real-time quality assurance system for contact centers - QART (pronounced cart). QART performs multi-faceted analysis on dialogue utterances, as they happen, using sophisticated statistical and rule-based natural language processing (NLP) techniques. It covers various aspects inspired by today's QA and C-Sat practices as well as introduces novel incremental dialogue summarization capability. QART front-end is an interactive dashboard providing views of ongoing dialogues at different granularity enabling agents' supervisors to monitor and take corrective actions as needed. We demonstrate effectiveness of different back-end modules as well as the overall system by experimental results on a real-life contact center chat dataset.

[1]  Everett H. Larguier.  Gestalt Psychology: A Survey of Facts and Principles , 1936 .

[2]  Alison Dean,et al.  The Impact of the Customer Orientation of Call Center Employees on Customers' Affective Commitment and Loyalty , 2007 .

[3]  Gary Geunbae Lee,et al.  Automatic Agenda Graph Construction from Human-Human Dialogs using Clustering Method , 2009, HLT-NAACL.

[4]  P. Diggle,et al.  Analysis of Longitudinal Data. , 1997 .

[5]  Geoffrey Zweig,et al.  Automated Quality Monitoring for Call Centers using Speech and NLP Technologies , 2006, NAACL.

[6]  Shourya Roy,et al.  Text classification, business intelligence, and interactivity: automating C-Sat analysis for services industry , 2008, KDD.

[7]  Werner H. Kunz,et al.  Managing overall service quality in customer care centers: Empirical findings of a multi‐perspective approach , 2005 .

[8]  Youngja Park,et al.  Automatic call section segmentation for contact-center calls , 2007, CIKM '07.

[9]  Peter D. Turney.  Thumbs Up or Thumbs Down? Semantic Orientation Applied to Unsupervised Classification of Reviews , 2002, ACL.

[10]  Vladan Devedzic,et al.  Synesketch: An Open Source Library for Sentence-Based Emotion Recognition , 2013, IEEE Transactions on Affective Computing.

[11]  Karthik Visweswariah,et al.  Semi-automated logging of contact center telephone calls , 2008, CIKM '08.

[12]  Shourya Roy,et al.  Text to Intelligence: Building and Deploying a Text Mining Solution in the Services Industry for Customer Satisfaction Analysis , 2008, 2008 IEEE International Conference on Services Computing.

[13]  Frédérik Cailliau,et al.  Mining Automatic Speech Transcripts for the Retrieval of Problematic Calls , 2013, CICLing.

[14]  George W. Hartmann,et al.  Gestalt Psychology: A Survey of Facts and Principles , 1936 .

[15]  R. Likert.  “Technique for the Measurement of Attitudes, A” , 2022, The SAGE Encyclopedia of Research Design.

[16]  William W. Cohen.  Fast Effective Rule Induction , 1995, ICML.

[17]  P. Ekman.  Universal facial expressions of emotion. , 1970 .

[18]  Ingrid Zukerman,et al.  An Empirical Study of Corpus-Based Response Automation Methods for an E-mail-Based Help-Desk Domain , 2009, CL.

[19]  Zhong Chen,et al.  Exploiting web reviews for generating customer service surveys , 2010, SMUC '10.

[20]  Deepak Padmanabhan,et al.  Mining conversational text for procedures with applications in contact centers , 2007, International Journal of Document Analysis and Recognition (IJDAR).

[21]  Shourya Roy,et al.  Automatic Generation of Domain Models for Call-Centers from Noisy Transcriptions , 2006, ACL.

[22]  Jacob Cohen.  A Coefficient of Agreement for Nominal Scales , 1960 .

[23]  R. Plutchik.  Emotions and Life: Perspectives from Psychology, Biology, and Evolution , 2002 .

[24]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[25]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[26]  Gilad Mishne,et al.  Automatic analysis of call-center conversations , 2005, CIKM '05.

[27]  Anat Rafaeli,et al.  The Impact of Call Center Employees' Customer Orientation Behaviors on Service Quality , 2008 .

[28]  Shourya Roy,et al.  Getting insights from the voices of customers: Conversation mining at a contact center , 2009, Inf. Sci..

[29]  Shourya Roy,et al.  An integrated system for automatic customer satisfaction analysis in the services industry , 2008, KDD.