First Experiments on Interaction Quality Modelling for Human-Human Conversation

This work presents the first experimental results on Interaction Quality modelling for human-human conversation, as an adaptation of the Interaction Quality metric for human-computer spoken interaction. The prediction of an Interaction Quality score can be formulated as a classification problem. In this paper we describe the results of applying several classification algorithms such as: Kernel Naive Bayes Classifier, k-Nearest Neighbours algorithm, Logistic Regression, and Support Vector Machines, to a data set. Moreover, we compare the results of modelling for two approaches for Interaction Quality labelling and consider the results depending on different emotion sets. The results of Interaction Quality modelling for human-human conversation may be used both for improving the service quality in call centres and for improving Spoken Dialogue Systems in terms of flexibility, user-friendliness and human-likeness.

[1]  J. Platt.  Sequential Minimal Optimization : A Fast Algorithm for Training Support Vector Machines , 1998 .

[2]  David Walker,et al.  Interaction Mining: the new Frontier of Call Center Analytics , 2011, DART@AI*IA.

[3]  Pat Langley,et al.  Estimating Continuous Distributions in Bayesian Classifiers , 1995, UAI.

[4]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[5]  Björn W. Schuller,et al.  Recent developments in openSMILE, the munich open-source multimedia feature extractor , 2013, ACM Multimedia.

[6]  Jordi Luque,et al.  Effect of gender and call duration on customer satisfaction in call center big data , 2015, INTERSPEECH.

[7]  Wolfgang Minker,et al.  Human-human task-oriented conversations corpus for interaction Quality modeling , 2016 .

[8]  John J. Kennedy,et al.  An Introduction to the Design and Analysis of Experiments in Behavioral Research , 1985 .

[9]  อนิรุธ สืบสิงห์,et al.  Data Mining Practical Machine Learning Tools and Techniques , 2014 .

[10]  Wolfgang Minker,et al.  Application and Evaluation of a Conditioned Hidden Markov Model for Estimating Interaction Quality of Spoken Dialogue Systems , 2014, Natural Interaction with Robots, Knowbots and Smartphones, Putting Spoken Dialog Systems into Practice.

[11]  Stefan Ultes,et al.  Interaction Quality: Assessing the quality of ongoing spoken dialog interaction by experts - And how it relates to user satisfaction , 2015, Speech Commun..

[12]  Wolfgang Minker,et al.  Modeling and Predicting Quality in Spoken Human-Computer Interaction , 2011, SIGDIAL Conference.

[13]  Youngja Park,et al.  Towards real-time measurement of customer satisfaction using automatically generated call transcripts , 2009, CIKM.

[14]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[15]  Wolfgang Minker,et al.  A Parameterized and Annotated Spoken Dialog Corpus of the CMU Let’s Go Bus Information System , 2012, LREC.

[16]  Anat Rafaeli,et al.  The Impact of Call Center Employees' Customer Orientation Behaviors on Service Quality , 2008 .

[17]  R. A. Bailey,et al.  Design of comparative experiments , 2008 .

[18]  Éric Gaussier,et al.  A Probabilistic Interpretation of Precision, Recall and F-Score, with Implication for Evaluation , 2005, ECIR.

[19]  S. Cessie,et al.  Ridge Estimators in Logistic Regression , 1992 .