Evaluating Human–Automation Etiquette Strategies to Mitigate User Frustration and Improve Learning in Affect-Aware Tutoring

Human–automation etiquette applies human–human etiquette conventions to human– computer interaction (HCI). The research described in this paper investigates how to mitigate user frustration and support student learning through changes in the style in which a computer tutor interacts with a learner. Frustration can significantly impact the quality of learning in tutoring. This study examined an approach to mitigate frustration through the use of different etiquette strategies to change the amount of imposition feedback placed on the learner. An experiment was conducted to explore how varying the interaction style of system feedback impacted aspects of the learning process. System feedback was varied through different etiquette strategies. Participants solved mathematics problems under different frustration conditions with feedback given in different etiquette styles. Changing etiquette strategies from one math problem to the next led to changes in motivation, confidence satisfaction, and performance. The most effective etiquette strategies changed depending on if the user was frustrated or not. This work aims to provide mechanisms to support the promotion of individualized learning in the context of high level math instruction by basing affect-aware adaptive tutoring system design on varying etiquette strategies.

[1]  Tom Murray,et al.  An Overview of Intelligent Tutoring System Authoring Tools: Updated Analysis of the State of the Art , 2003 .

[2]  Michael C. Dorneich,et al.  The Emotional, Cognitive, Physiological, and Performance Effects of Variable Time Delay in Robotic Teleoperation , 2017, Int. J. Soc. Robotics.

[3]  I L CHILD,et al.  Frustration and the quality of performance. III. An experimental study. , 1953, Journal of personality.

[4]  Rolf A. Zwaan,et al.  Pragmatics and Pedagogy: Conversational Rules and Politeness Strategies May Inhibit Effective Tutoring , 1995 .

[5]  Cynthia D. Fisher,et al.  A Within-Person Examination of Correlates of Performance and Emotions While Working , 2004, Emotion and Performance.

[6]  E. Goody.  Questions and politeness : strategies in social interaction , 1978 .

[7]  Ning Wang,et al.  The politeness effect: Pedagogical agents and learning outcomes , 2008, Int. J. Hum. Comput. Stud..

[8]  Cynthia Breazeal,et al.  Affective Learning — A Manifesto , 2004 .

[9]  Stuart Hansen,et al.  Engagement and frustration in programming projects , 2007, SIGCSE '07.

[10]  Samuel Lanes.  Frustration. The Development of a Scientific Concept , 1968 .

[11]  William Aspray,et al.  Relational Agents for Chronic Disease Self-Management , 2010 .

[12]  Ryan Shaun Joazeiro de Baker,et al.  Sequences of Frustration and Confusion, and Learning , 2013, EDM.

[13]  Santosh Mathan,et al.  Considering Etiquette in the Design of an Adaptive System , 2012 .

[14]  Ning Wang,et al.  Using Learner Focus of Attention to Detect Learner Motivation Factors , 2005, User Modeling.

[15]  Ivon Arroyo,et al.  Emotional intelligence for computer tutors , 2008 .

[16]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[17]  Kate S. Hone,et al.  Empathic agents to reduce user frustration: The effects of varying agent characteristics , 2006, Interact. Comput..

[18]  John A. Hoxmeier,et al.  System Response Time and User Satisfaction: An Experimental Study of Browser-based Applications , 2000 .

[19]  Rosalind W. Picard.  Computers That Recognize and Respond to User Emotion , 2003, User Modeling.

[20]  Rosalind W. Picard,et al.  An affective model of interplay between emotions and learning: reengineering educational pedagogy-building a learning companion , 2001, Proceedings IEEE International Conference on Advanced Learning Technologies.

[21]  Beverly Park Woolf,et al.  Repairing Disengagement With Non-Invasive Interventions , 2007, AIED.

[22]  Ben Shneiderman,et al.  Severity and impact of computer user frustration: A comparison of student and workplace users , 2006, Interact. Comput..

[23]  M. Murray,et al.  Frustration theory: An analysis of dispositional learning and memory , 1993 .

[24]  Richard E. Ferdig,et al.  Emotional Responses to Computers: Experiences in Unfairness, Anger, and Spite , 2004 .

[25]  Michael C. Dorneich,et al.  Towards a Characterization of Adaptive Systems: a Framework for Researchers and System Designers , 2017 .

[26]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[27]  Arthur C. Graesser,et al.  AutoTutor: an intelligent tutoring system with mixed-initiative dialogue , 2005, IEEE Transactions on Education.

[28]  Andrew Gelman,et al.  P values and statistical practice. , 2013, Epidemiology.

[29]  Deborah L. McGuinness,et al.  Toward establishing trust in adaptive agents , 2008, IUI '08.

[30]  L. Berkowitz.  Frustration-aggression hypothesis: examination and reformulation. , 1989, Psychological bulletin.

[31]  J. Hokanson,et al.  EFFECTS OF PHYSIOLOGICAL AROUSAL LEVEL, FRUSTRATION, AND TASK COMPLEXITY ON PERFORMANCE. , 1964, Journal of abnormal psychology.

[32]  Christopher A. Miller,et al.  Associates with Etiquette: Meta-Communication to Make Human-Automation Interaction more Natural, Productive and Polite , 2001 .

[33]  Bonnie Shellnut,et al.  Applying the ARCS model to the design and development of computer-based modules for manufacturing engineering courses , 1999 .

[34]  S. Mills.  Gender and Politeness , 2003 .

[35]  Neil T. Heffernan,et al.  Increasing Parent Engagement in Student Learning Using an Intelligent Tutoring System , 2011 .

[36]  D. Rempel,et al.  Influence of time pressure and verbal provocation on physiological and psychological reactions during work with a computer mouse , 2002, European Journal of Applied Physiology.

[37]  N. Miller,et al.  Frustration and aggression , 1939 .

[38]  P. Pintrich,et al.  Motivational and self-regulated learning components of classroom academic performance. , 1990 .

[39]  Anthony L Back,et al.  Efficacy of communication skills training for giving bad news and discussing transitions to palliative care. , 2007, Archives of internal medicine.

[40]  P. Brown,et al.  Universals in language usage: Politeness phenomena , 1978 .

[41]  Michael C. Dorneich,et al.  Affect-Aware Adaptive Tutoring Based on Human–Automation Etiquette Strategies , 2018, Hum. Factors.

[42]  Jonathan Klein,et al.  This computer responds to user frustration: Theory, design, and results , 2002, Interact. Comput..

[43]  Christopher A. Miller,et al.  - Human–Computer Etiquette: Should Computers Be Polite? , 2010 .

[44]  Stephen B. Gilbert,et al.  Authoring Effective Embedded Tutors: An Overview of the Extensible Problem Specific Tutor (xPST) System , 2015, International Journal of Artificial Intelligence in Education.

[45]  Sangeeta Malik,et al.  Effectiveness Of Arcs Model Of Motivational Design To Overcome Non Completion Rate Of Students In Distance Education , 2014 .

[46]  Christian Rauh,et al.  The effect of video feedback delay on frustration and emotion communication accuracy , 2011, Comput. Hum. Behav..

[47]  John M. Keller,et al.  The clinical use of motivational messages: an inquiry into the validity of the ARCS model of motivational design , 1990 .

[48]  Judith D. Wilson,et al.  Artificial Intelligence and Tutoring Systems , 1990 .

[49]  B. J. Fogg,et al.  Can computers be teammates? , 1996, Int. J. Hum. Comput. Stud..

[50]  Masanao Toda,et al.  Emotion and decision making , 1980 .

[51]  Peggy Wu,et al.  A Computational Approach to Etiquette: Operationalizing Brown and Levinson's Politeness Model , 2008, IEEE Intelligent Systems.

[52]  M. McDaniel,et al.  Learning Styles , 2008, Psychological science in the public interest : a journal of the American Psychological Society.

[53]  B. Weiner.  An attributional theory of achievement motivation and emotion. , 1985, Psychological review.

[54]  Sarmad Mohammad,et al.  Confidence -Motivation -Satisfaction- Performance (CMSP) Analysis of Blended Learning System in the Arab Open University Bahrain , 2012 .

[55]  J. Keller.  Development and use of the ARCS model of instructional design , 1987 .

[56]  Christopher A. Miller,et al.  Computational Models of Etiquette and Culture , 2016 .

[57]  S. Kiesling,et al.  Intercultural Discourse and Communication: The Essential Readings , 2005 .

[58]  J. Lerner,et al.  Emotion and decision making. , 2015, Annual review of psychology.

[59]  Beverly Park Woolf,et al.  Affect-aware tutors: recognising and responding to student affect , 2009, Int. J. Learn. Technol..

[60]  P. Ekman.  Universal facial expressions of emotion. , 1970 .

[61]  Christopher A. Miller,et al.  Trust and etiquette in high-criticality automated systems , 2004, CACM.

[62]  A. Amsel,et al.  Ontogeny of successive negative contrast and its dissociation from other paradoxical reward effects in preweanling rats. , 1981 .

[63]  Ben Shneiderman,et al.  Determining Causes and Severity of End-User Frustration , 2004, Int. J. Hum. Comput. Interact..

[64]  Michael C. Dorneich,et al.  Evaluation of Etiquette Strategies to Adapt Feedback In Affect-Aware Tutoring , 2016 .

[65]  Ina Fourie.  Health Informatics: A Patient‐centered Approach to Diabetes , 2011 .

[66]  Jonathan Klein,et al.  Computers that recognise and respond to user emotion: theoretical and practical implications , 2002, Interact. Comput..