The Acorformed Corpus: Investigating Multimodality in Human-Human and Human-Virtual Patient Interactions

The paper aims at presenting the Acorformed corpus composed of human-human and human-machine interactions in French in the specific context of training doctors to break bad news to patients. In the context of human-human interaction, an audiovisual corpus of interactions between doctors and actors playing the role of patients during real training sessions in French medical institutions have been collected and annotated. This corpus has been exploited to develop a platform to train doctors to break bad news with a virtual patient. The platform has been exploited to collect a corpus of human-virtual patient interactions annotated semi-automatically and collected in different virtual reality environments with different degree of immersion (PC, virtual reality headset and virtual reality room).

[1]  Peter Wittenburg,et al.  Annotation by Category: ELAN and ISO DCR , 2008, LREC.

[2]  M. Ochs,et al.  An architecture of virtual patient simulation platform to train doctors to break bad news , 2017 .

[3]  Paul Boersma,et al.  Praat, a system for doing phonetics by computer , 2002 .

[4]  J. Cassell,et al.  More Than Just Another Pretty Face: Embodied Conversational Interface Agents , 1999 .

[5]  Robert Stalnaker,et al.  Common Ground , 2002 .

[6]  Philippe Blache,et al.  Toward an Automatic Prediction of the Sense of Presence in Virtual Reality Environment , 2018, HAI.

[7]  Philippe Blache,et al.  A Semi-autonomous System for Creating a Human-Machine Interaction Corpus in Virtual Reality: Application to the ACORFORMed System for Training Doctors to Break Bad News , 2018, LREC.

[8]  Philippe Blache,et al.  Training doctors’ social skills to break bad news: evaluation of the impact of virtual environment displays on the sense of presence , 2019, Journal on Multimodal User Interfaces.

[9]  Brigitte Bigi,et al.  SPPAS: a tool for the phonetic segmentation of speech , 2012, LREC.

[10]  Daniel Mestre,et al.  Affective, anxiety and behavioral effects of an aversive stimulation during a simulated navigation task within a virtual environment: A pilot study , 2011, Comput. Hum. Behav..

[11]  W. Baile,et al.  SPIKES-A six-step protocol for delivering bad news: application to the patient with cancer. , 2000, The oncologist.

[12]  Kimberley R Monden,et al.  Delivering Bad News to Patients , 2016, Proceedings.

[13]  Evelyn Yarzebinski,et al.  The Effects of Culturally Congruent Educational Technologies on Student Achievement , 2013, AIED.

[14]  Philippe Blache,et al.  MarsaTag, a tagger for French written texts and speech transcriptions , 2014 .

[15]  M. Pickering,et al.  Why is conversation so easy? , 2004, Trends in Cognitive Sciences.

[16]  A. Ichikawa,et al.  An Analysis of Turn-Taking and Backchannels Based on Prosodic and Syntactic Features in Japanese Map Task Dialogs , 1998, Language and speech.

[17]  Nicole C. Krämer,et al.  Social Effects of Virtual Assistants. A Review of Empirical Results with Regard to Communication , 2008, IVA.

[18]  Catherine Pelachaud,et al.  The TARDIS Framework: Intelligent Virtual Agents for Social Coaching in Job Interviews , 2013, Advances in Computer Entertainment.

[19]  Marion Tellier,et al.  Multimodal analysis of hand gesture back-channel feedback , 2016 .

[20]  Dave J. Hobbs,et al.  Continuous Presence in Collaborative Virtual Environments: Towards a Hybrid Avatar-Agent Model for User Representation , 2001, IVA.

[21]  Susan Persky,et al.  The Independent and Interactive Effects of Embodied-Agent Appearance and Behavior on Self-Report, Cognitive, and Behavioral Markers of Copresence in Immersive Virtual Environments , 2005, Presence: Teleoperators & Virtual Environments.

[22]  R. Espesser,et al.  Le CID - Corpus of Interactional Data. Annotation et exploitation multimodale de parole conversationnelle [The “Corpus of Interactional Data” (CID) - Multimodal annotation of conversational speech”] , 2008, ICON.

[23]  Jorge G Ruiz,et al.  Avatar-mediated training in the delivery of bad news in a virtual world. , 2010, Journal of palliative medicine.

[24]  Russell Beale,et al.  Affective interaction: How emotional agents affect users , 2009, Int. J. Hum. Comput. Stud..

[25]  M. Dodson,et al.  On the entropy of curves , 2013 .

[26]  Marcy E Rosenbaum,et al.  Teaching Medical Students and Residents Skills for Delivering Bad News: A Review of Strategies , 2004, Academic medicine : journal of the Association of American Medical Colleges.

[27]  C. Pelachaud,et al.  A Methodology for the Automatic Extraction and Generation of Non-Verbal Signals Sequences Conveying Interpersonal Attitudes , 2019, IEEE Transactions on Affective Computing.

[28]  Magalie Ochs,et al.  Mining a multimodal corpus of doctor’s training for virtual patient’s feedbacks , 2017, ICMI.

[29]  Thomas W. Schubert.  The sense of presence in virtual environments: A three-component scale measuring spatial presence, involvement, and realness , 2003, Z. für Medienpsychologie.