Touching Virtual Agents: Embodiment and Mind

In this paper we outline the design and development of an embodied conversational agent setup that incorporates an augmented reality screen and tactile sleeve. With this setup the agent can visually and physically touch the user. We provide a literature overview of embodied conversational agents, as well as haptic technologies, and argue for the importance of adding touch to an embodied conversational agent. Finally, we provide guidelines for studies involving the touching virtual agent (TVA) setup.

[1]  Dirk Heylen,et al.  Self touch to touch others : designing the tactile sleeve for social touch , 2013 .

[2]  Nicolas Guéguen,et al.  Tactile Contact and Spontaneous Help: An Evaluation in a Natural Setting , 2003, The Journal of social psychology.

[3]  Gijs Huisman,et al.  A touch of affect: mediated social touch and affect , 2012, ICMI '12.

[4]  Jeremy N. Bailenson,et al.  Virtual Interpersonal Touch: Expressing and Recognizing Emotions Through Haptic Devices , 2007, Hum. Comput. Interact..

[5]  Hiroshi Ishii,et al.  ComTouch: design of a vibrotactile communication device , 2002, DIS '02.

[6]  Herbert H. Clark,et al.  Grounding in communication , 1991, Perspectives on socially shared cognition.

[7]  Clifford Nass,et al.  The media equation - how people treat computers, television, and new media like real people and places , 1996 .

[8]  C. H. Hansen,et al.  Post-compliance touch: an incentive for task performance. , 1994, The Journal of social psychology.

[9]  N. Guéguen,et al.  The effect of touch on compliance with a restaurant's employee suggestion , 2007 .

[10]  C. Spence,et al.  The science of interpersonal touch: An overview , 2010, Neuroscience & Biobehavioral Reviews.

[11]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[12]  Merijn Bruijnes,et al.  Computational models of social and emotional turn-taking for embodied conversational agents: a review , 2012 .

[13]  Stefan Kopp,et al.  A Conversational Agent as Museum Guide - Design and Evaluation of a Real-World Application , 2005, IVA.

[14]  Beat Fasel,et al.  Automati Fa ial Expression Analysis: A Survey , 1999 .

[15]  Matthew J. Hertenstein,et al.  The Communicative Functions of Touch in Humans, Nonhuman Primates, and Rats: A Review and Synthesis of the Empirical Research , 2006, Genetic, social, and general psychology monographs.

[16]  S. Kopp,et al.  Touch perception and emotional appraisal for a virtual agent , 2007 .

[17]  Ulrike Ehlert,et al.  Effects of different kinds of couple interaction on cortisol and heart rate responses to stress in women , 2007, Psychoneuroendocrinology.

[18]  SmithJocelyn,et al.  Communicating emotion through a haptic link , 2007 .

[19]  J. Fisher,et al.  Multidimensional reaction to therapeutic touch in a hospital setting. , 1979, Journal of personality and social psychology.

[20]  Stefan Kopp,et al.  Why emotions should be integrated into conversational agents , 2007 .

[21]  Maria Henricson,et al.  The outcome of tactile touch on stress parameters in intensive care: a randomized controlled trial. , 2008, Complementary therapies in clinical practice.

[22]  M. Tomasello,et al.  Does the chimpanzee have a theory of mind? 30 years later , 2008, Trends in Cognitive Sciences.

[23]  E. Goffman.  Interaction Ritual: Essays on Face-To-Face Behavior , 1967 .

[24]  Wijnand A. IJsselsteijn,et al.  Mediated social touch: a review of current research and future directions , 2006, Virtual Reality.

[25]  C. Kleinke,et al.  Compliance to requests made by gazing and touching experimenters in field settings. , 1977 .

[26]  Merijn Bruijnes,et al.  Affective Conversational Models: Interpersonal Stance in a Police Interview Context , 2013, 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction.

[27]  Radoslaw Niewiadomski,et al.  Laugh machine , 2013 .

[28]  W. Whitehead,et al.  Heart Rate Response to Touch1 , 1980, Psychosomatic medicine.

[29]  C. Cascio,et al.  TACTILE RESPONSIVENESS PATTERNS AND THEIR ASSOCIATION WITH CORE FEATURES IN AUTISM SPECTRUM DISORDERS. , 2012, Research in autism spectrum disorders.

[30]  Traci Yamashita,et al.  Massage Therapy versus Simple Touch to Improve Pain and Mood in Patients with Advanced Cancer , 2008, Annals of Internal Medicine.

[31]  S. Lederman,et al.  TACTUAL PERCEPTION , 2003 .

[32]  Alex Pentland,et al.  Face recognition using eigenfaces , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[33]  Malin Björnsdotter,et al.  Vicarious Responses to Social Touch in Posterior Insular Cortex Are Tuned to Pleasant Caressing Speeds , 2011, The Journal of Neuroscience.

[34]  Martin R. Gibbs,et al.  Hug over a distance , 2005, CHI Extended Abstracts.

[35]  Henriette Cramer,et al.  ‘Give me a hug’: the effects of touch and autonomy on people's responses to embodied social agents , 2009 .

[36]  Manuel Ferre.  Haptics: Perception, Devices and Scenarios, 6th International Conference, EuroHaptics 2008, Madrid, Spain, June 10-13, 2008, Proceedings , 2008, EuroHaptics.

[37]  R. Heslin,et al.  Meaning of touch: The case of touch from a stranger or same sex person , 1983 .

[38]  Illah R. Nourbakhsh,et al.  A survey of socially interactive robots , 2003, Robotics Auton. Syst..

[39]  Nicolas Guéguen,et al.  Nonverbal Encouragement of Participation in a Course: the Effect of Touching , 2004 .

[40]  Aude Billard,et al.  A survey of Tactile Human-Robot Interactions , 2010, Robotics Auton. Syst..

[41]  J. Weizenbaum.  From Computer Power and Human Reason From Judgment to Calculation , 2007 .

[42]  Randy Klaassen,et al.  User preferences for multi-device context-aware feedback in a digital coaching system , 2013, Journal on Multimodal User Interfaces.

[43]  Hong Z. Tan,et al.  The Body Surface as a Communication System: The State of the Art after 50 Years , 2007, PRESENCE: Teleoperators and Virtual Environments.

[44]  T. Field,et al.  Massage Therapy Research , 2006 .

[45]  Gijs Huisman,et al.  Touched by the Storyteller: The Influence of Remote Touch in the Context of Storytelling , 2013, 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction.

[46]  Stanley E. Jones,et al.  A naturalistic study of the meanings of touch , 1985 .

[47]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[48]  Youngwoo Park,et al.  POKE: a new way of sharing emotional touches during phone conversations , 2013, CHI Extended Abstracts.

[49]  H. Olausson,et al.  The skin as a social organ , 2010, Experimental Brain Research.

[50]  Dirk Heylen,et al.  ON THE NATURE OF ENGINEERING SOCIAL ARTIFICIAL COMPANIONS , 2011, Appl. Artif. Intell..

[51]  Mel Slater,et al.  Persuading people in a remote destination to sing by beaming there , 2013, VRST '13.

[52]  Staffan Larsson,et al.  Information state and dialogue management in the TRINDI dialogue move engine toolkit , 2000, Natural Language Engineering.

[53]  Jeremy N. Bailenson,et al.  Virtual interpersonal touch: Haptic interaction and copresence in collaborative virtual environments , 2008, Multimedia Tools and Applications.

[54]  M. Goldman,et al.  The Effect of Touching Different Body Areas on Prosocial Behavior , 1984 .

[55]  David J Anderson,et al.  Molecular genetic visualization of a rare subset of unmyelinated sensory neurons that may detect gentle touch , 2007, Nature Neuroscience.

[56]  T. Field.  Touch for Socioemotional and Physical Well-Being: A Review. , 2010 .

[57]  Haydn Jones,et al.  All together now , 1995 .

[58]  Clifford Nass,et al.  Force-Feedback in computer-mediated communication , 2001, HCI.

[59]  Hiroshi Ishiguro,et al.  Effect of robot's active touch on people's motivation , 2011, 2011 6th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[60]  Kory Floyd,et al.  All Touches are not Created Equal: Effects of Form and Duration on Observers' Interpretations of an Embrace , 1999 .

[61]  L. Nygaard,et al.  Communicating emotion: linking affective prosody and word meaning. , 2008, Journal of experimental psychology. Human perception and performance.

[62]  Dennis Reidsma,et al.  Elckerlyc - A BML Realizer for continuous, multimodal interaction with a Virtual Human , 2009 .

[63]  Ronald Poppe,et al.  A survey on vision-based human action recognition , 2010, Image Vis. Comput..

[64]  P. Andersen,et al.  International Patterns of Interpersonal Tactile Communication: A Field Study , 1998 .

[65]  J. Cassell,et al.  Embodied conversational agents , 2000 .

[66]  Scott Brave,et al.  inTouch: a medium for haptic interpersonal communication , 1997, CHI Extended Abstracts.

[67]  Antonella De Angeli,et al.  Mapping the demographics of virtual humans , 2007, BCS HCI.

[68]  Mel Slater,et al.  The Effect of Haptic Feedback on Basic Social Interaction within Shared Virtual Environments , 2008, EuroHaptics.

[69]  Nicolas Guéguen,et al.  Improving medication adherence by using practitioner nonverbal techniques: a field experiment on the effect of touch , 2010, Journal of Behavioral Medicine.

[70]  Jeffrey Tzu Kwan Valino Koh,et al.  Mobile implementation and user evaluation of the Huggy Pajama system , 2012, 2012 IEEE Haptics Symposium (HAPTICS).

[71]  Wijnand A. IJsselsteijn,et al.  Investigating response similarities between real and mediated social touch: a first test , 2007, CHI Extended Abstracts.

[72]  Benjamin Kuipers,et al.  Computer power and human reason , 1976, SGAR.

[73]  Kirsten Rassmus-Gröhn,et al.  Supporting presence in collaborative environments by haptic force feedback , 2000, TCHI.

[74]  P SumathiC.,et al.  Automatic Facial Expression Analysis A Survey , 2012 .

[75]  Roshan Lalintha Peiris,et al.  Huggy Pajama: a mobile parent and child hugging communication system , 2008, IDC.

[76]  Eva-Lotta Sallnäs,et al.  Haptic Feedback Increases Perceived Social Presence , 2010, EuroHaptics.

[77]  J. Wessberg,et al.  The neurophysiology of unmyelinated tactile afferents , 2010, Neuroscience & Biobehavioral Reviews.

[78]  M. Patterson,et al.  Touch, compliance, and interpersonal affect , 1986 .

[79]  Gabriel Skantze,et al.  A General, Abstract Model of Incremental Dialogue Processing , 2011 .

[80]  Dacher Keltner,et al.  Gender and the Communication of Emotion Via Touch , 2010, Sex roles.

[81]  Stefan Kopp,et al.  An Incremental Multimodal Realizer for Behavior Co-Articulation and Coordination , 2012, IVA.

[82]  Aduén Darriba Frederiks,et al.  Towards tactile expressions of emotion through mediated touch , 2013, CHI Extended Abstracts.

[83]  Jonathan Gratch,et al.  Expression of Moral Emotions in Cooperating Agents , 2009, IVA.

[84]  E. Schegloff,et al.  A simplest systematics for the organization of turn-taking for conversation , 1974 .

[85]  Jonathan D. Cohen,et al.  Rubber hands ‘feel’ touch that eyes see , 1998, Nature.

[86]  J. S. Wiggins,et al.  An informal history of the interpersonal circumplex tradition. , 1996, Journal of personality assessment.

[87]  Dirk Heylen,et al.  Keeping up stories: design considerations for a police interview training game , 2013 .

[88]  D. O’connell,et al.  Turn-taking: A critical analysis of the research tradition , 1990 .

[89]  D. Keltner,et al.  The communication of emotion via touch. , 2009, Emotion.

[90]  Kenneth O. Johnson,et al.  The roles and functions of cutaneous mechanoreceptors , 2001, Current Opinion in Neurobiology.

[91]  Rongrong Wang,et al.  Keep in touch: channel, expectation and experience , 2012, CHI.

[92]  J. Wessberg,et al.  Coding of pleasant touch by unmyelinated afferents in humans , 2009, Nature Neuroscience.

[93]  Brent Rossen,et al.  A crowdsourcing method to develop virtual human conversational agents , 2012, Int. J. Hum. Comput. Stud..

[94]  Wijnand A. IJsselsteijn,et al.  The Virtual Midas Touch: Helping Behavior After a Mediated Social Touch , 2009, IEEE Trans. Haptics.

[95]  Cynthia Breazeal,et al.  Affective Touch for Robotic Companions , 2005, ACII.

[96]  Wijnand A. IJsselsteijn,et al.  Embodiment and telepresence: Toward a comprehensive theoretical framework , 2012, Interact. Comput..

[97]  F. N. Willis,et al.  The use of interpersonal touch in securing compliance , 1980 .

[98]  J. Walther,et al.  Interpretations, Evaluations, and Consequences of Interpersonal Touch , 1992 .

[99]  Cédric Dumas,et al.  Influences of haptic communication on a shared manual task , 2011, Interact. Comput..

[100]  E. Friedmann,et al.  The effect of touch on nutritional intake of chronic organic brain syndrome patients. , 1986, Journal of gerontology.

[101]  M. Slater,et al.  The building blocks of the full body ownership illusion , 2013, Front. Hum. Neurosci..

[102]  Jeroen Stouten,et al.  A situational touch: How touch affects people's decision behavior , 2013 .

[103]  Timothy W. Bickmore,et al.  Empathic Touch by Relational Agents , 2010, IEEE Transactions on Affective Computing.

[104]  Merijn Bruijnes.  Social and Emotional Turn Taking for Embodied Conversational Agents , 2012, 2012 International Conference on Privacy, Security, Risk and Trust and 2012 International Confernece on Social Computing.

[105]  Dirk Heylen,et al.  Integrating Backchannel Prediction Models into Embodied Conversational Agents , 2012, IVA.

[106]  D. Keltner,et al.  Touch communicates distinct emotions. , 2006, Emotion.

[107]  Ben J. A. Kröse,et al.  The TaSSt: Tactile sleeve for social touch , 2013, 2013 World Haptics Conference (WHC).

[108]  Timothy Leary,et al.  Interpersonal diagnosis of personality : a functional theory and methodology for personality evaluation , 1958 .

[109]  Karon E. MacLean,et al.  Communicating emotion through a haptic link: Design space and methodology , 2007, Int. J. Hum. Comput. Stud..

[110]  H. Olausson,et al.  Feeling good: on the role of C fiber mediated touch in interoception , 2010, Experimental Brain Research.

[111]  Nicolas Guéguen,et al.  The effect of touch on tipping: an evaluation in a French bar , 2005 .

[112]  Jonathan Padilla San Diego,et al.  Design and Development of a Haptic Dental Training System - hapTEL , 2010, EuroHaptics.

[113]  Nicolas Guéguen,et al.  Failure of Tactile Contact to Increase Request Compliance : The Case of Blood Donation Behavior , 2011 .

[114]  Dirk Heylen,et al.  Disambiguating the functions of conversational sounds with prosody: the case of 'yeah' , 2010, INTERSPEECH.

[115]  C. Spence,et al.  The cognitive and neural correlates of “tactile consciousness”: A multisensory perspective , 2008, Consciousness and Cognition.