A Unified Framework for Individualized Avatar-Based Interactions

This paper presents a framework to interactively control avatars in remote environments. The system, called AMITIES, serves as the central component that connects people controlling avatars (inhabiters), various manifestations of these avatars (surrogates), and people interacting with these avatars (participants). A multiserver–client architecture, based on a low-demand network protocol, connects the participant environment(s), the inhabiter station(s), and the avatars. A human-in-the-loop metaphor provides an interface for remote operation, with support for multiple inhabiters, multiple avatars, and multiple participants. Custom animation blending routines and a gesture-based interface provide inhabiters with an intuitive avatar control paradigm. This gesture control is enhanced by genres of program-controlled behaviors that can be triggered by events or inhabiter choices for individual or groups of avatars. This mixed (agency and gesture-based) control paradigm reduces the cognitive and physical loads on the inhabiter while supporting natural bidirectional conversation between participants and the virtual characters or avatar counterparts, including ones with physical manifestations, for example, robotic surrogates. The associated system affords the delivery of personalized experiences that adapt to the actions and interactions of individual users, while staying true to each virtual character's personality and backstory. In addition to its avatar control paradigm, AMITIES provides processes for character and scenario development, testing, and refinement. It also has integrated capabilities for session recording and event tagging, along with automated tools for reflection and after-action review. We demonstrate effectiveness by describing an instantiation of AMITIES, called TeachLivE, that is widely used by colleges of education to prepare new teachers and provide continuing professional development to existing teachers. Finally, we show the system's flexibility by describing a number of other diverse applications, and presenting plans to enhance capabilities and application areas.

[1]  JoAnn Difede,et al.  Virtual Reality Applications to Address the Wounds of War , 2013 .

[2]  Arjun Nagendran,et al.  AMITIES: avatar-mediated interactive training and individualized experience system , 2013, VRST '13.

[3]  Hao Li,et al.  Realtime performance-based facial animation , 2011, ACM Trans. Graph..

[4]  J. F. Kelley,et al.  An iterative design methodology for user-friendly natural language office information applications , 1984, TOIS.

[5]  Nicholas R. Jennings,et al.  Intelligent agents: theory and practice , 1995, The Knowledge Engineering Review.

[6]  Dieter Schmalstieg,et al.  An Application Framework for Controlling an Avatar in a Desktop-Based Virtual Environment via a Software SSVEP BrainComputer Interface , 2010, PRESENCE: Teleoperators and Virtual Environments.

[7]  M. Black.  Avatars , 2008, BMJ : British Medical Journal.

[8]  Frank Biocca,et al.  The Effect of the Agency and Anthropomorphism on Users' Sense of Telepresence, Copresence, and Social Presence in Virtual Environments , 2003, Presence: Teleoperators & Virtual Environments.

[9]  Educating via entertainment media , 2003 .

[10]  Sung Yong Shin,et al.  Computer puppetry: An importance-based approach , 2001, TOGS.

[11]  Blair MacIntyre,et al.  Wizard of Oz support throughout an iterative design process , 2005, IEEE Pervasive Computing.

[12]  Maneesh Agrawala,et al.  3D puppetry: a kinect-based interface for 3D animation , 2012, UIST.

[13]  Edwin H. Blake,et al.  Conceptual priming as a determinant of presence in virtual environments , 2003, AFRIGRAPH '03.

[14]  Ingrid Heynderickx,et al.  The Effect of Priming Pictures and Videos on a QuestionAnswer Dialog Scenario in a Virtual Environment , 2013, PRESENCE: Teleoperators and Virtual Environments.

[15]  Ronald R. Mourant,et al.  Human Factors Issues in Virtual Environments: A Review of the Literature , 1998, Presence.

[16]  Stefano Faralli,et al.  CoPuppet: Collaborative Interaction in Virtual Puppetry , 2008 .

[17]  Arjun Nagendran,et al.  Continuum of virtual-human space: towards improved interaction strategies for physical-virtual avatars , 2012, VRCAI '12.

[18]  Laurel D. Riek,et al.  Wizard of Oz studies in HRI , 2012, J. Hum. Robot Interact..

[19]  Glenda Revelle,et al.  Educating via entertainment media: the Sesame Workshop approach , 2003, CIE.

[20]  Paul Clifton,et al.  Giving your self to the game: transferring a player's own movements to avatars using tangible interfaces , 2009, Sandbox@SIGGRAPH.

[21]  David J. Sturman,et al.  Computer Puppetry , 1998, IEEE Computer Graphics and Applications.

[22]  Greg Welch,et al.  Animatronic Shader Lamps Avatars , 2009, ISMAR.

[23]  Peter E. Rivard,et al.  Debriefing with good judgment: combining rigorous feedback with genuine inquiry. , 2007, Anesthesiology clinics.

[24]  Mel Slater,et al.  Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments , 2009, Philosophical Transactions of the Royal Society B: Biological Sciences.

[25]  Paul Clifton,et al.  I'm in the game: embodied puppet interface improves avatar control , 2010, TEI.

[26]  M. Press.  Presence : teleoperators and virtual environments. , 2014 .

[27]  Byron Reeves,et al.  Computer agents versus avatars: Responses to interactive game characters controlled by a computer or other player , 2010, Int. J. Hum. Comput. Stud..

[28]  Charles E. Hughes,et al.  Geppetto: An Environment for the Efficient Control and Transmission of Digital Puppetry , 2011, HCI.

[29]  Amy L. Baylor,et al.  The design of motivational agents and avatars , 2011 .

[30]  Ghassan Al-Qaimari,et al.  Evolving performance control systems for digital puppetry , 2000 .

[31]  Sudhanshu Kumar Semwal,et al.  Mapping Algorithms for Real-Time Control of an Avatar Using Eight Sensors , 1998, Presence.

[32]  Jessica K. Hodgins,et al.  Interactive control of avatars animated with human motion data , 2002, SIGGRAPH.

[33]  Arjun Nagendran,et al.  Symmetric telepresence using robotic humanoid surrogates , 2015, Comput. Animat. Virtual Worlds.

[34]  Crystal L. Hoyt,et al.  Immersive Virtual Environment Technology as a Methodological Tool for Social Psychology , 2002 .

[35]  Sharif Razzaque,et al.  The Responses of People to Virtual Humans in an Immersive Virtual Environment , 2005, Presence: Teleoperators & Virtual Environments.

[36]  Verónica Orvalho,et al.  Anim-actor: understanding interaction with digital puppetry using low-cost motion capture , 2011, Advances in Computer Entertainment Technology.

[37]  Jeremy N. Bailenson,et al.  A Meta-Analysis Quantifying the Effects of Avatars and Agents on Social Influence , 2010 .

[38]  J. Michael Moshell,et al.  Interactive Performance: Dramatic Improvisation in a Mixed Reality Environment for Learning , 2011, HCI.

[39]  Pattie Maes,et al.  Designing digital puppetry systems: guidelines and best practices , 2013, CHI Extended Abstracts.

[40]  Joseph Cohn,et al.  Evoking Affordances in Virtual Environments via Sensory-Stimuli Substitution , 2005, Presence: Teleoperators & Virtual Environments.

[41]  Mel Slater,et al.  The impact of avatar realism and eye gaze control on perceived quality of communication in a shared immersive virtual environment , 2003, CHI '03.