Duet: Improvising Spatial Dialogues with an Artificially Intelligent Agent

This paper presents an experimental framework for a virtual reality artwork, Duet, that employs a combination of live, full body motion capture and Oculus Rift HMD to construct an experience through which a human User can spatially interact with an artificially intelligent Agent. The project explores conceptual notions of embodied knowledge transfer, shared poetics of movement and distortions of the body schema. Within this context, both the User and the Agent become performers, constructing an intimate and spontaneously generated proximal space. The project generates a visualization of the relationship between the User and the Agent without the context of a fixed VR landscape or architecture. The Agent's ability to retain and accumulate movement knowledge in a way that mimics human learning transforms an interactive experience into a collaborative one. The virtual representation of both performers is distorted and amplified in a dynamic manner, enhancing the potential for creative dialogue between the Agent and the User.

[1]  Heinrich H. Bülthoff,et al.  The Effect of Viewing a Self-Avatar on Distance Judgments in an HMD-Based Virtual Environment , 2010, PRESENCE: Teleoperators and Virtual Environments.

[2]  E. Edwards. Communication theory. , 1967, Ergonomics.

[3]  Nicholas P. Holmes,et al.  The body schema and multisensory representation(s) of peripersonal space , 2004, Cognitive Processing.

[4]  T. Metzinger,et al.  Video Ergo Sum: Manipulating Bodily Self-Consciousness , 2007, Science.

[5]  Mel Slater,et al.  Drumming in Immersive Virtual Reality: The Body Shapes the Way We Play , 2013, IEEE Transactions on Visualization and Computer Graphics.

[6]  Saeid Nahavandi,et al.  Learning to dance with a human , 2013 .

[7]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory , 1988 .

[8]  Maria V. Sanchez-Vives,et al.  Extending Body Space in Immersive Virtual Reality: A Very Long Arm Illusion , 2012, PloS one.

[9]  N. Burgess,et al.  Spatial memory: how egocentric and allocentric combine , 2006, Trends in Cognitive Sciences.

[10]  Teuvo Kohonen,et al.  Self-organization and associative memory: 3rd edition , 1989 .

[11]  Saeid Nahavandi,et al.  Teaching a Digital Performing Agent: Artificial Neural Network and Hidden Markov Model for recognising and performing dance movement , 2014, MOCO.

[12]  Jacquelyn Ford Morie,et al.  Performing in (virtual) spaces: Embodiment and being in virtual environments , 2007 .

[13]  Y. Coello,et al.  Body Space in Social Interactions: A Comparison of Reaching and Comfort Distance in Immersive Virtual Reality , 2014, PloS one.

[14]  M. V. Velzen,et al.  Self-organizing maps , 2007 .

[15]  M. Slater,et al.  Illusory ownership of a virtual child body causes overestimation of object sizes and implicit attitude changes , 2013, Proceedings of the National Academy of Sciences.

[16]  Petra Bosch-Sijtsema,et al.  Interactive navigation interface for Virtual Reality using the human body , 2014, Comput. Environ. Urban Syst..