Design of a Virtual Player for Joint Improvisation with Humans in the Mirror Game

Joint improvisation is often observed among humans performing joint action tasks. Exploring the underlying cognitive and neural mechanisms behind the emergence of joint improvisation is an open research challenge. This paper investigates jointly improvised movements between two participants in the mirror game, a paradigmatic joint task example. First, experiments involving movement coordination of different dyads of human players are performed in order to build a human benchmark. No designation of leader and follower is given beforehand. We find that joint improvisation is characterized by the lack of a leader and high levels of movement synchronization. Then, a theoretical model is proposed to capture some features of their interaction, and a set of experiments is carried out to test and validate the model ability to reproduce the experimental observations. Furthermore, the model is used to drive a computer avatar able to successfully improvise joint motion with a human participant in real time. Finally, a convergence analysis of the proposed model is carried out to confirm its ability to reproduce joint movements between the participants.

[1]  Chao Zhai,et al.  A novel cognitive architecture for a human-like virtual player in the mirror game , 2014, 2014 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[2]  H. Haken,et al.  A theoretical model of phase transitions in human hand movements , 2004, Biological Cybernetics.

[3]  Michael J. Richardson,et al.  Bodily synchronization underlying joke telling , 2014, Front. Hum. Neurosci..

[4]  P. Groenen,et al.  Modern Multidimensional Scaling: Theory and Applications , 1999 .

[5]  Aslak Grinsted,et al.  Nonlinear Processes in Geophysics Application of the Cross Wavelet Transform and Wavelet Coherence to Geophysical Time Series , 2022 .

[6]  U. Alon,et al.  Would you like to play together? Adults’ attachment and the mirror game , 2016, Attachment & human development.

[7]  U. Alon,et al.  The mirror game as a paradigm for studying the dynamics of two people improvising motion together , 2011, Proceedings of the National Academy of Sciences.

[8]  A. Fuchs,et al.  Coordination: Neural, Behavioral and Social Dynamics , 2008 .

[9]  R. Keith Sawyer,et al.  Creating Conversations: Improvisation in Everyday Discourse , 2001 .

[10]  Chao Zhai,et al.  Adaptive tracking control of a virtual player in the mirror game , 2014, 53rd IEEE Conference on Decision and Control.

[11]  A. Opstal.  Dynamic Patterns: The Self-Organization of Brain and Behavior , 1995 .

[12]  U. Alon,et al.  Individuality and Togetherness in Joint Improvised Motion , 2014, PloS one.

[13]  Peter J. Bickel,et al.  The Earth Mover's distance is the Mallows distance: some insights from statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[14]  Michael J. Richardson,et al.  Dynamics of Interpersonal Coordination , 2008 .

[15]  V. Folkes,et al.  Forming Relationships and the Matching Hypothesis , 1982 .

[16]  Lior Noy,et al.  Being in the zone: physiological markers of togetherness in joint improvisation , 2015, Front. Hum. Neurosci..

[17]  Michael J. Richardson,et al.  Social Connection Through Joint Action and Interpersonal Coordination , 2009, Top. Cogn. Sci..

[18]  Mario di Bernardo,et al.  Kinematic characteristics of motion in the mirror game , 2014, 2014 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[19]  E. Kantorová,et al.  Visual Evoked Potential and Magnetic Resonance Imaging are More Effective Markers of Multiple Sclerosis Progression than Laser Polarimetry with Variable Corneal Compensation , 2014, Front. Hum. Neurosci..

[20]  Mario di Bernardo,et al.  Entrainment and synchronization in networks of Rayleigh–van der Pol oscillators with diffusive and Haken–Kelso–Bunz couplings , 2016, Biological Cybernetics.

[21]  G. Ermentrout.  Dynamic patterns: The self-organization of brain and behavior , 1997 .

[22]  Mario di Bernardo,et al.  A model predictive approach to control the motion of a virtual player in the mirror game , 2015, 2015 54th IEEE Conference on Decision and Control (CDC).

[23]  Scott S. Wiltermuth,et al.  Synchrony and Cooperation , 2009, Psychological science.

[24]  Guido Herrmann,et al.  Adaptive neural network dynamic surface control for musculoskeletal robots , 2014, 53rd IEEE Conference on Decision and Control.

[25]  J. Kelso,et al.  The human dynamic clamp as a paradigm for social interaction , 2014, Proceedings of the National Academy of Sciences.

[26]  Chao Zhai,et al.  Dynamic similarity promotes interpersonal coordination in joint action , 2015, Journal of The Royal Society Interface.

[27]  J. A. Scott Kelso,et al.  Virtual Partner Interaction (VPI): Exploring Novel Behaviors via Coordination Dynamics , 2009, PloS one.

[28]  K. Johnstone.  IMPRO: Improvisation and Theatre , 1979 .

[29]  Joze Guna,et al.  An Analysis of the Precision and Reliability of the Leap Motion Sensor and Its Suitability for Static and Dynamic Tracking , 2014, Sensors.

[30]  D. Lakens,et al.  If They Move in Sync, They Must Feel in Sync: Movement Synchrony Leads to Attributions of Rapport and Entitativity , 2011 .

[31]  J. A. Scott Kelso,et al.  The Virtual Teacher (VT) Paradigm: Learning New Patterns of Interpersonal Coordination Using the Human Dynamic Clamp , 2015, PloS one.

[32]  Chao Zhai,et al.  Comparing different control approaches to implement a human-like virtual player in the mirror game , 2016, 2016 European Control Conference (ECC).