Modeling Joint Improvisation between Human and Virtual Players in the Mirror Game

Joint improvisation is observed to emerge spontaneously among humans performing joint action tasks, and has been associated with high levels of movement synchrony and enhanced sense of social bonding. Exploring the underlying cognitive and neural mechanisms behind the emergence of joint improvisation is an open research challenge. This paper investigates the emergence of jointly improvised movements between two participants in the mirror game, a paradigmatic joint task example. A theoretical model based on observations and analysis of experimental data is proposed to capture the main features of their interaction. A set of experiments is carried out to test and validate the model ability to reproduce the experimental observations. Then, the model is used to drive a computer avatar able to improvise joint motion with a human participant in real time. Finally, a convergence analysis of the proposed model is carried out to confirm its ability to reproduce the emergence of joint movement between the participants.

[1]  J. Kelso,et al.  The human dynamic clamp as a paradigm for social interaction , 2014, Proceedings of the National Academy of Sciences.

[2]  Chao Zhai,et al.  Adaptive tracking control of a virtual player in the mirror game , 2014, 53rd IEEE Conference on Decision and Control.

[3]  M. Bernardo,et al.  Entrainment and Synchronization in Heterogeneous Networks of Haken-Kelso-Bunz (HKB) Oscillators , 2015, 1509.00753.

[4]  A. Fuchs,et al.  Coordination: Neural, Behavioral and Social Dynamics , 2008 .

[5]  R. Keith Sawyer,et al.  Creating Conversations: Improvisation in Everyday Discourse , 2001 .

[6]  Chao Zhai,et al.  A novel cognitive architecture for a human-like virtual player in the mirror game , 2014, 2014 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[7]  H. Haken,et al.  A theoretical model of phase transitions in human hand movements , 2004, Biological Cybernetics.

[8]  A. Opstal.  Dynamic Patterns: The Self-Organization of Brain and Behavior , 1995 .

[9]  J. A. Scott Kelso,et al.  Virtual Partner Interaction (VPI): Exploring Novel Behaviors via Coordination Dynamics , 2009, PloS one.

[10]  A. G. Butkovskiy,et al.  Optimal control of systems , 1966 .

[11]  D. Lakens,et al.  If They Move in Sync, They Must Feel in Sync: Movement Synchrony Leads to Attributions of Rapport and Entitativity , 2011 .

[12]  Aslak Grinsted,et al.  Nonlinear Processes in Geophysics Application of the Cross Wavelet Transform and Wavelet Coherence to Geophysical Time Series , 2022 .

[13]  Peter J. Bickel,et al.  The Earth Mover's distance is the Mallows distance: some insights from statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[14]  U. Alon,et al.  The mirror game as a paradigm for studying the dynamics of two people improvising motion together , 2011, Proceedings of the National Academy of Sciences.

[15]  Mario di Bernardo,et al.  Kinematic characteristics of motion in the mirror game , 2014, 2014 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[16]  Lior Noy,et al.  Being in the zone: physiological markers of togetherness in joint improvisation , 2015, Front. Hum. Neurosci..

[17]  Chao Zhai,et al.  Dynamic similarity promotes interpersonal coordination in joint action , 2015, Journal of The Royal Society Interface.

[18]  Michael J. Richardson,et al.  Bodily synchronization underlying joke telling , 2014, Front. Hum. Neurosci..

[19]  V. Folkes,et al.  Forming Relationships and the Matching Hypothesis , 1982 .

[20]  Angelika Foerster,et al.  Coordination Neural Behavioral And Social Dynamics , 2016 .

[21]  Patrick J. F. Groenen,et al.  Modern Multidimensional Scaling: Theory and Applications , 2003 .

[22]  Gavan Lintern,et al.  Dynamic patterns: The self-organization of brain and behavior , 1997, Complex.

[23]  Michael J. Richardson,et al.  Social Connection Through Joint Action and Interpersonal Coordination , 2009, Top. Cogn. Sci..

[24]  Mario di Bernardo,et al.  A model predictive approach to control the motion of a virtual player in the mirror game , 2015, 2015 54th IEEE Conference on Decision and Control (CDC).

[25]  J. A. Scott Kelso,et al.  The Virtual Teacher (VT) Paradigm: Learning New Patterns of Interpersonal Coordination Using the Human Dynamic Clamp , 2015, PloS one.

[26]  K. Johnstone.  IMPRO: Improvisation and Theatre , 1979 .

[27]  U. Alon,et al.  Individuality and Togetherness in Joint Improvised Motion , 2014, PloS one.

[28]  Scott S. Wiltermuth,et al.  Synchrony and Cooperation , 2009, Psychological science.