Network Analysis of Multimodal, Multiscale Coordination in Dyadic Problem Solving

Network Analysis of Multimodal, Multiscale Coordination in Dyadic Problem Solving Alexandra Paxton (paxton.alexandra@gmail.com) Drew H. Abney (dabney@ucmerced.edu) Christopher T. Kello (ckello@ucmerced.edu) Rick Dale (rdale@ucmerced.edu) Cognitive & Information Sciences, University of California, Merced Merced, CA 95343 USA Abstract A recent trend in dyadic interaction research utilizes multiple modalities to better understand phenomena encompassing behavior matching (e.g., synchrony, alignment). Concurrent research has focused on a complementary framework of interaction, assessing the matching of power law distributions of behavior across two people: complexity matching. While both frameworks provide useful insights into dyadic interaction, they have done so independent of one another. We visualize the multimodal, multiscale coordination of dyads engaged in a tower-building task as networks based on the analyses of behavioral and complexity matching in speech and movement. We find that network strength relates to task performance and that high-performing dyads have weaker network strength, which we argue opens up more degrees of freedom affording more flexibility in the dyadic system. Keywords: communication; complexity matching; convergence; dyadic interaction; interpersonal synchrony; networks Introduction Interpersonal communication lives across a number of timescales. During face-to-face interaction, our signals to one another can last from milliseconds to hours, and smooth interaction requires an effective juggling of incoming and outgoing signals. We readily perceive and quickly respond to more obvious signals like facial expressions and linguistic information, but we also influence one another in more subtle ways. From language (Brennan & Clark, 1996; Garrod & Pickering, 2004) and emotion (Hatfield, Cacioppo, & Rapson, 1995) to neural patterns (Stevens, Silbert, & Hasson, 2010) and physiological signals (Helm, Sbarra, & Ferrer, 2012), research on interpersonal convergence (or coordination, entrainment, or synchrony) highlights ways in which we influence and are influenced by those with whom we interact. This often builds on a large body of joint action literature (Clark, 1996), exploring how we come to work together. Previous research on convergence has tended to focus on specific behaviors or patterns (e.g., Louwerse, Dale, Bard, & Jeuniaux, 2012), but an emerging and exciting focus instead investigates interpersonal convergence of the statistical patterns of behavior. This focus is generally called complexity matching (e.g., Abney, Paxton, Kello, & Dale, under revision; Marmelat & Delignieres, 2012), contrasting with the behavior matching prevalent in traditional convergence research. While many behaviors targeted for study in behavior matching are overt and perceptible to others during interactions, complexity matching focuses on convergence at the distributional level of conversational properties. Complexity matching is a special case of convergence of distribution-level patterns of behaviors: It captures the matching of behaviors that follow power law distributions. Power law distributions are indicative of multiscale variations and are exhibited by complex systems (Sales- Pardo, Guimera, Moreira, & Amaral, 2007), hence the use of the term complexity matching to quantify the matching of these properties across two people in an interaction. The notion of complexity matching of two humans interacting is suggested by recent research showing that when the power law distributions of interacting complex systems match, optimal information transmission occurs (West, Geneston, & Grigolini, 2008). Therefore, we hypothesize that when the behaviors of two humans follow a power law distribution, the degree of matching between these quantitative patterns might reflect properties of the interaction like information flow, context, and valence. While the framework of behavior matching quantifies the one-to-one matching of behaviors during an interaction (e.g., gaze patterns; Louwerse et al., 2012), complexity matching quantifies the degree to which particular statistical patterns (e.g., patterns of behavior that are power-law distributed) match throughout an entire interaction. Thus, behavior matching and complexity matching are complementary measures of interpersonal convergence. In the present study, we use both behavior and complexity matching to create networks of speech and movement in dyadic interaction during a cooperative task.