Action sounds update the mental representation of arm dimension: contributions of kinaesthesia and agency

Auditory feedback accompanies almost all our actions, but its contribution to body-representation is understudied. Recently it has been shown that the auditory distance of action sounds recalibrates perceived tactile distances on one’s arm, suggesting that action sounds can change the mental representation of arm length. However, the question remains open of what factors play a role in this recalibration. In this study we investigate two of these factors, kinaesthesia, and sense of agency. Across two experiments, we asked participants to tap with their arm on a surface while extending their arm. We manipulated the tapping sounds to originate at double the distance to the tapping locations, as well as their synchrony to the action, which is known to affect feelings of agency over the sounds. Kinaesthetic cues were manipulated by having additional conditions in which participants did not displace their arm but kept tapping either close (Experiment 1) or far (Experiment 2) from their body torso. Results show that both the feelings of agency over the action sounds and kinaesthetic cues signaling arm displacement when displacement of the sound source occurs are necessary to observe changes in perceived tactile distance on the arm. In particular, these cues resulted in the perceived tactile distances on the arm being felt smaller, as compared to distances on a reference location. Moreover, our results provide the first evidence of consciously perceived changes in arm-representation evoked by action sounds and suggest that the observed changes in perceived tactile distance relate to experienced arm elongation. We discuss the observed effects in the context of forward internal models of sensorimotor integration. Our results add to these models by showing that predictions related to action sounds must fit with kinaesthetic cues in order for auditory inputs to change body-representation.

[1]  M. F. Nolan,et al.  Two-point discrimination assessment in the upper limb in young adult men and women. , 1982, Physical therapy.

[2]  M. F. Nolan,et al.  Quantitative measure of cutaneous sensation. Two-point discrimination values for the face and trunk. , 1985, Physical therapy.

[3]  M. F. Nolan,et al.  Quantitative Measure of Cutaneous Sensation , 1985 .

[4]  Robert J. Logan,et al.  Perception of acoustic source characteristics: walking sounds. , 1991, The Journal of the Acoustical Society of America.

[5]  Durand R. Begault,et al.  3-D Sound for Virtual Reality and Multimedia Cambridge , 1994 .

[6]  Michael I. Jordan,et al.  An internal model for sensorimotor integration. , 1995, Science.

[7]  D. Lewkowicz Perception of auditory-visual temporal synchrony in human infants. , 1996, Journal of experimental psychology. Human perception and performance.

[8]  Jonathan D. Cohen,et al.  Rubber hands ‘feel’ touch that eyes see , 1998, Nature.

[9]  David J. Lewkowicz,et al.  Chapter 16 The development of temporal and spatial intermodal perception , 1999 .

[10]  Craig D. Murray,et al.  Presence Accompanying Induced Hearing Loss: Implications for Immersive Virtual Environments , 2000, Presence: Teleoperators & Virtual Environments.

[11]  Zoubin Ghahramani,et al.  Computational principles of movement neuroscience , 2000, Nature Neuroscience.

[12]  D. Wolpert,et al.  Abnormalities in the awareness of action , 2002, Trends in Cognitive Sciences.

[13]  R. Passingham,et al.  That's My Hand! Activity in Premotor Cortex Reflects Feeling of Ownership of a Limb , 2004, Science.

[14]  H. Bülthoff,et al.  Merging the senses into a robust percept , 2004, Trends in Cognitive Sciences.

[15]  S. Nishida,et al.  Recalibration of audiovisual simultaneity , 2004, Nature Neuroscience.

[16]  P. Bertelson,et al.  Recalibration of temporal order perception by exposure to audio-visual asynchrony. , 2004, Brain research. Cognitive brain research.

[17]  Nicholas P. Holmes,et al.  The body schema and multisensory representation(s) of peripersonal space , 2004, Cognitive Processing.

[18]  P. Haggard,et al.  Keeping the world a constant size: object constancy in human touch , 2004, Nature Neuroscience.

[19]  A. Maravita,et al.  Tools for the body (schema) , 2004, Trends in Cognitive Sciences.

[20]  Eiichi Naito,et al.  Neural Substrate of Body Size: Illusory Feeling of Shrinking of the Waist , 2005, PLoS biology.

[21]  P. Haggard,et al.  Bodily Illusions Modulate Tactile Perception , 2005, Current Biology.

[22]  Kari Kallinen,et al.  Comparing speakers versus headphones in listening to news from a computer - individual differences and psychophysiological responses , 2004, Comput. Hum. Behav..

[23]  Stella F. Lourenco,et al.  Space perception and body morphology: extent of near space scales with arm length , 2007, Experimental Brain Research.

[24]  P. Haggard,et al.  Viewing the body modulates tactile receptive fields , 2007, Experimental Brain Research.

[25]  S. Aglioti,et al.  Neural Underpinnings of Gesture Discrimination in Patients with Limb Apraxia , 2008, The Journal of Neuroscience.

[26]  Daniel Västfjäll,et al.  Self-Representation in Mediated Environments: The Experience of Emotions Modulated by Auditory-Vibrotactile Heartbeat , 2008, Cyberpsychology Behav. Soc. Netw..

[27]  A. Roy,et al.  Tool-use induces morphological updating of the body schema , 2009, Current Biology.

[28]  David Lagnado,et al.  Feelings of control: Contingency determines experience of action , 2009, Cognition.

[29]  P. Haggard,et al.  More than skin deep: Body representation beyond primary somatosensory cortex , 2010, Neuropsychologia.

[30]  P. Haggard,et al.  Touch and the body , 2010, Neuroscience & Biobehavioral Reviews.

[31]  M. Vetterli,et al.  Feeling in control of your footsteps: Conscious gait monitoring and the auditory consequences of footsteps , 2010, Cognitive neuroscience.

[32]  S. Aglioti,et al.  Representing actions through their sound , 2010, Experimental Brain Research.

[33]  Federico Fontana,et al.  Expressive sonification of footstep sounds , 2010 .

[34]  F. Vignemont Body schema and body image—Pros and cons , 2010, Neuropsychologia.

[35]  H. Henrik Ehrsson,et al.  Being Barbie: The Size of One’s Own Body Determines the Perceived Size of the World , 2011, PloS one.

[36]  P. Haggard,et al.  Weber's illusion and body shape: anisotropy of tactile size perception on the hand. , 2011, Journal of experimental psychology. Human perception and performance.

[37]  Sally A. Linkenauger,et al.  Taking a hands-on approach: apparent grasping ability scales the perception of object size. , 2011, Journal of experimental psychology. Human perception and performance.

[38]  Ana Tajadura-Jiménez,et al.  I-Space: The Effects of Emotional Valence and Source of Music on Interpersonal Distance , 2011, PloS one.

[39]  F. Mast,et al.  Vestibular stimulation modifies the body schema , 2012, Neuropsychologia.

[40]  Maria V. Sanchez-Vives,et al.  Extending Body Space in Immersive Virtual Reality: A Very Long Arm Illusion , 2012, PloS one.

[41]  M. Tsakiris,et al.  Action sounds recalibrate perceived tactile distance , 2012, Current Biology.

[42]  Alessandro Farnè,et al.  Grab an object with a tool and change your body: tool-use-dependent changes of body representation for action , 2012, Experimental Brain Research.

[43]  Roger Newport,et al.  How Long is Your Arm? Using Multisensory Illusions to Modify Body Image from the Third Person Perspective , 2012, Perception.

[44]  Valentina Rastelli,et al.  Tool-use reshapes the boundaries of body and peripersonal space representations , 2013, Experimental Brain Research.

[45]  P. Haggard,et al.  Vestibular contributions to bodily awareness , 2013, Neuropsychologia.

[46]  G. Verni,et al.  Amputation and prosthesis implantation shape body and peripersonal space representations , 2013, Scientific Reports.

[47]  Luke E. Miller,et al.  Tool morphology constrains the effects of tool use on body representations. , 2014, Journal of experimental psychology. Human perception and performance.

[48]  Ana Tajadura-Jiménez,et al.  Motivating people with chronic pain to do physical activity: opportunities for technology design , 2014, CHI.

[49]  N. Bolognini,et al.  The Marble-Hand Illusion , 2013, PloS one.

[50]  Ana Tajadura-Jiménez,et al.  Sonification of virtual and real surface tapping: evaluation of behavior changes, surface perception and emotional indices , 2015 .

[51]  Konrad Paul Kording,et al.  Over my fake body: body ownership illusions for studying the multisensory basis of own-body perception , 2015, Front. Hum. Neurosci..

[52]  Ana Tajadura-Jiménez,et al.  Sonification of Surface Tapping Changes Behavior, Surface Perception, and Emotion , 2015, IEEE MultiMedia.

[53]  Ana Tajadura-Jiménez,et al.  As Light as your Footsteps: Altering Walking Sounds to Change Perceived Body Weight, Emotional State and Gait , 2015, CHI.

[54]  A. Freud Cognitive Contributions To The Perception Of Spatial And Temporal Events , 2016 .