Contingent sounds change the mental representation of one’s finger length

Mental body-representations are highly plastic and can be modified after brief exposure to unexpected sensory feedback. While the role of vision, touch and proprioception in shaping body-representations has been highlighted by many studies, the auditory influences on mental body-representations remain poorly understood. Changes in body-representations by the manipulation of natural sounds produced when one’s body impacts on surfaces have recently been evidenced. But will these changes also occur with non-naturalistic sounds, which provide no information about the impact produced by or on the body? Drawing on the well-documented capacity of dynamic changes in pitch to elicit impressions of motion along the vertical plane and of changes in object size, we asked participants to pull on their right index fingertip with their left hand while they were presented with brief sounds of rising, falling or constant pitches, and in the absence of visual information of their hands. Results show an “auditory Pinocchio” effect, with participants feeling and estimating their finger to be longer after the rising pitch condition. These results provide the first evidence that sounds that are not indicative of veridical movement, such as non-naturalistic sounds, can induce a Pinocchio-like change in body-representation when arbitrarily paired with a bodily action.

[1]  C. C. Pratt The spatial character of high and low tones. , 1930 .

[2]  R. Butler,et al.  Localization of tonal stimuli in the vertical plane. , 1968, The Journal of the Acoustical Society of America.

[3]  Jacob Cohen Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[4]  G. Lakoff,et al.  Metaphors We Live by , 1982 .

[5]  L E Marks,et al.  Perceiving similarity and comprehending metaphor. , 1988, Monographs of the Society for Research in Child Development.

[6]  J. Lackner Some proprioceptive influences on the perceptual representation of body shape and orientation. , 1988, Brain : a journal of neurology.

[7]  P. Lachenbruch Statistical Power Analysis for the Behavioral Sciences (2nd ed.) , 1989 .

[8]  Jonathan D. Cohen,et al.  Rubber hands ‘feel’ touch that eyes see , 1998, Nature.

[9]  V. Jousmäki,et al.  Parchment-skin illusion: sound-biased touch , 1998, Current Biology.

[10]  R. Passingham,et al.  That's My Hand! Activity in Premotor Cortex Reflects Feeling of Ownership of a Limb , 2004, Science.

[11]  Daphne Maurer,et al.  Do small white balls squeak? Pitch-object correspondences in young children , 2004, Cognitive, affective & behavioral neuroscience.

[12]  Ryota Kanai,et al.  Changing pitch induced visual motion illusion , 2004, Current Biology.

[13]  Eiichi Naito,et al.  Neural Substrate of Body Size: Illusory Feeling of Shrinking of the Waist , 2005, PLoS biology.

[14]  G. Barsh,et al.  Effects of Hypothalamic Neurodegeneration on Energy Balance , 2005, PLoS biology.

[15]  P. Haggard,et al.  Bodily Illusions Modulate Tactile Perception , 2005, Current Biology.

[16]  S. Rössner,et al.  BMI, waist-circumference and waist-hip-ratio as diagnostic tests for fatness in adolescents , 2005, International Journal of Obesity.

[17]  Bruno L. Giordano,et al.  Spatial representation of pitch height: the SMARC effect , 2006, Cognition.

[18]  Charles Spence,et al.  Multisensory synesthetic interactions in the speeded classification of visual size , 2006, Perception & psychophysics.

[19]  Wolfgang Sanz,et al.  Laser-optical investigation of stator-rotor interaction in a transonic turbine , 2006, J. Vis..

[20]  M. Tsakiris,et al.  Hands only illusion: multisensory integration elicits sense of ownership for body parts but not for non-corporeal objects , 2010, Experimental Brain Research.

[21]  P. Haggard,et al.  Distorting the visual size of the hand affects hand pre-shaping during grasping , 2009, Experimental Brain Research.

[22]  Joost X. Maier,et al.  Natural, Metaphoric, and Linguistic Auditory Direction Signals Have Distinct Influences on Visual Motion Processing , 2009, The Journal of Neuroscience.

[23]  Charles Spence,et al.  Compatibility effects between sound frequency and tactile elevation , 2009, Neuroreport.

[24]  M. Tsakiris My body in the brain: A neurocognitive model of body-ownership , 2010, Neuropsychologia.

[25]  C. McCabe,et al.  Body Perception Disturbance (BPD) in CRPS , 2010 .

[26]  Anne Treisman,et al.  Natural cross-modal mappings between visual and auditory features. , 2011, Journal of vision.

[27]  Patrick Haggard,et al.  An implicit body representation underlying human position sense , 2010, Proceedings of the National Academy of Sciences.

[28]  P Haggard,et al.  Increased plasticity of the bodily self in eating disorders , 2011, Psychological Medicine.

[29]  H. Henrik Ehrsson,et al.  Being Barbie: The Size of One’s Own Body Determines the Perceived Size of the World , 2011, PloS one.

[30]  Marcia Grabowecky,et al.  Changes in auditory frequency guide visual–spatial attention , 2011, Cognition.

[31]  Marjolein P. M. Kammers,et al.  Visual Distortion of Body Size Modulates Pain Perception , 2011, Psychological science.

[32]  Charles Spence,et al.  Does crossmodal correspondence modulate the facilitatory effect of auditory cues on visual search? , 2012, Attention, Perception, & Psychophysics.

[33]  P. Walker,et al.  A Common Scheme for Cross-Sensory Correspondences across Stimulus Domains , 2012, Perception.

[34]  T. Wheatley,et al.  Associations between Auditory Pitch and Visual Elevation Do Not Depend on Language: Evidence from a Remote Population , 2012, Perception.

[35]  M. Tsakiris,et al.  Action sounds recalibrate perceived tactile distance , 2012, Current Biology.

[36]  Daniel Casasanto,et al.  The Sound of Thickness: Prelinguistic Infants' Associations of Space and Pitch , 2012, CogSci.

[37]  Anina N. Rich,et al.  Cross-Modality Correspondence between Pitch and Spatial Location Modulates Attentional Orienting , 2012, Perception.

[38]  Alisa Mandrigin,et al.  Bodily ownership and self-location: Components of bodily self-consciousness , 2013, Consciousness and Cognition.

[39]  A. Majid,et al.  The Thickness of Musical Pitch , 2013, Psychological science.

[40]  Giulio Rosati,et al.  On the Role of Auditory Feedback in Robot-Assisted Movement Training after Stroke: Review of the Literature , 2013, Comput. Intell. Neurosci..

[41]  Sabina Pauen,et al.  Cross-modal mapping of visual and acoustic displays in infants: The effect of dynamic and static components , 2013 .

[42]  M. Slater,et al.  Illusory ownership of a virtual child body causes overestimation of object sizes and implicit attitude changes , 2013, Proceedings of the National Academy of Sciences.

[43]  Zohar Eitan,et al.  How pitch and loudness shape musical space and motion , 2013 .

[44]  Ophelia Deroy,et al.  How automatic are crossmodal correspondences? , 2013, Consciousness and Cognition.

[45]  Lawrence E Marks,et al.  Lower pitch is larger, yet falling pitches shrink. , 2014, Experimental psychology.

[46]  Ana Tajadura-Jiménez,et al.  Motivating people with chronic pain to do physical activity: opportunities for technology design , 2014, CHI.

[47]  Agnès Roby-Brami,et al.  Toward the use of augmented auditory feedback for the rehabilitation of arm movements in stroke patients , 2014 .

[48]  N. Bolognini,et al.  The Marble-Hand Illusion , 2013, PloS one.

[49]  Marc O Ernst,et al.  Natural auditory scene statistics shapes human spatial hearing , 2014, Proceedings of the National Academy of Sciences.

[50]  C. Spence,et al.  Modifying action sounds influences people's emotional responses and bodily sensations , 2014, i-Perception.

[51]  Ioannis Paraskevopoulos,et al.  Design guidelines for developing customised serious games for Parkinson's Disease rehabilitation using bespoke game sensors , 2014, Entertain. Comput..

[52]  M. Tsakiris,et al.  Action sounds update the mental representation of arm dimension: contributions of kinaesthesia and agency , 2015, Front. Psychol..

[53]  Myounghoon Jeon,et al.  Menu Navigation With In-Vehicle Technologies: Auditory Menu Cues Improve Dual Task Performance, Preference, and Workload , 2015, Int. J. Hum. Comput. Interact..

[54]  Ana Tajadura-Jiménez,et al.  As Light as your Footsteps: Altering Walking Sounds to Change Perceived Body Weight, Emotional State and Gait , 2015, CHI.

[55]  P. Walker,et al.  Cross-sensory mapping of feature values in the size-brightness correspondence can be more relative than absolute. , 2016, Journal of experimental psychology. Human perception and performance.

[56]  Demetrios Karis,et al.  Improving Remote Collaboration With Video Conferencing and Video Portals , 2016, Hum. Comput. Interact..

[57]  Ana Tajadura-Jiménez,et al.  Go-with-the-Flow: Tracking, Analysis and Sonification of Movement and Breathing to Build Confidence in Activity Despite Chronic Pain , 2016, Hum. Comput. Interact..

[58]  Vincent Hayward,et al.  Differentiated audio-tactile correspondences in sighted and blind individuals. , 2016, Journal of experimental psychology. Human perception and performance.

[59]  Laurence R Harris,et al.  Which Direction Is up for a High Pitch? , 2016, Multisensory research.

[60]  D. Swapp,et al.  Action Sounds Modulate Arm Reaching Movements , 2016, Front. Psychol..

[61]  Sally A. Linkenauger,et al.  Multimodal contributions to body representation , 2016 .

[62]  J. Navarra,et al.  The higher the pitch the larger its crossmodal influence on visuospatial processing , 2017 .