Touching words is not enough: How visual experience influences haptic–auditory associations in the “Bouba–Kiki” effect

Since Köhler's experiments in the 1920s, researchers have demonstrated a correspondence between words and shapes. Dubbed the "Bouba-Kiki" effect, these auditory-visual associations extend across cultures and are thought to be universal. More recently the effect has been shown in other modalities including taste, suggesting the effect is independent of vision. The study presented here tested the "Bouba-Kiki" effect in the auditory-haptic modalities, using 2D cut-outs and 3D models based on Köhler's original drawings. Presented with shapes they could feel but not see, sighted participants showed a robust "Bouba-Kiki" effect. However, in a sample of people with a range of visual impairments, from congenital total blindness to partial sight, the effect was significantly less pronounced. The findings suggest that, in the absence of a direct visual stimulus, visual imagery plays a role in crossmodal integration.

[1]  C. Spence Managing sensory expectations concerning products and brands: Capitalizing on the potential of sound and shape symbolism , 2012 .

[2]  R. Davis The fitness of names to drawings. A cross-cultural study in Tanganyika. , 1961, British journal of psychology.

[3]  A. Vouloumanos,et al.  Sound symbolism in infancy: evidence for sound-shape cross-modal correspondences in 4-month-olds. , 2013, Journal of experimental child psychology.

[4]  Amedeo D'Angiulli,et al.  The effects of interference on recognition of haptic pictures in blindfolded sighted participants: the modality of representation of haptic information. , 2012, Scandinavian journal of psychology.

[5]  Kurt E. Weaver,et al.  Functional characteristics of auditory cortex in the blind , 2009, Behavioural Brain Research.

[6]  Charles Spence,et al.  ‘Bouba’ and ‘Kiki’ in Namibia? A remote culture make similar shape–sound matches, but different shape–taste matches to westerners , 2013 .

[7]  Edward M. Hubbard,et al.  Neurocognitive Mechanisms of Synesthesia , 2005, Neuron.

[8]  Charles Spence,et al.  ‘The Sweet Taste of Maluma’: Crossmodal Associations Between Tastes and Words , 2012, Chemosensory Perception.

[9]  C. Spence,et al.  On the taste of “Bouba” and “Kiki”: An exploration of word–food associations in neurologically normal participants , 2011, Cognitive neuroscience.

[10]  Vilayanur S Ramachandran,et al.  Preliminary evidence for deficits in multisensory integration in autism spectrum disorders: The mirror neuron hypothesis , 2008, Social neuroscience.

[11]  A. Paivio Imagery and verbal processes , 1972 .

[12]  E. Zohary,et al.  Transcranial magnetic stimulation of the occipital pole interferes with verbal processing in blind subjects , 2004, Nature Neuroscience.

[13]  Amir Amedi,et al.  Multisensory visual–tactile object related network in humans: insights gained using a novel crossmodal adaptation approach , 2009, Experimental Brain Research.

[14]  P. Huttenlocher Neural Plasticity: The Effects of Environment on the Development of the Cerebral Cortex , 2002 .

[15]  R. Zatorre,et al.  Crossmodal recruitment of primary visual cortex following brief exposure to bimodal audiovisual stimuli , 2010, Neuropsychologia.

[16]  R. Hobson,et al.  The pathogenesis of autism: insights from congenital blindness. , 2003, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[17]  C. Spence,et al.  “Bouba” and “Kiki” in Namibia? A remote culture make similar shape–sound matches, but different shape–taste matches to Westerners , 2013, Cognition.

[18]  Albert Postma,et al.  Reference Frame Preferences in Haptics Differ for the Blind and Sighted in the Horizontal but Not in the Vertical Plane , 2011, Perception.

[19]  Jill Whitehead,et al.  What is audio description , 2005 .

[20]  Amir Amedi,et al.  A Putative Model of Multisensory Object Representation , 2009, Brain Topography.

[21]  F. Rösler,et al.  Altered auditory-tactile interactions in congenitally blind humans: an event-related potential study , 2004, Experimental Brain Research.

[22]  B. Röder,et al.  Basic Multisensory Functions Can Be Acquired After Congenital Visual Pattern Deprivation in Humans , 2012, Developmental neuropsychology.

[23]  M. Ryan,et al.  What do animal signals mean? , 2009, Animal Behaviour.

[24]  F. Rösler,et al.  Early visual deprivation impairs multisensory interactions in humans , 2007, Nature Neuroscience.

[25]  D. Rendall,et al.  The sound of round: evaluating the sound-symbolic role of consonants in the classic Takete-Maluma phenomenon. , 2011, Canadian journal of experimental psychology = Revue canadienne de psychologie experimentale.

[26]  M. Heller,et al.  Superior Haptic Perceptual Selectivity in Late-Blind and Very-Low-Vision Subjects , 2003, Perception.

[27]  V. Ramachandran,et al.  Synaesthesia? A window into perception, thought and language , 2001 .

[28]  A Koriat,et al.  Figural symbolism in Chinese ideographs , 1979, Journal of psycholinguistic research.

[29]  C. Spence,et al.  Spatial coordinate systems for tactile spatial attention depend on developmental vision: evidence from event‐related potentials in sighted and congenitally blind adult humans , 2008, The European journal of neuroscience.

[30]  David C. Reutens,et al.  Early but not late-blindness leads to enhanced auditory perception , 2010, Neuropsychologia.

[31]  Alvaro Pascual-Leone,et al.  Behavioral and neuroplastic changes in the blind: evidence for functionally relevant cross-modal interactions , 2004, Journal of Physiology-Paris.

[32]  C. Spence,et al.  Tasting shapes and words , 2011 .

[33]  Asher Koriat,et al.  The symbolic implications of vowels and of their orthographic representations in two natural languages , 1977 .

[34]  Federico Fontana,et al.  Association of Haptic Trajectories to Takete and Maluma , 2013, HAID.

[35]  Stephan Gramley,et al.  Cross-Cultural Pragmatics , 2008 .

[36]  Yong Liu,et al.  Altered Anatomical Network in Early Blindness Revealed by Diffusion Tensor Tractography , 2009, PloS one.

[37]  D. Maurer,et al.  The shape of boubas: sound-shape correspondences in toddlers and adults. , 2006, Developmental science.

[38]  Network 1000: Surveying the changing needs and circumstances of visually impaired adults in Great Britain , 2005 .

[39]  T. Vecchi,et al.  Blind Vision: The Neuroscience of Visual Impairment , 2011 .

[40]  Ophelia Deroy,et al.  Tasting Liquid Shapes: Investigating the Sensory Basis of Cross-modal Correspondences , 2011 .

[41]  Charles Spence,et al.  Assessing the shapes and speech sounds that people associate with chocolate samples varying in cocoa content , 2011 .

[42]  M. Lassonde,et al.  Cross-modal plasticity for the spatial processing of sounds in visually deprived subjects , 2008, Experimental Brain Research.

[43]  S. Palazzi Autism and Blindness. Research and Reflections , 2005 .