Composing with Cross-modal Correspondences: Music and Odors in Concert

We report two experiments designed to investigate cross-modal correspondences between a range of seven olfactory stimuli and both the pitch and instrument class of sounds as well as the angularity of visually presented shapes. The results revealed that odors were preferentially matched to musical features: For example, the odors of candied orange and iris flower were matched to significantly higher pitches than the odors of musk and roasted coffee. Meanwhile, the odor of crème brûlée was associated with a more rounded shape than the musk odor. Moreover, by simultaneously testing cross-modal correspondences between olfactory stimuli and matches in two other modalities, we were able to compare the ratings associated with each correspondence. Stimuli judged as happier, more pleasant, and sweeter tended to be associated to both higher pitch and a more rounded shape, whereas other ratings seemed to be more specifically correlated with the choice of either pitch or shape. Odors rated as more arousing tended to be associated with the angular shape, but not with a particular pitch; odors judged as brighter were associated with higher pitch and, to a lesser extent, rounder shapes. In a follow-up experiment, we investigated whether people could match specific pieces of music (composed to represent odors) to three of the odors (candied orange, crème brûlée, and ginger cookies). In one case (candied orange), a majority of the participants matched the odor to the intended piece of music. In another case (ginger cookies), another piece of music (than the one intended) was preferred. Finally, in the third case (crème brûlée), people showed no preference in matching the odor to the pieces of music. Both theoretical and practical implications of these results are discussed.

[1]  C. C. Pratt The spatial character of high and low tones. , 1930 .

[2]  Vowel Sounds in Poetry: Their Music and Tone Colour , 1942 .

[3]  B. Berglund,et al.  Multidimensional analysis of twenty-one odors. , 1973, Scandinavian journal of psychology.

[4]  L E Marks,et al.  On associations of light and sound: the mediation of brightness, pitch, and loudness. , 1974, The American journal of psychology.

[5]  R. Erickson,et al.  MULTIDIMENSIONAL SCALING OF ODORANTS: EXAMINATION OF PSYCHOLOGICAL AND PHYSICOCHEMICAL DIMENSIONS , 1977 .

[6]  B. Lyman Representation of Complex Emotional and Abstract Meanings by Simple Forms , 1979, Perceptual and motor skills.

[7]  G. Lakoff,et al.  Metaphors We Live By , 1980 .

[8]  Michael W. O'Boyle,et al.  Implications for phonetic symbolism: The relationship between pure tones and geometric figures , 1980, Journal of psycholinguistic research.

[9]  G. Lakoff,et al.  Metaphors We Live by , 1981 .

[10]  L E Marks,et al.  Perceiving similarity and comprehending metaphor. , 1988, Monographs of the Society for Research in Child Development.

[11]  Robert D. Melara,et al.  Interaction between synesthetically corresponding dimensions. , 1987 .

[12]  A. Gilbert,et al.  Cross-modal correspondence between vision and olfaction: the color of smells. , 1996, The American journal of psychology.

[13]  A. Gilbert,et al.  Auditory Pitch as a Perceptual Analogue to Odor Quality , 1997 .

[14]  Adrian C. North,et al.  In-store music affects product choice , 1997, Nature.

[15]  Adrian C. North,et al.  The influence of in-store music on wine selections. , 1999 .

[16]  Richard R. Klink Creating Brand Names With Meaning: The Use of Sound Symbolism , 2000 .

[17]  Katsumi Aoki,et al.  Recent development of flow visualization , 2004, J. Vis..

[18]  Charles Spence,et al.  Cross-modal associations between odors and colors. , 2006, Chemical senses.

[19]  M. L. Demattè,et al.  Cross-modal interactions between olfaction and touch. , 2006, Chemical senses.

[20]  M. Ernst Learning to integrate arbitrary signals from vision and touch. , 2007, Journal of vision.

[21]  M. Zarzo PSYCHOLOGIC DIMENSIONS IN THE PERCEPTION OF EVERYDAY ODORS: PLEASANTNESS AND EDIBILITY , 2008 .

[22]  Didier Grandjean,et al.  Mapping the semantic space for the subjective experience of emotional responses to odors. , 2008, Chemical senses.

[23]  Charles Spence,et al.  Grape expectations: The role of cognitive influences in color–flavor interactions , 2010, Consciousness and Cognition.

[24]  N. Sobel,et al.  An odor is not worth a thousand words: from multidimensional odors to unidimensional odor objects. , 2013, Annual review of psychology.

[25]  Anne Treisman,et al.  Natural cross-modal mappings between visual and auditory features. , 2011, Journal of vision.

[26]  J. Enns,et al.  Spatial selection and target identification are separable processes in visual search. , 2010, Journal of vision.

[27]  T. Hummel,et al.  Cross-modal integration between odors and abstract symbols , 2010, Neuroscience Letters.

[28]  C. Spence,et al.  As bitter as a trombone: Synesthetic correspondences in nonsynesthetes between tastes/flavors and musical notes , 2010, Attention, perception & psychophysics.

[29]  Charles Spence,et al.  Assessing the shapes and speech sounds that people associate with chocolate samples varying in cocoa content , 2011 .

[30]  C. Spence,et al.  Multisensory design: Reaching out to touch the consumer , 2011 .

[31]  C. Spence Crossmodal correspondences: A tutorial review , 2011, Attention, perception & psychophysics.

[32]  Karen B. Schloss,et al.  Color, Music, and Emotion , 2011 .

[33]  Bruno Mesz,et al.  The Taste of Music , 2011, Perception.

[34]  Ophelia Deroy,et al.  Tasting Liquid Shapes: Investigating the Sensory Basis of Cross-modal Correspondences , 2011 .

[35]  T. Hummel,et al.  Auditory-olfactory integration: congruent or pleasant sounds amplify odor pleasantness. , 2011, Chemical senses.

[36]  Georgios A. Keliris,et al.  Introduction to Research Topic – Binocular Rivalry: A Gateway to Studying Consciousness , 2012, Front. Hum. Neurosci..

[37]  C. Spence,et al.  A bittersweet symphony: Systematically modulating the taste of food by changing the sonic properties of the soundtrack playing in the background , 2012 .

[38]  M. Sigman,et al.  A composition algorithm based on crossmodal taste-music correspondences , 2012, Front. Hum. Neurosci..

[39]  P. Walker,et al.  A Common Scheme for Cross-Sensory Correspondences across Stimulus Domains , 2012, Perception.

[40]  Adrian C. North,et al.  The effect of background music on the taste of wine. , 2012, British journal of psychology.

[41]  Anina N. Rich,et al.  The Nature and Origin of Cross-Modal Associations to Odours , 2012, Perception.

[42]  C. Spence,et al.  Assessing the appropriateness of ‘synaesthetic’ messaging on crisps packaging , 2012 .

[43]  C. Spence,et al.  A fruity note: crossmodal associations between odors and musical notes. , 2012, Chemical senses.

[44]  Charles Spence,et al.  Auditory contributions to flavour perception and feeding behaviour , 2012, Physiology & Behavior.

[45]  C. Spence,et al.  Smelling shapes: crossmodal correspondences between odors and shapes. , 2013, Chemical senses.

[46]  M. Jacquot,et al.  Contribution to understanding odour–colour associations , 2013 .

[47]  Charles Spence,et al.  ‘Bouba’ and ‘Kiki’ in Namibia? A remote culture make similar shape–sound matches, but different shape–taste matches to westerners , 2013 .

[48]  Ophelia Deroy,et al.  “Having a drink in a bar”: An immersive approach to explore the effects of context on drink choice , 2013 .