Impact of sound on image-evoked emotions

In two experiments the effect of sound on visual information was investigated. In Experiment 1 the effect of the visual appearance of product types with an expensive deign and with an inexpensive design on the experience of the sound recordings of these products was investigated. Recordings and pictures were systematically interchanged. Thus, for example, the visual image of an expensive design was combined with a recording of the sound of an inexpensive and of an expensive design. It was found that product appearance did not affect the judgment on luxury, pleasantness, quality, and ease-of-use but that the experience of the sound dominated over the visual experience. In Experiment 2, pictures from the international affective pictures set were combined with frequency-modulated tones that varied in the amount of sensory pleasantness by manipulating the amount of roughness. The combination of sounds and pictures were rated on the valence and arousal dimensions of the circumplex model of core affect. It was found that the sounds only negatively affected the experience of the pictures on the valence dimension. The arousal level was not affected by the sounds. Both experiments show that sound can affect the perception and experience of pictures.

[1]  M. Kutas,et al.  Combined perception of emotion in pictures and musical sounds , 2006, Brain Research.

[2]  H. Schifferstein,et al.  Visualising Fragrances through Colours: The Mediating Role of Emotions , 2004, Perception.

[3]  Andrew Ortony,et al.  The Cognitive Structure of Emotions , 1988 .

[4]  Judy Edworthy,et al.  Warning design : a research prospective , 1996 .

[5]  René van Egmond,et al.  Emotional experience of Frequency Modulated sounds: Implications for the design of alarm sounds , 2004 .

[6]  R. Plutchik Emotion, a psychoevolutionary synthesis , 1980 .

[7]  M. Posner,et al.  Visual dominance: an information-processing account of its origins and significance. , 1976, Psychological review.

[8]  P. Ekman Universals and cultural differences in facial expressions of emotion. , 1972 .

[9]  René van Egmond,et al.  3 – THE EXPERIENCE OF PRODUCT SOUNDS , 2008 .

[10]  Frank E. Block,et al.  Optimization of Alarms: A Study on Alarm Limits, Alarm Sounds, and False Alarms, Intended to Reduce Annoyance , 1999, Journal of Clinical Monitoring and Computing.

[11]  C. Spence,et al.  Audiotactile interactions in roughness perception , 2002, Experimental Brain Research.

[12]  H. Schifferstein The perceived importance of sensory modalities in product usage: a study of self-reports. , 2006, Acta psychologica.

[13]  V. Jousmäki,et al.  Parchment-skin illusion: sound-biased touch , 1998, Current Biology.

[14]  C Spence,et al.  The Role of Auditory Cues in Modulating the Perception of Electric Toothbrushes , 2003, Journal of dental research.

[15]  J. Russell,et al.  Evidence for a three-factor theory of emotions , 1977 .

[16]  Seiichiro Namba,et al.  Subjective evaluation of car door sound , 2002 .

[17]  Charles Spence,et al.  Multisensory product experience , 2008 .

[18]  Rosalind W. Picard Toward computers that recognize and respond to user emotion , 2000, IBM Syst. J..

[19]  M. Bradley,et al.  Measuring emotion: the Self-Assessment Manikin and the Semantic Differential. , 1994, Journal of behavior therapy and experimental psychiatry.

[20]  J. Russell A circumplex model of affect. , 1980 .

[21]  Karin Bijsterveld,et al.  Mechanical Sound: Technology, Culture, and Public Problems of Noise in theTwentieth Century , 2008 .

[22]  Guyot Frédérique,et al.  Characterization of the sound quality of vacuum cleaners , 1995 .

[23]  C. Spence,et al.  Auditory contributions to multisensory product perception , 2006 .

[24]  Paul Boersma,et al.  Praat: doing phonetics by computer , 2003 .

[25]  Ren√© Egmond,et al.  Compact disc cover design: transfer of musical style and emotional contents into graphical design , 2003 .

[26]  ACTIVE SOUND DESIGN: VACUUM CLEANER , 2002 .

[27]  Ira J. Roseman Appraisal determinants of discrete emotions. , 1991 .

[28]  P. Desmet,et al.  15 – PRODUCT EMOTION , 2008 .

[29]  Scott D. Lipscomb,et al.  Perceptual judgement of the relationship between musical and visual components in film. , 1994 .

[30]  K. Hevner Expression in music: a discussion of experimental studies and theories. , 1935 .

[31]  H. Schifferstein,et al.  Capturing product experiences: a split-modality approach. , 2005, Acta psychologica.