Affective acoustic ecology: towards emotionally enhanced sound events

Sound events can carry multiple information, related to the sound source and to ambient environment. However, it is well-known that sound evokes emotions, a fact that is verified by works in the disciplines of Music Emotion Recognition and Music Information Retrieval that focused on the impact of music to emotions. In this work we introduce the concept of affective acoustic ecology that extends the above relation to the general concept of sound events. Towards this aim, we define sound event as a novel audio structure with multiple components. We further investigate the application of existing emotion models employed for music affective analysis to sonic, non-musical, content. The obtained results indicate that although such application is feasible, no significant trends and classification outcomes are observed that would allow the definition of an analytic relation between the technical characteristics of a sound event waveform and raised emotions.

[1]  M. Marcell,et al.  Identifying, rating, and remembering environmental sound events , 2007, Behavior research methods.

[2]  B. Newman,et al.  And now, acoustic ecology. , 2005, Optometry.

[3]  Leonard J. Paul,et al.  Granulation of Sound in Video Games , 2011 .

[4]  K. Hevner.  Experimental studies of the elements of expression in music , 1936 .

[5]  Petri Toiviainen,et al.  Exploring relationships between audio features and emotion in music , 2009 .

[6]  M. Bradley,et al.  Measuring emotion: the Self-Assessment Manikin and the Semantic Differential. , 1994, Journal of behavior therapy and experimental psychiatry.

[7]  Piotr Synak,et al.  Extracting Emotions from Music Data , 2005, ISMIS.

[8]  Urszula Markowska-Kaczmar,et al.  Extraction of Emotional Content from Music Data , 2008, 2008 7th Computer Information Systems and Industrial Management Applications.

[9]  M J Gang,et al.  Individual differences in heart rate responses to affective sound. , 1975, Psychophysiology.

[10]  Paul Jennings,et al.  The development and application of the emotional dimensions of a soundscape , 2013 .

[11]  Mark Grimshaw,et al.  A climate of fear: considerations for designing a virtual acoustic ecology of fear , 2011, AM '11.

[12]  Andrzej Czyzewski,et al.  Representing Musical Instrument Sounds for Their Automatic Classification , 2001 .

[13]  John E. W. Poxon,et al.  An exploratory evaluation of perceptual, psychoacoustic and acoustical properties of urban soundscapes , 2013 .

[14]  Lie Lu,et al.  Automatic mood detection and tracking of music audio signals , 2006, IEEE Transactions on Audio, Speech, and Language Processing.

[15]  Shiliang Zhang,et al.  Affective MTV analysis based on arousal and valence features , 2008, 2008 IEEE International Conference on Multimedia and Expo.

[16]  Yueting Zhuang,et al.  Popular music retrieval by detecting mood , 2003, SIGIR.

[17]  K. Scherer.  Which Emotions Can be Induced by Music? What Are the Underlying Mechanisms? And How Can We Measure Them? , 2004 .

[18]  Yi-Hsuan Yang,et al.  Music emotion classification: a fuzzy approach , 2006, MM '06.

[19]  K. MacDorman,et al.  Automatic Emotion Prediction of Song Excerpts: Index Construction, Algorithm Design, and Empirical Comparison , 2007 .

[20]  A. Damasio,et al.  Basic emotions are associated with distinct patterns of cardiorespiratory activity. , 2006, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[21]  William W. Gaver.  What in the World Do We Hear? An Ecological Approach to Auditory Event Perception , 1993 .

[22]  A. Ortony,et al.  What's basic about basic emotions? , 1990, Psychological review.

[23]  Yi-Hsuan Yang,et al.  Mr. Emo: music retrieval in the emotion plane , 2008, ACM Multimedia.

[24]  R. M. Schafer,et al.  The tuning of the world , 1977 .

[25]  Zhang Naiyao,et al.  User-adaptive music emotion recognition , 2004, Proceedings 7th International Conference on Signal Processing, 2004. Proceedings. ICSP '04. 2004..

[26]  Craig A. Smith,et al.  Patterns of cognitive appraisal in emotion. , 1985, Journal of personality and social psychology.

[27]  Junqing Yu,et al.  An improved valence-arousal emotion space for video affective content representation and recognition , 2009, 2009 IEEE International Conference on Multimedia and Expo.

[28]  Tao Li,et al.  Content-based music similarity search and emotion detection , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[29]  Bram van de Laar.  Emotion detection in music, a survey , 2006 .

[30]  Roberto Bresin,et al.  Sound Design for Affective Interaction , 2007, ACII.

[31]  Ichiro Fujinaga,et al.  jAudio: An Feature Extraction Library , 2005, ISMIR.

[32]  J. Russell.  A circumplex model of affect. , 1980 .

[33]  Craig Hedge,et al.  The Immersive Virtual Environment of the digital fulldome: Considerations of relevant psychological processes , 2012, Int. J. Hum. Comput. Stud..

[34]  Tao Li,et al.  Detecting emotion in music , 2003, ISMIR.

[35]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[36]  Daniel Leech-Wilkinson,et al.  European Society for the Cognitive Sciences of Music (ESCOM) , 1991 .

[37]  Chia-Hung Yeh,et al.  An efficient emotion detection scheme for popular music , 2009, 2009 IEEE International Symposium on Circuits and Systems.

[38]  P. Laukka,et al.  Communication of emotions in vocal expression and music performance: different channels, same code? , 2003, Psychological bulletin.

[39]  Randolph R. Cornelius.  THEORETICAL APPROACHES TO EMOTION , 2000 .

[40]  M. Bradley,et al.  Affective reactions to acoustic stimuli. , 2000, Psychophysiology.

[41]  K. Scherer,et al.  Vocal expression of affect , 2005 .

[42]  Lie Lu,et al.  Automatic mood detection from acoustic music data , 2003, ISMIR.

[43]  P. Ekman.  An argument for basic emotions , 1992 .