The Perceived Emotion of Isolated Synthetic Audio: The EmoSynth Dataset and Results

The ability of sound to enhance human wellbeing has been known since ancient civilisations, and methods can be found today across domains of health and within a variety of cultures. There are an abundance of sound-based methods which show benefits for both physical and mental-states of wellbeing. Current methods vary from low frequency vibrations to high frequency distractions, and from drone-like sustain to rhythmical pulsing, with limited knowledge of a listeners psycho-physical perception of this. In this regard, for the presented study 40 listeners were asked to evaluate the perceived emotional dimensions of Valence and Arousal from a dataset of 144 isolated synthetic periodic waveforms. Results show that Arousal does correlate moderately to fundamental frequency, and that the sine waveform is perceived as significantly different to square and sawtooth waveforms when evaluating perceived Arousal. The general results suggest that isolated synthetic audio can be modelled as a means of evoking affective states of emotion.

[1]  V. J. Krichagin.  Health effects of noise exposure , 1978 .

[2]  Heiga Zen,et al.  WaveNet: A Generative Model for Raw Audio , 2016, SSW.

[3]  Craig Zimring,et al.  Soundscape evaluations in two critical healthcare settings with different designs. , 2011, The Journal of the Acoustical Society of America.

[4]  Jussi Timonen,et al.  circulation and bone metabolism in frail old men and women Effects of a low-frequency sound wave therapy programme on functional capacity, blood , 2011 .

[5]  Björn W. Schuller,et al.  Stimulation of psychological listener experiences by semi-automatically composed electroacoustic environments , 2017, 2017 IEEE International Conference on Multimedia and Expo (ICME).

[6]  Meinard Mller,et al.  Fundamentals of Music Processing: Audio, Analysis, Algorithms, Applications , 2015 .

[7]  R. M. Schafer,et al.  The Soundscape: Our Sonic Environment and the Tuning of the World , 1993 .

[8]  S. Khalfa,et al.  Event-related skin conductance responses to musical emotions in humans , 2002, Neuroscience Letters.

[9]  Jian Kang,et al.  Towards standardization in soundscape preference assessment , 2011 .

[10]  I. Cuza,et al.  EFFECTS OF PHYSICAL AND VIBROACOUSTIC THERAPY IN CHRONIC PAIN IN JUVENILE ARTHRITIS , 2011 .

[11]  Elizabeth Grierson,et al.  Designing sound for health and wellbeing , 2012 .

[12]  Brigitte Schulte-Fortkamp,et al.  Soundscape and the Built Environment , 2015 .

[13]  P. Laukka,et al.  Expression, Perception, and Induction of Musical Emotions: A Review and a Questionnaire Study of Everyday Listening , 2004 .

[14]  S. Janssen,et al.  Auditory and non-auditory effects of noise on health , 2014, The Lancet.

[15]  Philip Samartzis,et al.  Original sound compositions reduce anxiety in emergency department patients: a randomised controlled trial , 2011, The Medical journal of Australia.

[16]  Meera Blattner,et al.  Earcons and Icons: Their Structure and Common Design Principles , 1989, Hum. Comput. Interact..

[17]  M. Gribaudo,et al.  2002 , 2001, Cell and Tissue Research.

[18]  M. Bradley,et al.  Affective reactions to acoustic stimuli. , 2000, Psychophysiology.

[19]  Charles Dodge,et al.  Computer Music: Synthesis, Composition, and Performance , 1997 .

[20]  Meinard Müller,et al.  Fundamentals of Music Processing , 2015, Springer International Publishing.

[21]  Daniël Lakens,et al.  Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and ANOVAs , 2013, Front. Psychol..

[22]  Marie Strand Skånland.  Music, health, and well-being , 2013, International journal of qualitative studies on health and well-being.

[23]  Mert Bay,et al.  The 2007 MIREX Audio Mood Classification Task: Lessons Learned , 2008, ISMIR.

[24]  Sheldon B. Michaels,et al.  Some Aspects of Fundamental Frequency and Envelope Amplitude as Related to the Emotional Content of Speech , 1962 .

[25]  J. Russell.  A circumplex model of affect. , 1980 .

[26]  J. Avner,et al.  Altered states of consciousness. , 2006, Pediatrics in review.

[27]  György Fazekas,et al.  Music Emotion Recognition: From Content- to Context-Based Models , 2012, CMMR.

[28]  Thierry Bertin-Mahieux,et al.  The Million Song Dataset , 2011, ISMIR.

[29]  J. Marozeau,et al.  Multidimensional scaling of emotional responses to music: The effect of musical expertise and of the duration of the excerpts , 2005 .

[30]  Alice C. Eldridge,et al.  Systematic literature review on the association between soundscape and ecological/human wellbeing , 2018 .

[31]  Lee Bartel,et al.  The effect of low-frequency sound stimulation on patients with fibromyalgia: A clinical study , 2015, Pain research & management.

[32]  H. M. Miller,et al.  History of Western music , 1991 .

[33]  A. James.  2010 , 2011, Philo of Alexandria: an Annotated Bibliography 2007-2016.

[34]  Björn W. Schuller,et al.  iHEARu-PLAY: Introducing a game for crowdsourced data collection for affective computing , 2015, 2015 International Conference on Affective Computing and Intelligent Interaction (ACII).

[35]  Barbara J. Crowe,et al.  An Overview of Sound Healing Practices: Implications for the Profession of Music Therapy , 1996 .

[36]  A. L. Brown,et al.  Soundscapes and environmental noise management , 2010 .

[37]  Sercan Ömer Arik,et al.  Deep Voice 3: Scaling Text-to-Speech with Convolutional Sequence Learning , 2017, ICLR.

[38]  J. Timonen,et al.  Effects of a low-frequency sound wave therapy programme on functional capacity, blood circulation and bone metabolism in frail old men and women , 2009, Clinical rehabilitation.

[39]  Ulrich Karrenberg.  An interactive multimedia introduction to signal processing , 2002 .

[40]  Raymond MacDonald,et al.  Music, Health, and Wellbeing , 2012 .

[41]  G. Evans,et al.  Ambient neighbourhood noise and children's mental health , 2002, Occupational and environmental medicine.

[42]  Ghorbanali Mohammadi,et al.  AN INVESTIGATION OF COMMUNITY RESPONSE TO URBAN TRAFFIC NOISE , 2009 .

[43]  Dae-Gyu Kang,et al.  “ CREATE MUSIC THAT WILL OPEN A PERSON ’ S HEART ” : A PERSPECTIVE ON EMOTIONAL AND SOCIAL WELLBEING AS DEPICTED IN THREE FILMS , 2016 .

[44]  D. Västfjäll,et al.  Emotional responses to music: the need to consider underlying mechanisms. , 2008, The Behavioral and brain sciences.

[45]  Alice Eldridge,et al.  Emotional associations with soundscape reflect human-environment relationships , 2018 .

[46]  M. Mukaka,et al.  Statistics corner: A guide to appropriate use of correlation coefficient in medical research. , 2012, Malawi medical journal : the journal of Medical Association of Malawi.

[47]  Karen Simonyan,et al.  Neural Audio Synthesis of Musical Notes with WaveNet Autoencoders , 2017, ICML.

[48]  Victor Lazzarini,et al.  Csound: A Sound and Music Computing System , 2016 .

[49]  Masataka Goto,et al.  RWC Music Database: Music genre database and musical instrument sound database , 2003, ISMIR.