Auditory sensory saliency as a better predictor of change than sound amplitude in pleasantness assessment of reproduced urban soundscapes

The sonic environment of the urban public space is often experienced while walking through it. Nevertheless, city dwellers are usually not actively listening to the environment when traversing the city. Therefore, sound events that are salient, i.e. stand out of the sonic environment, are the ones that trigger attention and contribute highly to the perception of the soundscape. In a previously reported audiovisual perception experiment, the pleasantness of a recorded urban sound walk was continuously evaluated by a group of participants. To detect salient events in the soundscape, a biologically-inspired computational model for auditory sensory saliency based on spectrotemporal modulations is proposed. Using the data from a sound walk, the present study validates the hypothesis that salient events detected by the model contribute to changes in soundscape rating and are therefore important when evaluating the urban soundscape. Finally, when using the data from an additional experiment without a strong visual component, the importance of auditory sensory saliency as a predictor for change in pleasantness assessment is found to be even more pronounced.

[1]  Michael T. Lippert,et al.  Mechanisms for Allocating Auditory Attention: An Auditory Saliency Map , 2005, Current Biology.

[2]  Dick Botteldooren,et al.  The temporal structure of urban soundscapes , 2006 .

[3]  T. H. Pedersen AUDIBILITY OF IMPULSIVE SOUNDS IN ENVIRONMENTAL NOISE , 2014 .

[4]  Xiaoping Hu,et al.  Investigating Effective Brain Connectivity from fMRI Data: Past Findings and Current Issues with Reference to Granger Causality Analysis , 2012, Brain Connect..

[5]  C. Granger Investigating Causal Relations by Econometric Models and Cross-Spectral Methods , 1969 .

[6]  L. Craighero,et al.  Leadership in Orchestra Emerges from the Causal Relationships of Movement Kinematics , 2012, PloS one.

[7]  Anil K. Seth,et al.  Granger causality , 2007, Scholarpedia.

[8]  D. Botteldooren,et al.  Personal Audiovisual Aptitude Influences the Interaction Between Landscape and Soundscape Appraisal , 2018, Front. Psychol..

[9]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[10]  Mounya Elhilali,et al.  Modelling auditory attention , 2017, Philosophical Transactions of the Royal Society B: Biological Sciences.

[11]  M. Scanziani,et al.  Equalizing Excitation-Inhibition Ratios across Visual Cortical Neurons , 2014, Nature.

[12]  S. Shamma,et al.  Interaction between Attention and Bottom-Up Saliency Mediates the Representation of Foreground and Background in an Auditory Scene , 2009, PLoS biology.

[13]  David V. Anderson,et al.  Identifying salient sounds using dual-task experiments , 2013, 2013 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics.

[14]  Gemma María Echevarría Sánchez,et al.  Using Virtual Reality for assessing the role of noise in the audio-visual design of an urban public space , 2017 .

[15]  Petros Maragos,et al.  COGNIMUSE: a multimodal video database annotated with saliency, events, semantics and emotion with application to summarization , 2017, EURASIP J. Image Video Process..

[16]  Bert De Coensel,et al.  Identifying and recognizing noticeable sounds from physical measurements and their effect on soundscape , 2015 .

[17]  Bert De Coensel,et al.  Attention-driven auditory stream segregation using a SOM coupled with an excitatory-inhibitory ANN , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[18]  Sarah Verhulst,et al.  Computational modeling of the human auditory periphery: Auditory-nerve responses, evoked potentials and hearing loss , 2017, Hearing Research.

[19]  M. Nieuwenhuijsen,et al.  Where to put your best foot forward: Psycho-physiological responses to walking in natural and urban environments , 2016 .

[20]  Rainer Goebel,et al.  Investigating directed cortical interactions in time-resolved fMRI data using vector autoregressive modeling and Granger causality mapping. , 2003, Magnetic resonance imaging.

[21]  Hans Colonius,et al.  The optimal time window of visual-auditory integration: a reaction time analysis. , 2010, Frontiers in integrative neuroscience.

[22]  Mounya Elhilali,et al.  Speech processing using adaptive auditory receptive fields , 2017 .

[23]  S. Luck An Introduction to the Event-Related Potential Technique , 2005 .

[24]  S. Bressler,et al.  Granger Causality: Basic Theory and Application to Neuroscience , 2006, q-bio/0608035.

[25]  Roi Cohen Kadosh,et al.  The effect of transcranial direct current stimulation: a role for cortical excitation/inhibition balance? , 2013, Front. Hum. Neurosci..

[26]  Jutta Kretzberg,et al.  A comparative study of seven human cochlear filter models. , 2016, The Journal of the Acoustical Society of America.

[27]  G. Fitzgerald,et al.  'I. , 2019, Australian journal of primary health.

[28]  C. Granger Investigating causal relations by econometric models and cross-spectral methods , 1969 .

[29]  S. Shamma,et al.  Spectro-temporal modulation transfer functions and speech intelligibility. , 1999, The Journal of the Acoustical Society of America.

[30]  Jens Timmer,et al.  Handbook of time series analysis : recent theoretical developments and applications , 2006 .

[31]  A. Poremba,et al.  Achilles’ Ear? Inferior Human Short-Term and Recognition Memory in the Auditory Modality , 2014, PloS one.

[32]  S Ullman,et al.  Shifts in selective visual attention: towards the underlying neural circuitry. , 1985, Human neurobiology.

[33]  N. Kraus,et al.  Musical experience shapes top-down auditory mechanisms: Evidence from masking and auditory attention performance , 2010, Hearing Research.

[34]  Catherine Lavandier,et al.  The Personal Viewpoint on the Meaning of Tranquility Affects the Appraisal of the Urban Park Soundscape , 2017 .

[35]  Malcolm Slaney,et al.  An Efficient Implementation of the Patterson-Holdsworth Auditory Filter Bank , 1997 .

[36]  Miguel P. Eckstein,et al.  Gaze control and memory for objects while walking in a real world environment , 2009 .

[37]  Amir Khan,et al.  Influence of soundscape and interior design on anxiety and perceived tranquillity of patients in a healthcare setting , 2016 .

[38]  Luigi Maffei,et al.  Ten questions on the soundscapes of the built environment , 2016 .

[39]  Semra Aytur,et al.  Complexity in Built Environment, Health, and Destination Walking: A Neighborhood-Scale Analysis , 2012, Journal of Urban Health.

[40]  Emery Schubert,et al.  Acoustic Intensity Causes Perceived Changes in Arousal Levels in Music: An Experimental Investigation , 2011, PloS one.

[41]  Powen Ru,et al.  Multiresolution spectrotemporal analysis of complex sounds. , 2005, The Journal of the Acoustical Society of America.

[42]  M. Corbetta,et al.  How Local Excitation–Inhibition Ratio Impacts the Whole Brain Dynamics , 2014, The Journal of Neuroscience.

[43]  Olli Rummukainen,et al.  Categorization of Natural Dynamic Audiovisual Scenes , 2014, PloS one.

[44]  Petros Maragos,et al.  On amplitude and frequency demodulation using energy operators , 1993, IEEE Trans. Signal Process..

[45]  Pierre Aumond,et al.  Influence of experimental conditions on sound pleasantness evaluations , 2016 .

[46]  B. Berglund,et al.  A principal components model of soundscape perception. , 2010, The Journal of the Acoustical Society of America.

[47]  Wulfram Gerstner,et al.  Neuronal Dynamics: From Single Neurons To Networks And Models Of Cognition , 2014 .

[48]  Keechoo Choi,et al.  To Walk or Not to Walk: Testing the Effect of Path Walkability on Transit Users' Access Mode Choices to the Station , 2015 .

[49]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[50]  William W. Lytton,et al.  Emergence of Physiological Oscillation Frequencies in a Computer Model of Neocortex , 2011, Front. Comput. Neurosci..

[51]  Laurel H. Carney,et al.  Speech Coding in the Brain: Representation of Vowel Formants by Midbrain Neurons Tuned to Sound Fluctuations1,2,3 , 2015, eNeuro.

[52]  Clive W. J. Granger,et al.  Modelling Economic Series: Readings in Econometric Methodology , 1991 .

[53]  David V. Anderson,et al.  Using auditory saliency to understand complex auditory scenes , 2007, 2007 15th European Signal Processing Conference.

[54]  Sunil Puria,et al.  Measurements of human middle ear forward and reverse acoustics: implications for otoacoustic emissions. , 2003, The Journal of the Acoustical Society of America.

[55]  S. Miller Are saving and investment co-integrated? , 1988 .

[56]  Paul Jennings,et al.  The development and application of the emotional dimensions of a soundscape , 2013 .

[57]  M. Winterhalder,et al.  17 Granger Causality : Basic Theory and Application to Neuroscience , 2006 .

[58]  C. Schroeder,et al.  The Spectrotemporal Filter Mechanism of Auditory Selective Attention , 2013, Neuron.

[59]  Andrew L. Maas Rectifier Nonlinearities Improve Neural Network Acoustic Models , 2013 .

[60]  S. Shamma,et al.  An account of monaural phase sensitivity. , 2002, The Journal of the Acoustical Society of America.

[61]  G. Evans,et al.  Environmental Stress , 2018, Environmental Psychology.

[62]  Bert De Coensel,et al.  A model of saliency-based auditory attention to environmental sound , 2010 .

[63]  M. Schönwiesner,et al.  Spectro-temporal modulation transfer function of single voxels in the human auditory cortex measured with high-resolution fMRI , 2009, Proceedings of the National Academy of Sciences.

[64]  Mounya Elhilali,et al.  Feedback-Driven Sensory Mapping Adaptation for Robust Speech Activity Detection , 2017, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[65]  T. Dau,et al.  A computational model of human auditory signal processing and perception. , 2008, The Journal of the Acoustical Society of America.

[66]  Richard Hans Robert Hahnloser,et al.  Digital selection and analogue amplification coexist in a cortex-inspired silicon circuit , 2000, Nature.

[67]  Pierre Aumond,et al.  Global and Continuous Pleasantness Estimation of the Soundscape Perceived during Walking Trips through Urban Environments , 2017 .