The Use of Spatio-Temporal Connectionist Models in Psychological Studies of Musical Emotions

THIS ARTICLE PRESENTS A NOVEL METHODOLOGY TO analyze the dynamics of emotional responses to music. It consists of a computational investigation based on spatiotemporal neural networks, which "mimic" human affective responses to music and predict the responses to novel music sequences. The results provide evidence suggesting that spatiotemporal patterns of sound resonate with affective features underlying judgments of subjective feelings (arousal and valence). A significant part of the listener9s affective response is predicted from a set of six psychoacoustic features of sound——loudness, tempo, texture, mean pitch, pitch variation, and sharpness. A detailed analysis of the network parameters and dynamics also allows us to identify the role of specific psychoacoustic variables (e.g., tempo and loudness) in music emotional appraisal. This work contributes new evidence and insights to the study of musical emotions, with particular relevance to the music perception and cognition research community.

[1]  R. Parncutt.  Harmony: A Psychoacoustical Approach , 1989 .

[2]  A. Lehmann,et al.  Tracking Performance Correlates of Changes in Perceived Intensity of Emotion During Different Interpretations of a Chopin Piano Prelude , 2001 .

[3]  R. Menezes,et al.  Entropy-Based Independence Test , 2006 .

[4]  George Tzanetakis,et al.  MARSYAS: a framework for audio analysis , 1999, Organised Sound.

[5]  L. Cuddy,et al.  Expectancies generated by melodic intervals: Perceptual judgments of melodic continuity , 1995, Perception & psychophysics.

[6]  A. Damasio.  The Feeling of What Happens: Body and Emotion in the Making of Consciousness , 1999 .

[7]  P. Kivy.  Music Alone: Philosophical Reflections on the Purely Musical Experience , 1991 .

[8]  Marketta Korhonen.  MODELING CONTINUOUS EMOTIONAL APPRAISALS OF MUSIC USING SYSTEM IDENTIFICATION , 2004 .

[9]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[10]  J. Russell.  A circumplex model of affect. , 1980 .

[11]  Carol L. Krumhansl,et al.  Melodic structure: Theoretical and empirical descriptions , 1991 .

[12]  K. Hevner.  Experimental studies of the elements of expression in music , 1936 .

[13]  C. Krumhansl.  An exploratory study of musical emotions and psychophysiology. , 1997, Canadian journal of experimental psychology = Revue canadienne de psychologie experimentale.

[14]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[15]  A. Gabrielsson,et al.  The influence of musical structure on emotional expression. , 2001 .

[16]  Leonard B. Meyer.  Emotion and Meaning in Music , 1957 .

[17]  R. Zatorre,et al.  Intensely pleasurable responses to music correlate with activity in brain regions implicated in reward and emotion , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[18]  Emery Schubert.  Measuring emotion continuously: Validity and reliability of the two-dimensional emotion-space , 1999 .

[19]  C. Krumhansl.  A Perceptual Analysis of Mozart's Piano Sonata K. 282: Segmentation, Tension, and Musical Ideas , 1996 .

[20]  J. Panksepp.  Affective Neuroscience: The Foundations of Human and Animal Emotions , 1998 .

[21]  Stefan C. Kremer,et al.  Spatiotemporal Connectionist Networks: A Taxonomy and Review , 2001, Neural Computation.

[22]  Stephen N. Malloch,et al.  Timbre and technology : an analytical partnership : the development of an analytical technique and its application to music by Lutoslawski and Ligeti , 1997 .

[23]  K. Scherer.  Which Emotions Can be Induced by Music? What Are the Underlying Mechanisms? And How Can We Measure Them? , 2004 .

[24]  K. Scherer,et al.  Handbook of affective sciences. , 2003 .

[25]  E. Narmour.  The Analysis and Cognition of Melodic Complexity: The Implication-Realization Model , 1992 .

[26]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[27]  J. Sloboda,et al.  Music and emotion: Theory and research , 2001 .

[28]  M. Bradley,et al.  Measuring emotion: the Self-Assessment Manikin and the Semantic Differential. , 1994, Journal of behavior therapy and experimental psychiatry.

[29]  J. Panksepp,et al.  Emotional sounds and the brain: the neuro-affective foundations of musical appreciation , 2002, Behavioural Processes.

[30]  K. Scherer,et al.  Cue utilization in emotion attribution from auditory stimuli , 1977 .

[31]  Densil Cabrera,et al.  ' PSYSOUND' : A COMPUTER PROGRAM FOR PSYCHOACOUSTICAL ANALYSIS , 1999 .

[32]  I. Peretz,et al.  Music and emotion: perceptual determinants, immediacy, and isolation after brain damage , 1998, Cognition.

[33]  W. Thompson,et al.  A Cross-Cultural Investigation of the Perception of Emotion in Music: Psychophysical and Cultural Cues , 1999 .

[34]  Richard A. Davis,et al.  Time Series: Theory and Methods (2nd ed.). , 1992 .

[35]  Alan C. Evans,et al.  Emotional responses to pleasant and unpleasant music correlate with activity in paralimbic brain regions , 1999, Nature Neuroscience.

[36]  P. Juslin,et al.  Emotional Expression in Music Performance: Between the Performer's Intention and the Listener's Experience , 1996 .

[37]  DeWitt H. Parker,et al.  Philosophy in a New Key , 1941 .

[38]  Emery Schubert.  Modeling Perceived Emotion With Continuous Musical Features , 2004 .

[39]  Jeffrey L. Elman,et al.  Distributed Representations, Simple Recurrent Networks, and Grammatical Structure , 1991, Mach. Learn..

[40]  J. Russell.  MEASURES OF EMOTION , 1989 .

[41]  C. Granger,et al.  USING THE MUTUAL INFORMATION COEFFICIENT TO IDENTIFY LAGS IN NONLINEAR MODELS , 1994 .