Analyzing Temporal Dynamics in Music: : Differential Calculus, Physics, and Functional Data Analysis Techniques

THIS ARTICLE INTRODUCES THEORETICAL and analytical tools for research involving musical emotion or musical change. We describe techniques for visualizing and analyzing data drawn from timevarying processes, such as continuous tension judgments, movement tracking, and performance tempo curves. Functional Data Analysis tools are demonstrated with real-time judgments of musical tension (a proxy for musical affect) to reveal patterns of tension and resolution in a listener9s experience. The derivatives of tension judgment curves are shown to change with cycles of expectation and release in music, indexing the dynamics of musical tension. We explore notions of potential energy and kinetic energy in music and propose that affective energy is stored or released in the listener as musical tension increases and decreases. Differential calculus (and related concepts) are introduced as tools for the analysis of temporal dynamics in musical performances, and phase-plane plots are described as a means to quantify and to visualize musical change.

[1]  C. Krumhansl,et al.  Cross-modal interactions in the perception of musical performance , 2006, Cognition.

[2]  William E. Fredrickson,et al.  Perception of tension in music: musicians versus nonmusicians. , 2000, Journal of music therapy.

[3]  Albert S. Bregman,et al.  The Auditory Scene. (Book Reviews: Auditory Scene Analysis. The Perceptual Organization of Sound.) , 1990 .

[4]  Leonard B. Meyer.  Emotion and Meaning in Music , 1957 .

[5]  Emmanuel Bigand,et al.  Book Review: Tonal Pitch Space , 2003 .

[6]  W. Thompson,et al.  A Cross-Cultural Investigation of the Perception of Emotion in Music: Psychophysical and Cultural Cues , 1999 .

[7]  B. Silverman,et al.  Functional Data Analysis , 1997 .

[8]  W. Jay Dowling,et al.  The Development of Music Perception and Cognition , 1999 .

[9]  Stephen McAdams,et al.  Musical Forces and Melodic Expectations: Comparing Computer Models and Experimental Results , 2004 .

[10]  Steve L Arson.  Musical Forces and Melodic Expectations: Comparing Computer Models and Experimental Results , 2004 .

[11]  S. Trehub.  Musical Predispositions in Infancy , 2001, Annals of the New York Academy of Sciences.

[12]  D. Massaro,et al.  The science of the mind : 2001 and beyond , 1996 .

[13]  C. Krumhansl.  An exploratory study of musical emotions and psychophysiology. , 1997, Canadian journal of experimental psychology = Revue canadienne de psychologie experimentale.

[14]  Diego Vega.  A Perceptual Experiment on Harmonic Tension and Melodic Attraction in Lerdahl's Tonal Pitch Space , 2003 .

[15]  C. Krumhansl.  Music: A Link Between Cognition and Emotion , 2002 .

[16]  Marcelo M. Wanderley,et al.  Performance Gestures of Musicians: What Structural and Emotional Information Do They Convey? , 2003, Gesture Workshop.

[17]  K. Scherer,et al.  EMOTIONAL EFFECTS OF MUSIC: PRODUCTION RULES , 2001 .

[18]  C. Krumhansl,et al.  Can Dance Reflect the Structural and Expressive Qualities of Music? A Perceptual Experiment on Balanchine's Choreography of Mozart's Divertimento No. 15 , 1997 .

[19]  Stephen McAdams,et al.  Influences of Large-Scale Form on Continuous Ratings in Response to a Contemporary Piece in a Live Concert Setting , 2004 .

[20]  J. Sloboda,et al.  Music and emotion: Theory and research , 2001 .

[21]  R. Jackendoff,et al.  A Generative Theory of Tonal Music , 1985 .

[22]  R. Shepard.  Ecological constraints on internal representation: resonant kinematics of perceiving, imagining, thinking, and dreaming. , 1984, Psychological review.

[23]  A. Lehmann,et al.  Tracking Performance Correlates of Changes in Perceived Intensity of Emotion During Different Interpretations of a Chopin Piano Prelude , 2001 .

[24]  B. Repp.  Diversity and commonality in music performance: an analysis of timing microstructure in Schumann's "Träumerei". , 1992, The Journal of the Acoustical Society of America.

[25]  R. Shepard,et al.  Toward a universal law of generalization for psychological science. , 1987, Science.

[26]  B. Repp.  Patterns of note onset asynchronies in expressive piano performance. , 1996, The Journal of the Acoustical Society of America.

[27]  W. E. Fredrickson,et al.  Elementary, Middle, and High School Student Perceptions of Tension in Music , 1997 .

[28]  Eric Clarke,et al.  Rhythm and Timing in Music , 1999 .

[29]  S. T. Buckland,et al.  An Introduction to the Bootstrap. , 1994 .

[30]  S. Kuwano,et al.  Continuous judgment of level-fluctuating sounds and the relationship between overall loudness and instantaneous loudness , 1985, Psychological research.

[31]  Daniel W. Russell,et al.  The cognition–emotion process in achievement-related contexts. , 1979 .

[32]  C. Krumhansl.  A Perceptual Analysis of Mozart's Piano Sonata K. 282: Segmentation, Tension, and Musical Ideas , 1996 .

[33]  W. E. Fredrickson,et al.  A Comparison of Perceived Musical Tension and Aesthetic Response , 1995 .

[34]  A. Storr,et al.  The dynamics of creation , 1972 .

[35]  Fred Lerdahl,et al.  Tonal Pitch Space , 2001 .

[36]  Keith J. Worsley,et al.  Rotation space random fields with an application to fMRI data , 2003 .

[37]  D. Temperley.  The Cognition of Basic Musical Structures , 2001 .

[38]  Laurel J. Trainor,et al.  Memory for melody: infants use a relative pitch code , 2005, Cognition.

[39]  B. Repp.  Expressive Timing in a Debussy Prelude: A Comparison of Student and Expert Pianists , 1997 .

[40]  J. Russell.  Affective space is bipolar. , 1979 .

[41]  Neil P. McAngus Todd,et al.  Motion in Music: A Neurobiological Perspective , 1999 .

[42]  William E. Fredrickson.  Effect of Musical Performance on Perception of Tension in Gustav Hoist's First Suite in E-flat , 1999 .

[43]  Marcelo M. Wanderley,et al.  Quantitative Analysis of Non-obvious Performer Gestures , 2001, Gesture Workshop.

[44]  H. Grice.  Logic and conversation , 1975 .

[45]  K. Scherer,et al.  Emotional states generated by music: An exploratory study of music experts , 2001 .

[46]  Clifford K. Madsen,et al.  The Experience of Musical Tension: A Replication of Nielsen's Research Using the Continuous Response Digital Interface , 1993 .

[47]  Emery Schubert.  Correlation analysis of continuous emotional response to music: Correcting for the effects of serial correlation , 2001 .

[48]  Carl de Boor,et al.  A Practical Guide to Splines , 1978, Applied Mathematical Sciences.

[49]  James O. Ramsay,et al.  Functional Components of Variation in Handwriting , 2000 .

[50]  B. Efron.  Bootstrap Methods: Another Look at the Jackknife , 1979 .

[51]  E. Bigand,et al.  Perceiving musical tension in long chord sequences , 1999, Psychological research.

[52]  Aniruddh D. Patel,et al.  Language, music, syntax and the brain , 2003, Nature Neuroscience.

[53]  Michael Unser,et al.  Splines: a perfect fit for signal and image processing , 1999, IEEE Signal Process. Mag..

[54]  J. Sundberg,et al.  Does music performance allude to locomotion? A model of final ritardandi derived from measurements of stopping , 1999 .

[55]  Manfred Clynes,et al.  THE LIVING QUALITY OF MUSIC Neurobiologic Patterns of Communicating Feeling , 1982 .

[56]  Nicholas A. Smith,et al.  Perceptions of Musical Dimensions in Beethoven's Waldsiein Sonata: An Application of Tonal Pitch Space Theory , 2003 .

[57]  James O. Ramsay,et al.  Applied Functional Data Analysis: Methods and Case Studies , 2002 .

[58]  Fred Lerdahl,et al.  Calculating Tonal Tension , 1996 .