From acceleration to rhythmicity: Smartphone-assessed movement predicts properties of music

ABSTRACT Querying music is still a disembodied process in Music Information Retrieval. Thus, the goal of the presented study was to explore how free and spontaneous movement captured by smartphone accelerometer data can be related to musical properties. Motion features related to tempo, smoothness, size, and regularity were extracted and shown to predict the musical qualities ‘rhythmicity’ (R² = .45), ‘pitch level + range’ (R² = .06) and ‘complexity (R² = .15). We conclude that (rhythmic) music properties can be predicted from movement, and that an embodied approach to MIR is feasible.

[1]  Bruno L. Giordano,et al.  The Production and Perception of Emotionally Expressive Walking Sounds: Similarities between Musical Performance and Everyday Motor Activity , 2014, PloS one.

[2]  Marc Leman,et al.  Toward E-Motion-Based Music Retrieval a Study of Affective Gesture Recognition , 2012, IEEE Transactions on Affective Computing.

[3]  W. Schlotz,et al.  Understanding music-selection behavior via statistical learning , 2018 .

[4]  D. Moelants,et al.  Walking on music. , 2007, Human movement science.

[5]  C. Drake,et al.  The development of rhythmic attending in auditory sequences: attunement, referent period, focal attending , 2000, Cognition.

[6]  Mari Romarheim Haugen,et al.  Exploring Sound-Motion Similarity in Musical Experience , 2016 .

[7]  P. Janata,et al.  Embodied music cognition and mediation technology , 2009 .

[8]  Marc Leman,et al.  Microtiming Patterns and Interactions with Musical Properties in Samba Music , 2011 .

[9]  T. Wheatley,et al.  Music and movement share a dynamic structure that supports universal expressions of emotion , 2012, Proceedings of the National Academy of Sciences.

[10]  P. Gomez,et al.  Relationships between musical structure and psychophysiological measures of emotion. , 2007, Emotion.

[11]  Suvi Saarikallio,et al.  Effects of the Big Five and musical genre on music-induced movement , 2010 .

[12]  D. Moelants,et al.  The Impact of the Bass Drum on Human Dance Movement , 2013 .

[13]  Birgitta Burger,et al.  Influences of Rhythm- and Timbre-Related Musical Features on Characteristics of Music-Induced Movement , 2013, Front. Psychol..

[14]  M. Leman,et al.  The Role of Embodiment in the Perception of Music , 2015 .

[15]  Marc R. Thompson,et al.  Embodied Meter: Hierarchical Eigenmodes in Music-Induced Movement , 2010 .

[16]  Marc Leman,et al.  Action-based effects on music perception , 2013, Front. Psychol..

[17]  Justin London,et al.  Synchronization to metrical levels in music depends on low-frequency spectral components and tempo , 2018, Psychological research.

[18]  M. R. Jones,et al.  Time, our lost dimension: toward a new theory of perception, attention, and memory. , 1976, Psychological review.

[19]  M. Leman,et al.  The Routledge companion to embodied music interaction , 2017 .

[20]  Lawrence E Marks,et al.  Lower pitch is larger, yet falling pitches shrink. , 2014, Experimental psychology.

[21]  Birgitta Burger,et al.  Hunting for the beat in the body: on period and phase locking in music-induced movement , 2014, Front. Hum. Neurosci..

[22]  S. McAdams,et al.  Empathy and Emotional Contagion as a Link Between Recognized and Felt Emotions in Music Listening , 2013 .

[23]  D. Moelants.  Preferred tempo reconsidered. , 2002 .

[24]  Alexander Refsum Jensenius,et al.  Analyzing Free-Hand Sound-Tracings of Melodic Phrases , 2018 .

[25]  R. Zatorre,et al.  When the brain plays music: auditory–motor interactions in music perception and production , 2007, Nature Reviews Neuroscience.

[26]  Fabien Gouyon,et al.  Syncopation creates the sensation of groove in synthesized music examples , 2014, Front. Psychol..

[27]  Jessica A. Grahn,et al.  Feeling the Beat: Premotor and Striatal Interactions in Musicians and Nonmusicians during Beat Perception , 2009, The Journal of Neuroscience.

[28]  Alexander Truslit.  Gestaltung und Bewegung in der Musik , 1938 .

[29]  Henkjan Honing,et al.  Without it no music: beat induction as a fundamental musical trait , 2012, Annals of the New York Academy of Sciences.

[30]  Maria A. G. Witek,et al.  Syncopation, Body-Movement and Pleasure in Groove Music , 2014, PloS one.

[31]  Hauke Egermann,et al.  From Motion to Emotion: Accelerometer Data Predict Subjective Experience of Music , 2016, PloS one.

[32]  Antonio Camurri,et al.  Multimodal Analysis of Expressive Gesture in Music and Dance Performances , 2003, Gesture Workshop.

[33]  Dan Tidhar,et al.  Musicians are more consistent: Gestural cross-modal mappings of pitch, loudness and tempo in real-time , 2014, Front. Psychol..