Dance to your own drum: Identification of musical genre and individual dancer from motion capture using machine learning

ABSTRACT Machine learning has been used to accurately classify musical genre using features derived from audio signals. Musical genre, as well as lower-level audio features of music, have also been shown to influence music-induced movement, however, the degree to which such movements are genre-specific has not been explored. The current paper addresses this using motion capture data from participants dancing freely to eight genres. Using a Support Vector Machine model, data were classified by genre and by individual dancer. Against expectations, individual classification was notably more accurate than genre classification. Results are discussed in terms of embodied cognition and culture.

[1]  G. Johansson.  Visual perception of biological motion and a model for its analysis , 1973 .

[2]  J. Cutting,et al.  Recognizing friends by their walk: Gait perception without familiarity cues , 1977 .

[3]  W. Straw.  "Characterizing Rock Music Cultures: The Case of Heavy Metal" , 1984 .

[4]  Bethany Bryson,et al.  Anything but heavy metal : Symbolic exclusion and musical dislikes , 1996 .

[5]  H. Spring.  Swing and the Lindy Hop: Dance, Venue, Media, and Tradition , 1997 .

[6]  G. Lakoff.  Philosophy in the flesh , 1999 .

[7]  Matti Karjalainen,et al.  A computationally efficient multipitch analysis model , 2000, IEEE Trans. Speech Audio Process..

[8]  Guodong Guo,et al.  Face recognition by support vector machines , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[9]  Bruno Nettl.  An ethnomusicologist contemplates universals in musical sound and musical culture , 2000 .

[10]  Martine Villeneuve,et al.  Heavy Metal Music and Adolescent Suicidal Risk , 2001 .

[11]  T. Monaghan.  Why Study the Lindy Hop? , 2001, Dance Research Journal.

[12]  Lucy Johnston,et al.  Victim Selection and Kinematics: A Point-Light Investigation of Vulnerability to Attack , 2002 .

[13]  George Tzanetakis,et al.  Musical genre classification of audio signals , 2002, IEEE Trans. Speech Audio Process..

[14]  Robert Tibshirani,et al.  1-norm Support Vector Machines , 2003, NIPS.

[15]  Antonio Camurri,et al.  Recognizing emotion from dance movement: comparison of spectator recognition and automated techniques , 2003, Int. J. Hum. Comput. Stud..

[16]  François Pachet,et al.  Representing Musical Genre: A State of the Art , 2003 .

[17]  A. Ng.  Feature selection, L1 vs. L2 regularization, and rotational invariance , 2004, Twenty-first international conference on Machine learning - ICML '04.

[18]  N. Troje,et al.  Person identification from biological motion: Effects of structural and kinematic cues , 2005, Perception & psychophysics.

[19]  N. Scaringella,et al.  Automatic genre classification of music content: a survey , 2006, IEEE Signal Process. Mag..

[20]  Adrian Hilton,et al.  A survey of advances in vision-based human motion capture and analysis , 2006, Comput. Vis. Image Underst..

[21]  M. Leman.  Embodied Music Cognition and Mediation Technology , 2007 .

[22]  Larry Shapiro.  The Embodied Cognition Research Programme , 2007 .

[23]  Cord Westhoff,et al.  Kinematic cues for person identification from biological motion , 2007, Perception & psychophysics.

[24]  Dave Snell,et al.  Heavy Metal, identity and the social negotiation of a community of practice , 2007 .

[25]  Weifeng Liu,et al.  Correntropy: Properties and Applications in Non-Gaussian Signal Processing , 2007, IEEE Transactions on Signal Processing.

[26]  Andreas Christmann,et al.  Support vector machines , 2008, Data Mining and Knowledge Discovery Handbook.

[27]  Yannis Stylianou,et al.  Musical Genre Classification Using Nonnegative Matrix Factorization-Based Features , 2008, IEEE Transactions on Audio, Speech, and Language Processing.

[28]  Jennifer C. Lena,et al.  Classification as Culture: Types and Trajectories of Music Genres , 2008 .

[29]  Mark Shevy,et al.  Music genre as cognitive schema: extramusical associations with country and hip-hop music , 2008 .

[30]  Fatih Murat Porikli,et al.  Pedestrian Detection via Classification on Riemannian Manifolds , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Nello Cristianini,et al.  Support vector machines , 2009 .

[32]  Òscar Celma,et al.  The Quest for Musical Genres: Do the Experts and the Wisdom of Crowds Agree? , 2008, ISMIR.

[33]  Bernard De Baets,et al.  How potential users of music search and retrieval systems describe the semantic quality of music , 2008, J. Assoc. Inf. Sci. Technol..

[34]  Paul Lamere,et al.  Social Tagging and Music Information Retrieval , 2008 .

[35]  Øivind Skare,et al.  Simultaneous estimation of effects of gender, age and walking speed on kinematic gait data. , 2009, Gait & posture.

[36]  Tapio Elomaa,et al.  A Walk from 2-Norm SVM to 1-Norm SVM , 2009, 2009 Ninth IEEE International Conference on Data Mining.

[37]  Peter E Keller,et al.  Self‐recognition in the Perception of Actions Performed in Synchrony with Music , 2009, Annals of the New York Academy of Sciences.

[38]  Janusz Konrad,et al.  Action Recognition in Video by Covariance Matching of Silhouette Tunnels , 2009, 2009 XXII Brazilian Symposium on Computer Graphics and Image Processing.

[39]  N. Troje,et al.  Embodiment of Sadness and Depression—Gait Patterns Associated With Dysphoric Mood , 2009, Psychosomatic medicine.

[40]  Israel Cohen,et al.  Musical genre classification of audio signals using geometric methods , 2010, 2010 18th European Signal Processing Conference.

[41]  S. Demorest,et al.  INFLUENCE OF MUSICAL FEATURES ON CHARACTERISTICS OF MUSIC-INDUCED MOVEMENTS , 2010 .

[42]  Suvi Saarikallio,et al.  Effects of the Big Five and musical genre on music-induced movement , 2010 .

[43]  Ronald Poppe,et al.  A survey on vision-based human action recognition , 2010, Image Vis. Comput..

[44]  H. Erdjument-Bromage,et al.  TLR signaling augments macrophage bactericidal activity through mitochondrial ROS , 2011, Nature.

[45]  George A. Tsihrintzis,et al.  Automatic Music Genre Classification Using Hybrid Genetic Algorithms , 2011 .

[46]  Marc R. Thompson,et al.  Learning and Synchronising Dance Movements in South African Songs - Cross-cultural Motion-capture Study , 2011 .

[47]  Sofia Dahl,et al.  Striking movements: A survey of motion analysis of percussionists , 2011 .

[48]  David J. Teachout,et al.  Genre identification of very brief musical excerpts , 2012 .

[49]  Martin A. Riedmiller,et al.  Unsupervised Learning of Local Features for Music Classification , 2012, ISMIR.

[50]  Maggie Shiffrar,et al.  People Watching: Social, Perceptual, and Neurophysiological Studies of Body Perception , 2012 .

[51]  S. Mahadevan,et al.  Learning Theory , 2001 .

[52]  Hfd Chang,et al.  Shape-Independent Processing of Biological Motion , 2013 .

[53]  S. Trehub,et al.  Culture and Evolution , 2013 .

[54]  Petri Toiviainen,et al.  MOCAP TOOLBOX - A MATLAB TOOLBOX FOR COMPUTATIONAL ANALYSIS OF MOVEMENT DATA , 2013 .

[55]  Birgitta Burger,et al.  Influences of Rhythm- and Timbre-Related Musical Features on Characteristics of Music-Induced Movement , 2013, Front. Psychol..

[56]  Andrew D. Wilson,et al.  Embodied Cognition is Not What you Think it is , 2013, Front. Psychology.

[57]  D. Moelants,et al.  The Impact of the Bass Drum on Human Dance Movement , 2013 .

[58]  Robert L. Goldstone,et al.  Similarity-Dissimilarity Competition in Disjunctive Classification Tasks , 2013, Front. Psychology.

[59]  Marc Leman,et al.  Expressing Induced Emotions Through Free Dance Movement , 2013 .

[60]  Stephen S. Hudson.  METAL MOVEMENTS: HEADBANGING AS A LEGACY OF AFRICAN AMERICAN DANCE , 2015 .

[61]  Anne Danielsen,et al.  Effects of instructed timing and tempo on snare drum sound in drum kit performance. , 2015, The Journal of the Acoustical Society of America.

[62]  Benjamin Rosman,et al.  Single-labelled music genre classification using content-based features , 2015, 2015 Pattern Recognition Association of South Africa and Robotics and Mechatronics International Conference (PRASA-RobMech).

[63]  N. Morgan,et al.  Exploring dress, identity and performance in contemporary dance music culture , 2015 .

[64]  Justin London,et al.  Conscientiousness and Extraversion relate to responsiveness to tempo in dance. , 2016, Human movement science.

[65]  Mari Romarheim Haugen,et al.  Exploring Sound-Motion Similarity in Musical Experience , 2016 .

[66]  Soumen Bhowmik,et al.  A Literature Survey on Human Identification by Gait , 2016 .

[67]  Loris Nanni,et al.  Combining visual and acoustic features for music genre classification , 2016, Expert Syst. Appl..

[68]  Joachim Richter,et al.  “It Don’t Mean a Thing if It Ain’t Got that Swing”– an Alternative Concept for Understanding the Evolution of Dance and Music in Human Beings , 2016, Front. Hum. Neurosci..

[69]  Kemal Leblebicioglu,et al.  Time series classification with feature covariance matrices , 2018, Knowledge and Information Systems.

[70]  Birgitta Burger,et al.  Personality and Musical Preference Using Social-Tagging in Excerpt-Selection , 2017 .

[71]  P. Morris,et al.  Evidence of Big Five and Aggressive Personalities in Gait Biomechanics , 2016, Journal of nonverbal behavior.

[72]  A. Jensenius,et al.  Pleasurable and Intersubjectively Embodied Experiences of Electronic Dance Music , 2017 .

[73]  Birgitta Burger,et al.  Dance Like Someone is Watching , 2018 .

[74]  Katsumi Watanabe,et al.  Contribution of global and local biological motion information to speed perception and discrimination. , 2018, Journal of Vision.

[75]  B. Bläsing,et al.  My Action, My Self: Recognition of Self-Created but Visually Unfamiliar Dance-Like Actions From Point-Light Displays , 2018, Front. Psychol..

[76]  Birgitta Burger,et al.  Embodiment in Electronic Dance Music: Effects of musical content and structure on body movement , 2018, Musicae Scientiae.