Encouraging Attention and Exploration in a Hybrid Recommender System for Libraries of Unfamiliar Music

There are few studies of user interaction with music libraries comprising solely of unfamiliar music, despite such music being represented in national music information centre collections. We aim to develop a system that encourages exploration of such a library. This study investigates the influence of 69 users’ pre-existing musical genre and feature preferences on their ongoing continuous real-time psychological affect responses during listening and the acoustic features of the music on their liking and familiarity ratings for unfamiliar art music (the collection of the Australian Music Centre) during a sequential hybrid recommender-guided interaction. We successfully mitigated the unfavorable starting conditions (no prior item ratings or participants’ item choices) by using each participant’s pre-listening music preferences, translated into acoustic features and linked to item view count from the Australian Music Centre database, to choose their seed item. We found that first item liking/familiarity ratings were on average higher than the subsequent 15 items and comparable with the maximal values at the end of listeners’ sequential responses, showing acoustic features to be useful predictors of responses. We required users to give a continuous response indication of their perception of the affect expressed as they listened to 30-second excerpts of music, with our system successfully providing either a “similar” or “dissimilar” next item, according to—and confirming—the utility of the items’ acoustic features, but chosen from the affective responses of the preceding item. We also developed predictive statistical time series analysis models of liking and familiarity, using music preferences and preceding ratings. Our analyses suggest our users were at the starting low end of the commonly observed inverted-U relationship between exposure and both liking and perceived familiarity, which were closely related. Overall, our hybrid recommender worked well under extreme conditions, with 53 unique items from 100 chosen as “seed” items, suggesting future enhancement of our approach can productively encourage exploration of libraries of unfamiliar music.

[1]  William T. M. Dunsmuir,et al.  Time series analysis of real-time music perception: approaches to the assessment of individual and expertise differences in perception of expressed affect , 2014 .

[2]  Shuyang Zhao,et al.  A personalized hybrid music recommender based on empirical estimation of user-timbre preference , 2014 .

[3]  P. Mahalanobis On the generalized distance in statistics , 1936 .

[4]  R. Dean The Oxford Handbook of Computer Music , 2011 .

[5]  J. Russell A circumplex model of affect. , 1980 .

[6]  A. Gabrielsson Emotion perceived and emotion felt: Same or different? , 2001 .

[7]  Markus Schedl,et al.  Investigating country-specific music preferences and music recommendation algorithms with the LFM-1b dataset , 2017, International Journal of Multimedia Information Retrieval.

[8]  Jyh-Shing Roger Jang,et al.  Music Genre Classification via Compressive Sampling , 2010, ISMIR.

[9]  Markus Schedl,et al.  On the Interrelation Between Listener Characteristics and the Perception of Emotions in Classical Orchestra Music , 2018, IEEE Transactions on Affective Computing.

[10]  Petri Toiviainen,et al.  A Matlab Toolbox for Music Information Retrieval , 2007, GfKl.

[11]  Nicholas J Hudson,et al.  Musical beauty and information compression: Complex to the ear but simple to the mind? , 2011, BMC Research Notes.

[12]  Emery Schubert,et al.  Using Psychological Principles of Memory Storage and Preference to Improve Music Recommendation Systems , 2018, Leonardo Music Journal.

[13]  Eric Allamanche,et al.  Content-based Identification of Audio Material Using MPEG-7 Low Level Description , 2001, ISMIR.

[14]  Yvonne Leung,et al.  What Constitutes a Phrase in Sound-Based Music? A Mixed-Methods Investigation of Perception and Acoustics , 2016, PloS one.

[15]  Mikhail Malt,et al.  Zsa . Descriptors : a library for real-time descriptors analysis , 2008 .

[16]  E. L. Cerroni-long Accounting for Tastes: Australian Everyday Cultures , 2001 .

[17]  Charu C. Aggarwal,et al.  Recommender Systems: The Textbook , 2016 .

[18]  Stephen McAdams,et al.  Perspectives on the Contribution of Timbre to Musical Structure , 1999, Computer Music Journal.

[19]  Michael A. Casey,et al.  General sound classification and similarity in MPEG-7 , 2001, Organised Sound.

[20]  Robin D. Burke,et al.  Hybrid Recommender Systems: Survey and Experiments , 2002, User Modeling and User-Adapted Interaction.

[21]  Roger T. Dean,et al.  Modelling perception of structure and affect in music : spectral centroid and Wishart's Red Bird , 2011 .

[22]  Catherine J. Stevens,et al.  A Continuous measure of musical engagement contributes to prediction of perceived arousal and valence , 2014 .

[23]  Roger T. Dean,et al.  Listener Detection of Segmentation in Computer-Generated Sound: An Exploratory Experimental Study , 2007 .

[24]  Julián Urbano,et al.  The MediaEval 2018 AcousticBrainz Genre Task: Content-based Music Genre Recognition from Multiple Sources , 2017, MediaEval.

[25]  Wolfgang A. Halang,et al.  Incremental collaborative filtering based on Mahalanobis distance and fuzzy membership for recommender systems , 2013, Int. J. Gen. Syst..

[26]  Catherine J. Stevens,et al.  Both acoustic intensity and loudness contribute to time-series models of perceived affect in response to music. , 2015 .

[27]  Freya Bailes,et al.  Facilitation and Coherence Between the Dynamic and Retrospective Perception of Segmentation in Computer-Generated Music , 2007 .

[28]  Emilia Gómez,et al.  Semantic audio content-based music recommendation and visualization based on user preference examples , 2013, Inf. Process. Manag..

[29]  Leigh Landy Sound-Based Music 4 All , 2009 .

[30]  Òscar Celma,et al.  Music Recommendation and Discovery - The Long Tail, Long Fail, and Long Play in the Digital Music Space , 2010 .

[31]  Miller Puckette,et al.  Real-time audio analysis tools for Pd and MSP , 1998, ICMC.

[32]  Peter Knees,et al.  Music Recommender Systems , 2015, Recommender Systems Handbook.

[33]  Catherine Guastavino,et al.  User studies in the Music Information Retrieval Literature , 2011, ISMIR.

[34]  Hendrik Schreiber,et al.  Genre Ontology Learning: Comparing Curated with Crowd-Sourced Ontologies , 2016, ISMIR.

[35]  Freya Bailes,et al.  Shared and distinct mechanisms of individual and expertise-group perception of expressed arousal in four works , 2014 .

[36]  Emery Schubert Measuring emotion continuously: Validity and reliability of the two-dimensional emotion-space , 1999 .

[37]  George Tzanetakis,et al.  Musical genre classification of audio signals , 2002, IEEE Trans. Speech Audio Process..

[38]  James Murdoch Australian Music Centre , 1975 .

[39]  Emery Schubert Modeling Perceived Emotion With Continuous Musical Features , 2004 .

[40]  Peter Knees,et al.  Music Similarity and Retrieval: An Introduction to Audio- and Web-based Strategies , 2016 .

[41]  Daniel J Levitin,et al.  The structure of musical preferences: a five-factor model. , 2011, Journal of personality and social psychology.

[42]  Shiu-li Huang,et al.  Designing utility-based recommender systems for e-commerce: Evaluation of preference-elicitation methods , 2011, Electron. Commer. Res. Appl..

[43]  Freya Bailes,et al.  Listeners Discern Affective Variation in Computer-Generated Musical Sounds , 2009, Perception.

[44]  Sophie Ahrens,et al.  Recommender Systems , 2012 .

[45]  Francesco Ricci,et al.  Prediction of music pairwise preferences from facial expressions , 2019, IUI.

[46]  Emery Schubert,et al.  Acoustic Intensity Causes Perceived Changes in Arousal Levels in Music: An Experimental Investigation , 2011, PloS one.

[47]  Xavier Serra,et al.  Unifying Low-Level and High-Level Music Similarity Measures , 2011, IEEE Transactions on Multimedia.

[48]  Emery Schubert,et al.  Continuous Self-Report Methods , 1993 .

[49]  Freya Bailes,et al.  Comparative time series analysis of perceptual responses to electroacoustic music , 2012 .

[50]  Jürgen Schmidhuber,et al.  Driven by Compression Progress: A Simple Principle Explains Essential Aspects of Subjective Beauty, Novelty, Surprise, Interestingness, Attention, Curiosity, Creativity, Art, Science, Music, Jokes , 2008, ABiALS.

[51]  S. McAdams,et al.  Acoustic correlates of timbre space dimensions: a confirmatory study using synthetic tones. , 2005, The Journal of the Acoustical Society of America.

[52]  Alf Gabrielsson The Relationship between Musical Structure and Perceived Expression , 2008 .

[53]  Freya Bailes,et al.  Time Series Analysis as a Method to Examine Acoustical Influences on Real-time Perception of Music , 2010 .

[54]  Scott Sanner,et al.  Social collaborative filtering for cold-start recommendations , 2014, RecSys '14.