Online learning to rank for sequential music recommendation

The prominent success of music streaming services has brought increasingly complex challenges for music recommendation. In particular, in a streaming setting, songs are consumed sequentially within a listening session, which should cater not only for the user's historical preferences, but also for eventual preference drifts, triggered by a sudden change in the user's context. In this paper, we propose a novel online learning to rank approach for music recommendation aimed to continuously learn from the user's listening feedback. In contrast to existing online learning approaches for music recommendation, we leverage implicit feedback as the only signal of the user's preference. Moreover, to adapt rapidly to preference drifts over millions of songs, we represent each song in a lower dimensional feature space and explore multiple directions in this space as duels of candidate recommendation models. Our thorough evaluation using listening sessions from Last.fm demonstrates the effectiveness of our approach at learning faster and better compared to state-of-the-art online learning approaches.

[1]  Andreu Vall,et al.  Listener-Inspired Automated Music Playlist Generation , 2015, RecSys.

[2]  Shuai Li,et al.  On Context-Dependent Clustering of Bandits , 2016, ICML.

[3]  Steven C. H. Hoi,et al.  On Effective Personalized Music Retrieval by Exploring Online User Behaviors , 2016, SIGIR.

[4]  Paul Lamere,et al.  Steerable Playlist Generation by Learning Song Similarity from Radio Station Playlists , 2009, ISMIR.

[5]  Tim Pohle,et al.  Dynamic Playlist Generation Based on Skipping Behavior , 2005, ISMIR.

[6]  Mohan S. Kankanhalli,et al.  Exploiting Music Play Sequence for Music Recommendation , 2017, IJCAI.

[7]  Bamshad Mobasher,et al.  Adapting to User Preference Changes in Interactive Recommendation , 2015, IJCAI.

[8]  Òscar Celma,et al.  Music Recommendation and Discovery - The Long Tail, Long Fail, and Long Play in the Digital Music Space , 2010 .

[9]  Dietmar Jannach,et al.  Automated Generation of Music Playlists: Survey and Experiments , 2014, ACM Comput. Surv..

[10]  Cormac Herley,et al.  Inferring similarity between music objects with application to playlist generation , 2005, MIR '05.

[11]  Y. Song,et al.  A Survey of Music Recommendation Systems and Future Perspectives , 2012 .

[12]  Etienne E. Kerre,et al.  A fuzzy framework for defining dynamic playlist generation heuristics , 2009, Fuzzy Sets Syst..

[13]  Thorsten Joachims,et al.  Interactively optimizing information retrieval systems as a dueling bandits problem , 2009, ICML '09.

[14]  David Hsu,et al.  Exploration in Interactive Personalized Music Recommendation: A Reinforcement Learning Approach , 2013, TOMM.

[15]  Beth Logan,et al.  Content-Based Playlist Generation: Exploratory Experiments , 2002, ISMIR.

[16]  Gerhard Widmer,et al.  Playlist Generation using Start and End Songs , 2008, ISMIR.

[17]  Lior Rokach,et al.  Introduction to Recommender Systems Handbook , 2011, Recommender Systems Handbook.

[18]  Dietmar Jannach,et al.  Sequence-Aware Recommender Systems , 2018, UMAP.

[19]  James C. French,et al.  Flycasting: using collaborative filtering to generate a playlist for online radio , 2001, Proceedings First International Conference on WEB Delivering of Music. WEDELMUSIC 2001.

[20]  Xavier Serra,et al.  Unifying Low-Level and High-Level Music Similarity Measures , 2011, IEEE Transactions on Multimedia.

[21]  John Langford,et al.  The Epoch-Greedy Algorithm for Multi-armed Bandits with Side Information , 2007, NIPS.

[22]  Linas Baltrunas,et al.  Towards Time-Dependant Recommendation based on Implicit Feedback , 2009 .

[23]  Etienne E. Kerre,et al.  Evaluating and Analysing Dynamic Playlist Generation Heuristics Using Radio Logs and Fuzzy Set Theory , 2009, ISMIR.

[24]  Anne Schuth,et al.  Online Learning to Rank for Recommender Systems , 2017, RecSys.

[25]  Jaime Teevan,et al.  Implicit feedback for inferring user preference: a bibliography , 2003, SIGF.

[26]  Martin Szomszor,et al.  Comparison of implicit and explicit feedback from an online music recommendation service , 2010, HetRec '10.

[27]  Ye Wang,et al.  Enhancing Collaborative Filtering Music Recommendation by Balancing Exploration and Exploitation , 2014, ISMIR.

[28]  Peter Stone,et al.  DJ-MC: A Reinforcement-Learning Agent for Music Playlist Recommendation , 2014, AAMAS.

[29]  Robin Burke,et al.  Context-aware music recommendation based on latenttopic sequential patterns , 2012, RecSys.

[30]  Paul Lamere.  I've got 10 million songs in my pocket: now what? , 2012, RecSys '12.

[31]  Yoshua Bengio,et al.  Algorithms for Hyper-Parameter Optimization , 2011, NIPS.

[32]  P. Snickars.  More of the Same – On Spotify Radio , 2017 .

[33]  Katja Hofmann,et al.  Fast and reliable online learning to rank for information retrieval , 2013, SIGIR Forum.

[34]  Wei Chu,et al.  Unbiased offline evaluation of contextual-bandit-based news article recommendation algorithms , 2010, WSDM '11.

[35]  M. de Rijke,et al.  Multileave Gradient Descent for Fast Online Learning to Rank , 2016, WSDM.

[36]  Vaiva Imbrasaite,et al.  Generating Music Playlists with Hierarchical Clustering and Q-Learning , 2015, ECIR.

[37]  Thorsten Joachims,et al.  Playlist prediction via metric embedding , 2012, KDD.

[38]  Maksims Volkovs,et al.  Effective Latent Models for Binary Feedback in Recommender Systems , 2015, SIGIR.

[39]  Wei Chu,et al.  A contextual-bandit approach to personalized news article recommendation , 2010, WWW '10.

[40]  Enric Plaza,et al.  Case-Based Sequential Ordering of Songs for Playlist Recommendation , 2006, ECCBR.