Of Spiky SVDs and Music Recommendation

The truncated singular value decomposition is a widely used methodology in music recommendation for direct similar-item retrieval and downstream tasks embedding musical items. This paper investigates a curious effect that we show naturally occurring on many recommendation datasets: spiking formations in the embedding space. We first propose a metric to quantify this spiking organization’s strength, then mathematically prove its origin tied to underlying communities of items of varying internal popularity. With this new-found theoretical understanding, we finally open the topic with an industrial use case of estimating how music embeddings’ top-k similar items will change over time under the addition of data.

[1]  Romain Hennequin,et al.  Learning Unsupervised Hierarchies of Audio Concepts , 2022, ISMIR.

[2]  Elena V. Epure,et al.  Explainability in Music Recommender Systems , 2022, AI Mag..

[3]  Aäron van den Oord,et al.  Towards Learning Universal Audio Representations , 2021, ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[4]  Romain Hennequin,et al.  Cold Start Similar Artists Ranking with Gravity-Inspired Graph Autoencoders , 2021, RecSys.

[5]  Walid Bendada,et al.  A Semi-Personalized System for User Cold Start Recommendation on Music Streaming Apps , 2021, KDD.

[6]  Ruoming Jin,et al.  Towards a Better Understanding of Linear Models for Recommendation , 2021, KDD.

[7]  Stephan Günnemann,et al.  Reliable Graph Neural Networks via Robust Aggregation , 2020, NeurIPS.

[8]  Matthew Roughan,et al.  Popularity and Centrality in Spotify Networks: Critical transitions in eigenvector centrality , 2020, J. Complex Networks.

[9]  Walid Krichene,et al.  Neural Collaborative Filtering vs. Matrix Factorization Revisited , 2020, RecSys.

[10]  Dietmar Jannach,et al.  A Troubling Analysis of Reproducibility and Progress in Recommender Systems Research , 2019, ACM Trans. Inf. Syst..

[11]  Stephan Günnemann,et al.  Diffusion Improves Graph Learning , 2019, NeurIPS.

[12]  Dietmar Jannach,et al.  Are we really making much progress? A worrying analysis of recent neural recommendation approaches , 2019, RecSys.

[13]  Michalis Vazirgiannis,et al.  Gravity-Inspired Graph Autoencoders for Directed Link Prediction , 2019, CIKM.

[14]  Yehuda Koren,et al.  On the Difficulty of Evaluating Baselines: A Study on Recommender Systems , 2019, ArXiv.

[15]  Feiping Nie,et al.  Regularized Singular Value Decomposition and Application to Recommender System , 2018, ArXiv.

[16]  Hugo Caselles-Dupré,et al.  Word2vec applied to recommendation: hyperparameters matter , 2018, RecSys.

[17]  Hamed Zamani,et al.  Current challenges and visions in music recommender systems research , 2017, International Journal of Multimedia Information Retrieval.

[18]  Chao Gao,et al.  Community Detection in Degree-Corrected Block Models , 2016, The Annals of Statistics.

[19]  F. Maxwell Harper,et al.  The MovieLens Datasets: History and Context , 2016, TIIS.

[20]  Omer Levy,et al.  Improving Distributional Similarity with Lessons Learned from Word Embeddings , 2015, TACL.

[21]  Andreas Krause,et al.  Advances in Neural Information Processing Systems (NIPS) , 2014 .

[22]  Omer Levy,et al.  Neural Word Embedding as Implicit Matrix Factorization , 2014, NIPS.

[23]  A. Rinaldo,et al.  Consistency of spectral clustering in stochastic block models , 2013, 1312.2050.

[24]  Tai Qin,et al.  Regularized Spectral Clustering under the Degree-Corrected Stochastic Blockmodel , 2013, NIPS.

[25]  Jiashun Jin,et al.  Fast network community detection by SCORE , 2012, ArXiv.

[26]  Thorsten Joachims,et al.  Playlist prediction via metric embedding , 2012, KDD.

[27]  Julie Allen.  The Filter Bubble: What the Internet Is Hiding from You , 2012 .

[28]  Przemyslaw Kazienko,et al.  GED: the method for group evolution discovery in social networks , 2012, Social Network Analysis and Mining.

[29]  Eli Pariser.  FILTER BUBBLE: Wie wir im Internet entmündigt werden , 2012 .

[30]  Daniele Quercia,et al.  Auralist: introducing serendipity into music recommendation , 2012, WSDM '12.

[31]  David F. Gleich,et al.  Tall and skinny QR factorizations in MapReduce architectures , 2011, MapReduce '11.

[32]  Mouzhi Ge,et al.  Beyond accuracy: evaluating recommender systems by coverage and serendipity , 2010, RecSys '10.

[33]  Roberto Turrin,et al.  Performance of recommender algorithms on top-n recommendation tasks , 2010, RecSys '10.

[34]  Mark E. J. Newman,et al.  Stochastic blockmodels and community structure in networks , 2010, Physical review. E, Statistical, nonlinear, and soft matter physics.

[35]  Derek Greene,et al.  Tracking the Evolution of Communities in Dynamic Social Networks , 2010, 2010 International Conference on Advances in Social Networks Analysis and Mining.

[36]  Nathan Halko,et al.  Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions , 2009, SIAM Rev..

[37]  Lars Schmidt-Thieme,et al.  BPR: Bayesian Personalized Ranking from Implicit Feedback , 2009, UAI.

[38]  Mark B. Sandler,et al.  Musically Meaningful or Just Noise? An Analysis of On-line Artist Networks , 2009, CMMR.

[39]  Santo Fortunato,et al.  Community detection in graphs , 2009, ArXiv.

[40]  Yifan Hu,et al.  Collaborative Filtering for Implicit Feedback Datasets , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[41]  Pedro Cano,et al.  From hits to niches?: or how popular artists can bias music recommendation and discovery , 2008, NETFLIX '08.

[42]  Yehuda Koren,et al.  Factorization meets the neighborhood: a multifaceted collaborative filtering model , 2008, KDD.

[43]  J. Buldú,et al.  Topology of music recommendation networks. , 2005, Chaos.

[44]  Claudio Castellano,et al.  Defining and identifying communities in networks. , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[45]  I. Cross.  Music, Cognition, Culture, and Evolution , 2001, Annals of the New York Academy of Sciences.

[46]  G. Stewart.  Perturbation theory for the singular value decomposition , 1990 .

[47]  Kathryn B. Laskey,et al.  Stochastic blockmodels: First steps , 1983 .

[48]  C. Eckart,et al.  The approximation of one matrix by another of lower rank , 1936 .

[49]  O. Perron.  Zur Theorie der Matrices , 1907 .

[50]  Spotify , 2020, The SAGE International Encyclopedia of Mass Media and Society.

[51]  Mohan S. Kankanhalli,et al.  Exploiting Music Play Sequence for Music Recommendation , 2017, IJCAI.

[52]  Sune Lehmann,et al.  Community Detection, Current and Future Research Trends , 2014, Encyclopedia of Social Network Analysis and Mining.

[53]  Gert R. G. Lanckriet,et al.  Hypergraph Models of Playlist Dialects , 2012, ISMIR.

[54]  Òscar Celma,et al.  Music recommendation and discovery in the long tail , 2008 .