Expertise and Dynamics within Crowdsourced Musical Knowledge Curation: A Case Study of the Genius Platform

Many platforms collect crowdsourced information primarily from volunteers. As this type of knowledge curation has become widespread, contribution formats vary substantially and are driven by diverse processes across differing platforms. Thus, models for one platform are not necessarily applicable to others. Here, we study the temporal dynamics of Genius, a platform primarily designed for user-contributed annotations of song lyrics. A unique aspect of Genius is that the annotations are extremely local -- an annotated lyric may just be a few lines of a song -- but also highly related, e.g., by song, album, artist, or genre. We analyze several dynamical processes associated with lyric annotations and their edits, which differ substantially from models for other platforms. For example, expertise on song annotations follows a ``U shape'' where experts are both early and late contributors with non-experts contributing intermediately; we develop a user utility model that captures such behavior. We also find several contribution traits appearing early in a user's lifespan of contributions that distinguish (eventual) experts from non-experts. Combining our findings, we develop a model for early prediction of user expertise.

[1]  Robert E. Kraut,et al.  Early detection of potential experts in question answering communities , 2011, UMAP'11.

[2]  Ivan Beschastnikh,et al.  Wikipedian Self-Governance in Action: Motivating the Policy Lens , 2021, ICWSM.

[3]  Rishabh Mehrotra,et al.  The Music Streaming Sessions Dataset , 2018, WWW.

[4]  Reinventing Genius in the .com Age: Austrian Rap Music and a New Way of Knowing , 2018 .

[5]  A. Swartz MusicBrainz: A Semantic Web Service , 2002, IEEE Intell. Syst..

[6]  Tsai-Ching Lu,et al.  Temporal Motifs Reveal the Dynamics of Editor Interactions in Wikipedia , 2012, ICWSM.

[7]  Animesh Mukherjee,et al.  Analysis and Prediction of Question Topic Popularity in Community Q&A Sites: A Case Study of Quora , 2015, ICWSM.

[8]  Jure Leskovec,et al.  From amateurs to connoisseurs: modeling the evolution of user expertise through online reviews , 2013, WWW.

[9]  Jure Leskovec,et al.  No country for old members: user lifecycle and linguistic change in online communities , 2013, WWW.

[10]  Alexandros Tsaptsinos Lyrics-Based Music Genre Classification Using a Hierarchical Attention Network , 2017, ISMIR.

[11]  Adrian Paschke,et al.  Investigating the Effect of Attributes on User Trust in Social Media , 2018, DEXA Workshops.

[12]  Nicole Novielli,et al.  Mining Successful Answers in Stack Overflow , 2015, 2015 IEEE/ACM 12th Working Conference on Mining Software Repositories.

[13]  F. Maxwell Harper,et al.  Exploring Question Selection Bias to Identify Experts and Potential Experts in Community Question Answering , 2012, TOIS.

[14]  Christoph Treude,et al.  Crowd Documentation : Exploring the Coverage and the Dynamics of API Discussions on Stack Overflow , 2012 .

[15]  Premkumar T. Devanbu,et al.  Mining Stack Exchange: Expertise Is Evident from Initial Contributions , 2012, 2012 International Conference on Social Informatics.

[16]  Taha Yasseri,et al.  The distorted mirror of Wikipedia: a quantitative analysis of Wikipedia coverage of academics , 2013, EPJ Data Science.

[17]  Jure Leskovec,et al.  Motifs in Temporal Networks , 2016, WSDM.

[18]  Markus Schedl,et al.  Music Information Retrieval: Recent Developments and Applications , 2014, Found. Trends Inf. Retr..

[19]  Yanjiao Chen,et al.  An Incentive Mechanism for Crowdsourcing Systems with Network Effects , 2019, ACM Trans. Internet Techn..

[20]  Jure Leskovec,et al.  Steering user behavior with badges , 2013, WWW.

[21]  Ji-Rong Wen,et al.  Characterizing and Predicting Early Reviewers for Effective Product Marketing on E-Commerce Websites , 2018, IEEE Transactions on Knowledge and Data Engineering.

[22]  Yehuda Koren,et al.  Yahoo! music recommendations: modeling music ratings with temporal dynamics and item taxonomy , 2011, RecSys '11.

[23]  Thierry Bertin-Mahieux,et al.  The Million Song Dataset , 2011, ISMIR.

[24]  Finn Årup Nielsen,et al.  “The sum of all human knowledge”: A systematic review of scholarly research on the content of Wikipedia , 2015, J. Assoc. Inf. Sci. Technol..

[25]  Baoxin Li,et al.  Towards Predicting the Best Answers in Community-based Question-Answering Services , 2013, ICWSM.

[26]  Kyumin Lee,et al.  Detecting experts on Quora: by their activity, quality of answers, linguistic characteristics and temporal behaviors , 2016, Social Network Analysis and Mining.

[27]  Mark S. Ackerman,et al.  Expertise networks in online communities: structure and algorithms , 2007, WWW '07.

[28]  Yair Movshovitz-Attias,et al.  Analysis of the reputation system and user contributions on a question answering website: StackOverflow , 2013, 2013 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM 2013).

[29]  Joshua Evan Blumenstock,et al.  Size matters: word count as a measure of quality on wikipedia , 2008, WWW.

[30]  John Domingue,et al.  It's all in the content: state of the art best answer prediction based on discretisation of shallow linguistic features , 2014, WebSci '14.

[31]  Sunil Kumar,et al.  Congestible services and network effects , 2010, EC '10.

[32]  Ed H. Chi,et al.  Want to be Retweeted? Large Scale Analytics on Factors Impacting Retweet in Twitter Network , 2010, 2010 IEEE Second International Conference on Social Computing.

[33]  D. Jannach,et al.  Music Recommendation , 2016 .

[34]  Jure Leskovec,et al.  Loyalty in Online Communities , 2017, ICWSM.

[35]  Ahmed Hadj Kacem,et al.  How to Organize the Annotation Systems in Human-Computer Environment: Study, Classification and Observations , 2015, INTERACT.

[36]  Peng Qi,et al.  The Evolution of Wikipedia , 2013 .

[37]  David van Dijk,et al.  Early Detection of Topical Expertise in Community Question Answering , 2015, SIGIR.

[38]  Ravi Kumar,et al.  Great Question! Question Quality in Community Q&A , 2014, ICWSM.

[39]  Jure Leskovec,et al.  Discovering value from community activity on focused question answering sites: a case study of stack overflow , 2012, KDD.

[40]  J. Giles Internet encyclopaedias go head to head , 2005, Nature.

[41]  Gang Wang,et al.  Wisdom in the social crowd: an analysis of quora , 2013, WWW.

[42]  Eric Gilbert,et al.  Understanding deja reviewers , 2010, CSCW '10.

[43]  Lada A. Adamic,et al.  Knowledge sharing and yahoo answers: everyone knows something , 2008, WWW.

[44]  Les Gasser,et al.  Assessing Information Quality of a Community-Based Encyclopedia , 2005, ICIQ.

[45]  Jamal Al Qundus Technical Analysis of the Social Media Platform Genius , 2018 .

[46]  Adam R. Brown Wikipedia as a Data Source for Political Scientists: Accuracy and Completeness of Coverage , 2011, PS: Political Science & Politics.

[47]  Ye Wang,et al.  Quantifying Lexical Novelty in Song Lyrics , 2015, ISMIR.