LinkLive: discovering Web learning resources for developers from Q&A discussions

Software developers need access to correlated information (e.g., API documentation, Wikipedia pages, Stack Overflow questions and answers) which are often dispersed among different Web resources. This paper is concerned with the situation where a developer is visiting a Web page, but at the same time is willing to explore correlated Web resources to extend his/her knowledge or to satisfy his/her curiosity. Specifically, we present an item-based collaborative filtering technique, named LinkLive, for automatically recommending a list of correlated Web resources for a particular Web page. The recommendation is done by exploiting hyperlink associations from the crowdsourced knowledge on Stack Overflow. We motivate our research using an exploratory study of hyperlink dissemination patterns on Stack Overflow. We then present our LinkLive technique that uses multiple features, including hyperlink co-occurrences in Q&A discussions, locations (e.g., question, answer, or comment) in which hyperlinks are referenced, and votes for posts/comments in which hyperlinks are referenced. Experiments using 7 years of Stack Overflow data show that, our technique recommends correlated Web resources with promising accuracy in an open setting. A user study of 6 participants suggests that practitioners find the recommended Web resources useful for Web discovery.

[1]  Tie-Yan Liu,et al.  Learning to Rank for Information Retrieval , 2011 .

[2]  Ebrahim Bagheri,et al.  Semantic tagging and linking of software engineering social content , 2014, Automated Software Engineering.

[3]  Mao Ye,et al.  Exploiting geographical influence for collaborative point-of-interest recommendation , 2011, SIGIR.

[4]  Òscar Celma,et al.  Foafing the Music: Bridging the Semantic Gap in Music Recommendation , 2006, SEMWEB.

[5]  Jing Li,et al.  Software-specific part-of-speech tagging: an experimental study on stack overflow , 2016, SAC.

[6]  Roi Blanco,et al.  Entity Recommendations in Web Search , 2013, SEMWEB.

[7]  Ming Zhou,et al.  Learning Sentiment-Specific Word Embedding for Twitter Sentiment Classification , 2014, ACL.

[8]  Carol Tenopir,et al.  Communication Patterns of Engineers , 2004 .

[9]  Juan C. Burguillo,et al.  A hybrid content-based and item-based collaborative filtering approach to recommend TV programs enhanced with singular value decomposition , 2010, Inf. Sci..

[10]  Roi Blanco,et al.  From "Selena Gomez" to "Marlon Brando": Understanding Explorative Entity Search , 2015, WWW.

[11]  Mehrbakhsh Nilashi,et al.  A recommender system based on collaborative filtering using ontology and dimensionality reduction techniques , 2018, Expert Syst. Appl..

[12]  Jiawei Han,et al.  On building entity recommender systems using user click log and freebase knowledge , 2014, WSDM.

[13]  George Karypis,et al.  Evaluation of Item-Based Top-N Recommendation Algorithms , 2001, CIKM '01.

[14]  Michael S. Bernstein,et al.  Direct answers for search queries in the long tail , 2012, CHI.

[15]  Scott Sanner,et al.  AutoRec: Autoencoders Meet Collaborative Filtering , 2015, WWW.

[16]  Jing Li,et al.  From discussion to wisdom: web resource recommendation for hyperlinks in stack overflow , 2016, SAC.

[17]  Po Hu,et al.  Learning Continuous Word Embedding with Metadata for Question Retrieval in Community Question Answering , 2015, ACL.

[18]  Jing Li,et al.  Learning to answer programming questions with software documentation through social context embedding , 2018, Inf. Sci..

[19]  Jaideep Srivastava,et al.  Web mining: information and pattern discovery on the World Wide Web , 1997, Proceedings Ninth IEEE International Conference on Tools with Artificial Intelligence.

[20]  Jing Li,et al.  Software-Specific Named Entity Recognition in Software Engineering Social Content , 2016, 2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER).

[21]  Peter Brusilovsky,et al.  Open user profiles for adaptive news systems: help or harm? , 2007, WWW '07.

[22]  Xuejiao Zhao,et al.  amAssist: In-IDE ambient search of online programming resources , 2015, 2015 IEEE 22nd International Conference on Software Analysis, Evolution, and Reengineering (SANER).

[23]  Christoph Treude,et al.  Crowd Documentation : Exploring the Coverage and the Dynamics of API Discussions on Stack Overflow , 2012 .

[24]  Carlo Strapparava,et al.  Improving User Modelling with Content-Based Techniques , 2001, User Modeling.

[25]  J. Bobadilla,et al.  Recommender systems survey , 2013, Knowl. Based Syst..

[26]  Paolo Tomeo,et al.  Content-Based Recommendations via DBpedia and Freebase: A Case Study in the Music Domain , 2015, International Semantic Web Conference.

[27]  David Heckerman,et al.  Empirical Analysis of Predictive Algorithms for Collaborative Filtering , 1998, UAI.

[28]  Gediminas Adomavicius,et al.  Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions , 2005, IEEE Transactions on Knowledge and Data Engineering.

[29]  Qingsheng Zhu,et al.  Incremental Collaborative Filtering recommender based on Regularized Matrix Factorization , 2012, Knowl. Based Syst..

[30]  Jun Wang,et al.  Unifying user-based and item-based collaborative filtering approaches by similarity fusion , 2006, SIGIR.

[31]  Erkan Bostanci,et al.  A genetic algorithm solution to the collaborative filtering problem , 2016, Expert Syst. Appl..

[32]  Michael J. Pazzani,et al.  User Modeling for Adaptive News Access , 2000, User Modeling and User-Adapted Interaction.

[33]  Qi Zhang,et al.  Hashtag Recommendation Using Attention-Based Convolutional Neural Network , 2016, IJCAI.

[34]  Jure Leskovec,et al.  Discovering value from community activity on focused question answering sites: a case study of stack overflow , 2012, KDD.

[35]  Jing Li,et al.  BPMiner: mining developers' behavior patterns from screen-captured task videos , 2016, SAC.

[36]  Yoav Shoham,et al.  Fab: content-based, collaborative recommendation , 1997, CACM.

[37]  Amiram Yehudai,et al.  Example Overflow: Using social media for code recommendation , 2012, 2012 Third International Workshop on Recommendation Systems for Software Engineering (RSSE).

[38]  Michael Gamon,et al.  Active objects: actions for entity-centric search , 2012, WWW.

[39]  Ahmed Eldawy,et al.  LARS: A Location-Aware Recommender System , 2012, 2012 IEEE 28th International Conference on Data Engineering.

[40]  Ian Soboroff. Charles Nicholas.  Combining Content and Collaboration in Text Filtering , 1999 .

[41]  Greg Linden,et al.  Amazon . com Recommendations Item-to-Item Collaborative Filtering , 2001 .

[42]  Anoop K. Sinha,et al.  Suede: a Wizard of Oz prototyping tool for speech user interfaces , 2000, UIST '00.

[43]  Gao Cong,et al.  Graph-based Point-of-interest Recommendation with Geographical and Temporal Influences , 2014, CIKM.

[44]  Pattie Maes,et al.  Evolving agents for personalized information filtering , 1993, Proceedings of 9th IEEE Conference on Artificial Intelligence for Applications.

[45]  Alexandros Karatzoglou,et al.  Question recommendation for collaborative question answering systems with RankSLDA , 2014, RecSys '14.

[46]  Christoph Seebach,et al.  Searching for Answers--Knowledge Exchange through Social Media in Organizations , 2012, 2012 45th Hawaii International Conference on System Sciences.

[47]  Rozana Zakaria,et al.  A multi-criteria recommendation system using dimensionality reduction and Neuro-Fuzzy techniques , 2015, Soft Comput..

[48]  David M. Pennock,et al.  Applying collaborative filtering techniques to movie search for better ranking and browsing , 2007, KDD '07.

[49]  Yu Li,et al.  A hybrid collaborative filtering method for multiple-interests and multiple-content recommendation in E-Commerce , 2005, Expert Syst. Appl..

[50]  H. Lilliefors.  On the Kolmogorov-Smirnov Test for Normality with Mean and Variance Unknown , 1967 .

[51]  David Lo,et al.  An empirical study on developer interactions in StackOverflow , 2013, SAC '13.

[52]  Michael J. Pazzani,et al.  Learning and Revising User Profiles: The Identification of Interesting Web Sites , 1997, Machine Learning.

[53]  Alexandros Karatzoglou,et al.  Session-based Recommendations with Recurrent Neural Networks , 2015, ICLR.

[54]  Ricardo Baeza-Yates,et al.  Essential Web Pages Are Easy to Find , 2015, WWW.

[55]  John Riedl,et al.  Item-based collaborative filtering recommendation algorithms , 2001, WWW '01.

[56]  Raymond J. Mooney,et al.  Content-boosted collaborative filtering for improved recommendations , 2002, AAAI/IAAI.

[57]  Tetsuya Sakai,et al.  Serendipitous search via wikipedia: a query log analysis , 2009, SIGIR.

[58]  David M. Pennock,et al.  Categories and Subject Descriptors , 2001 .

[59]  Pasquale Lops,et al.  A content-collaborative recommender that exploits WordNet-based user profiles for neighborhood formation , 2007, User Modeling and User-Adapted Interaction.

[60]  Christoph Treude,et al.  How do programmers ask and answer questions on the web?: NIER track , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[61]  Yunming Ye,et al.  DeepFM: A Factorization-Machine based Neural Network for CTR Prediction , 2017, IJCAI.

[62]  Lina Yao,et al.  Deep Learning Based Recommender System , 2017, ACM Comput. Surv..

[63]  Abhinandan Das,et al.  Google news personalization: scalable online collaborative filtering , 2007, WWW '07.

[64]  Svetha Venkatesh,et al.  Collaborative filtering via sparse Markov random fields , 2016, Inf. Sci..

[65]  Andreas Zeller,et al.  eROSE: guiding programmers in eclipse , 2005, OOPSLA '05.

[66]  Volker Tresp,et al.  Type-Constrained Representation Learning in Knowledge Graphs , 2015, SEMWEB.

[67]  Peng Zhang,et al.  IRGAN: A Minimax Game for Unifying Generative and Discriminative Information Retrieval Models , 2017, SIGIR.

[68]  Loriene Roy,et al.  Content-based book recommending using learning for text categorization , 1999, DL '00.

[69]  Zhenchang Xing,et al.  Leveraging Official Content and Social Context to Recommend Software Documentation , 2018, IEEE Transactions on Services Computing.

[70]  Jonathan L. Herlocker,et al.  A collaborative filtering algorithm and evaluation metric that accurately model the user experience , 2004, SIGIR '04.

[71]  Tat-Seng Chua,et al.  Neural Collaborative Filtering , 2017, WWW.

[72]  Christoph Treude,et al.  Augmenting API Documentation with Insights from Stack Overflow , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE).

[73]  Michael McGill,et al.  Introduction to Modern Information Retrieval , 1983 .

[74]  Gary Marchionini,et al.  Exploratory search , 2006, Commun. ACM.

[75]  Robert J. Walker,et al.  Strathcona example recommendation tool , 2005, ESEC/FSE-13.

[76]  David Lo,et al.  EnTagRec: An Enhanced Tag Recommendation System for Software Information Sites , 2014, ICSME.

[77]  Gabriele Bavota,et al.  Prompter: A Self-Confident Recommender System , 2014, 2014 IEEE International Conference on Software Maintenance and Evolution.

[78]  Nemanja Djuric,et al.  E-commerce in Your Inbox: Product Recommendations at Scale , 2015, KDD.

[79]  Michele Lanza,et al.  Seahawk: Stack Overflow in the IDE , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[80]  Feng-hsu Wang,et al.  An Effective Content-based Recommendation Method for Web Browsing Based on Keyword Context Matching , 2006 .

[81]  Leif Singer,et al.  A study of innovation diffusion through link sharing on stack overflow , 2013, 2013 10th Working Conference on Mining Software Repositories (MSR).

[82]  Reid Holmes,et al.  Live API documentation , 2014, ICSE.

[83]  Philip J. Guo,et al.  Two studies of opportunistic programming: interleaving web foraging, learning, and writing code , 2009, CHI.

[84]  Ryen W. White,et al.  Exploratory Search: Beyond the Query-Response Paradigm , 2009, Exploratory Search: Beyond the Query-Response Paradigm.

[85]  I. Bretherton,et al.  Attachment theory: Retrospect and prospect. , 1985 .

[86]  Zhenchang Xing,et al.  API Caveat Explorer -- Surfacing Negative Usages from Practice: An API-oriented Interactive Exploratory Search System for Programmers , 2018, SIGIR.

[87]  Taghi M. Khoshgoftaar,et al.  A Survey of Collaborative Filtering Techniques , 2009, Adv. Artif. Intell..

[88]  Andrei Broder,et al.  A taxonomy of web search , 2002, SIGF.

[89]  Ji-Rong Wen,et al.  Using anchor texts with their hyperlink structure for web search , 2009, SIGIR.

[90]  Uday V. Kulkarni,et al.  Hybrid personalized recommender system using centering-bunching based clustering algorithm , 2012, Expert Syst. Appl..

[91]  Geoffrey E. Hinton,et al.  Restricted Boltzmann machines for collaborative filtering , 2007, ICML '07.

[92]  Ronald R. Yager,et al.  Fuzzy logic methods in recommender systems , 2003, Fuzzy Sets Syst..

[93]  Sung-Bae Cho,et al.  Location-Based Recommendation System Using Bayesian User's Preference Model in Mobile Devices , 2007, UIC.