Bad Users or Bad Content?: Breaking the Vicious Cycle by Finding Struggling Students in Community Question-Answering

Community Question Answering (CQA) services have become popular methods to seek and share information. In CQA, users with an information need, or askers, post a question that community members can answer. This question-answering process allows both askers and answerers to learn through the exchange of information. CQA services have also been widely used in the education domain, as some of such services are designed specifically for students' information seeking. However, due to insufficient knowledge, lack of experience, and other reasons, students often struggle in producing quality or even appropriate content. This low quality production causes their content to be flagged or deleted, further discouraging them from participating in the CQA process and instigating a vicious cycle of bad users and bad content. In an effort to break this cycle, the work reported here focuses on identifying users whose postings demonstrate a high deletion rate with a presumption that the bad content is an indication of a struggling student rather than a malicious user. In this work, experiments are conducted on a large student-oriented online CQA community to understand struggling students' behaviors. A framework is proposed to find these users based solely on their activities. Finally, community feedback (i.e. human judgment) such as moderator evaluation or community votes for good content is used to detect these users in the early stages of their respective struggles. To evaluate this framework, we used data from Brainly, a large educational CQA service that is used in two different markets with more than 3.7 million users and 10.7 million answers. The results show that the human judgment feature identifies early-stage struggling users with high accuracy. Identifying these struggling users (students) could help educators to determine suitable ways to help their students instead of presuming them to be bad users and cutting them off from the community.

[1]  Hanghang Tong,et al.  MET: A Fast Algorithm for Minimizing Propagation in Large Graphs with Small Eigen-Gaps , 2015, SDM.

[2]  Eric Horvitz,et al.  Why Stop Now? Predicting Worker Engagement in Online Crowdsourcing , 2013, HCOMP.

[3]  Gang Wang,et al.  Wisdom in the social crowd: an analysis of quora , 2013, WWW.

[4]  Jennifer Preece,et al.  The top five reasons for lurking: improving community experiences for everyone , 2004, Comput. Hum. Behav..

[5]  Yoelle Maarek,et al.  Can You Believe an Anonymous Contributor? On Truthfulness in Yahoo! Answers , 2012, 2012 International Conference on Privacy, Security, Risk and Trust and 2012 International Confernece on Social Computing.

[6]  Ee-Peng Lim,et al.  Characterizing Silent Users in Social Media Communities , 2015, ICWSM.

[7]  Ivan Srba,et al.  Askalot: Community Question Answering as a Means for Knowledge Sharing in an Educational Organization , 2015, CSCW Companion.

[8]  Yossi Richter,et al.  Predicting Customer Churn in Mobile Networks through Analysis of Social Groups , 2010, SDM.

[9]  Erik Choi,et al.  Utilizing content moderators to investigate critical factors for assessing the quality of answers on brainly, social learning Q&A platform for students: A pilot study , 2015, ASIST.

[10]  Chun How Tan,et al.  Trust, but verify: predicting contribution quality for knowledge base construction and curation , 2014, WSDM.

[11]  Jeffrey Pomerantz,et al.  Evaluating and predicting answer quality in community QA , 2010, SIGIR.

[12]  Chirag Shah,et al.  "How much change do you get from 40$?" - Analyzing and addressing failed questions on social Q&A , 2012, ASIST.

[13]  N. Hari Narayanan,et al.  Facilitating students' collaboration and learning in a question and answer system , 2013, CSCW '13.

[14]  Susan T. Dumais,et al.  Understanding User Behavior Through Log Data and Analysis , 2014, Ways of Knowing in HCI.

[15]  Imed Zitouni,et al.  Understanding User Satisfaction with Intelligent Assistants , 2016, CHIIR.

[16]  Lada A. Adamic,et al.  Knowledge sharing and yahoo answers: everyone knows something , 2008, WWW.

[17]  R. P. Fishburne,et al.  Derivation of New Readability Formulas (Automated Readability Index, Fog Count and Flesch Reading Ease Formula) for Navy Enlisted Personnel , 1975 .

[18]  David Lo,et al.  Collective Churn Prediction in Social Network , 2012, 2012 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining.

[19]  Michael Vitale,et al.  The Wisdom of Crowds , 2015, Cell.

[20]  Chirag Shah,et al.  Evaluating the quality of educational answers in community question-answering , 2016, 2016 IEEE/ACM Joint Conference on Digital Libraries (JCDL).

[21]  Jure Leskovec,et al.  Antisocial Behavior in Online Discussion Communities , 2015, ICWSM.

[22]  Rayid Ghani,et al.  A Machine Learning Framework to Identify Students at Risk of Adverse Academic Outcomes , 2015, KDD.

[23]  Luanne Freund,et al.  Investigating the Role of User Engagement in Digital Reading Environments , 2016, CHIIR.

[24]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[25]  Joseph A. Konstan,et al.  Evolution of Experts in Question Answering Communities , 2012, ICWSM.

[26]  Pei-Luen Patrick Rau,et al.  Understanding lurkers in online communities: A literature review , 2014, Comput. Hum. Behav..

[27]  Maurice Tchuente,et al.  Churn Prediction in a Real Online Social Network Using Local CommunIty Analysis , 2012, 2012 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining.

[28]  Laks V. S. Lakshmanan,et al.  On minimizing budget and time in influence propagation over social networks , 2012, Social Network Analysis and Mining.

[29]  Nitesh V. Chawla,et al.  Data Mining for Imbalanced Datasets: An Overview , 2005, The Data Mining and Knowledge Discovery Handbook.

[30]  L. Toledo-Pereyra.  Trust , 2006, Mediation Behaviour.

[31]  Meredith Ringel Morris,et al.  Factors Affecting Response Quantity, Quality, and Speed for Questions Asked Via Social Network Status Messages , 2011, ICWSM.

[32]  Ryen W. White,et al.  Effects of community size and contact rate in synchronous social q&a , 2011, CHI.

[33]  Matthew Lease,et al.  Modeling Temporal Crowd Work Quality with Limited Supervision , 2015, HCOMP.

[34]  Chirag Shah,et al.  Retrieving Rising Stars in Focused Community Question-Answering , 2016, ACIIDS.

[35]  Danai Koutra,et al.  Network similarity via multiple social theories , 2013, 2013 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM 2013).