Developing the Measurement Scale of Information Quality for Social Q&A Sites

Social Q&A sites take advantage of the wisdom of crowds and produce huge number of answers every day. This research-in-progress paper aims to develop a measurement scale of Information Quality (IQ) for social Q&A from the perspective of the user perception. Based on the previous studies with a set of attributes, a conceptual framework of IQ in social Q&A context is developed. Five scholars in the IS field are also interviewed for helping us modify this scale. Baidu Knows, which is the most influential Chinese social Q&A website, is used to conduct the empirical study. This paper employs the Exploratory Factor Analysis (EFA) to test our preliminary measurement scale and make some further revisions. The findings show the IQ of social Q&A includes four dimensions, i.e., Content Quality, Source Quality, Performance Quality, and Emotion Quality with a total of 19 key indicators. This tentative study of IQ evaluation is effective in identifying high-quality answers on social Q&A sites. Moreover, it also has the potential to contribute to the interaction design of social Q&A sites.

[1]  W. Bruce Croft,et al.  A framework to predict the quality of answers with non-textual features , 2006, SIGIR.

[2]  Alton Yeow-Kuan Chua,et al.  Comparative Evaluation of Community Question Answering Websites , 2012, ICADL.

[3]  Carol L. Barry,et al.  Users' Criteria for Relevance Evaluation: A Cross-situational Comparison , 1998, Inf. Process. Manag..

[4]  Soo Young Rieh,et al.  Developing a unifying framework of credibility assessment: Construct, heuristics, and interaction in context , 2008, Inf. Process. Manag..

[5]  Tat-Seng Chua,et al.  Discovering high quality answers in community question answering archives using a hierarchy of classifiers , 2014, Inf. Sci..

[6]  Kexin Zhao,et al.  Information Quality and System Quality in Online Communities: an Empirical Investigation , 2009 .

[7]  Soo Young Rieh.  Judgment of information quality and cognitive authority in the Web , 2002, J. Assoc. Inf. Sci. Technol..

[8]  Adam Worrall,et al.  Quality evaluation of health answers in Yahoo! Answers: A comparison between experts and users , 2011, ASIST.

[9]  Christy M. K. Cheung,et al.  Why users keep answering questions in online question answering communities: A theoretical and empirical investigation , 2013, Int. J. Inf. Manag..

[10]  Alton Yeow-Kuan Chua,et al.  Measuring the effectiveness of answers in Yahoo! Answers , 2015, Online Inf. Rev..

[11]  Rich Gazan,et al.  Seekers, sloths and social reference: Homework questions submitted to a question-answering community , 2007, New Rev. Hypermedia Multim..

[12]  Richard Y. Wang,et al.  Anchoring data quality dimensions in ontological foundations , 1996, CACM.

[13]  Bo Li,et al.  Algorithm for recommending answer providers in community-based question answering , 2012, J. Inf. Sci..

[14]  Chirag Shah,et al.  Modalities, motivations, and materials – investigating traditional and social online Q&A services , 2014, J. Inf. Sci..

[15]  Soo Young Rieh.  Judgement of information quality and cognitive authority in the Web , 2002 .

[16]  Wen-Lian Hsu,et al.  journal homepage: www.elsevier.com/locate/im , 2022 .

[17]  Iryna Gurevych,et al.  A Multi-Dimensional Model for Assessing the Quality of Answers in Social Q&A Sites , 2009, ICIQ.

[18]  Qinghua Zhu,et al.  An integrated framework of online generative capability: interview from digital immigrants , 2014, Aslib J. Inf. Manag..

[19]  Ephraim R. McLean,et al.  The DeLone and McLean Model of Information Systems Success: A Ten-Year Update , 2003, J. Manag. Inf. Syst..

[20]  Sanghee Oh,et al.  Users' relevance criteria for evaluating answers in a social Q&A site , 2009, J. Assoc. Inf. Sci. Technol..

[21]  Sung-Hyon Myaeng,et al.  Effects of answer weight boosting in strategy-driven question answering , 2012, Inf. Process. Manag..

[22]  Eugene Agichtein,et al.  Discovering authorities in question answer communities by using link analysis , 2007, CIKM '07.

[23]  Pnina Shachaf,et al.  The paradox of expertise: is the Wikipedia Reference Desk as good as your library? , 2009, J. Documentation.

[24]  Olivia R. Liu Sheng,et al.  Examining the Technology Acceptance Model Using Physician Acceptance of Telemedicine Technology , 1999, J. Manag. Inf. Syst..

[25]  Felix Naumann,et al.  Assessment Methods for Information Quality Criteria , 2000, IQ.

[26]  Shengrui Wang,et al.  Identifying authoritative actors in question-answering forums: the case of Yahoo! answers , 2008, KDD.

[27]  Tefko Saracevic.  Relevance: A review of the literature and a framework for thinking on the notion in information science. Part III: Behavior and effects of relevance , 2007 .

[28]  H. Kaiser,et al.  A Study Of A Measure Of Sampling Adequacy For Factor-Analytic Correlation Matrices. , 1977, Multivariate behavioral research.

[29]  Bo Qu,et al.  An evaluation of classification models for question topic categorization , 2012, J. Assoc. Inf. Sci. Technol..

[30]  Sanghee Oh,et al.  Best-answer selection criteria in a social Q&A site from the user-oriented relevance perspective , 2008, ASIST.

[31]  Wing S. Chow,et al.  An Exploratory Study of the Success Factors for Extranet Adoption in E-Supply Chain , 2004, J. Glob. Inf. Manag..

[32]  Les Gasser,et al.  Assessing Information Quality of a Community-Based Encyclopedia , 2005, ICIQ.

[33]  Atreyi Kankanhalli,et al.  Task and Social Information Seeking: Whom Do We Prefer and Whom Do We Approach? , 2010, J. Manag. Inf. Syst..

[34]  Rich Gazan,et al.  Social Q&A , 2011, J. Assoc. Inf. Sci. Technol..

[35]  Pnina Fichman,et al.  A comparative assessment of answer quality on four question answering sites , 2011, J. Inf. Sci..

[36]  Alton Yeow-Kuan Chua,et al.  Selection of the Best Answer in CQA Services , 2010, 2010 Seventh International Conference on Information Technology: New Generations.

[37]  Pnina Fichman,et al.  Information Quality on Yahoo! Answers , 2013 .

[38]  Kaushik Dutta,et al.  Identifying Fake Profiles in LinkedIn , 2020, PACIS.

[39]  C. Dziuban,et al.  When is a correlation matrix appropriate for factor analysis? Some decision rules. , 1974 .

[40]  Pnina Shachaf,et al.  Answer Reliability on Q&A Sites , 2010, AMCIS.

[41]  Anne Morris,et al.  Web Wisdom: How to Evaluate and Create Information Quality on the Web , 2000 .

[42]  Ephraim R. McLean,et al.  Information systems success revisited , 2002, Proceedings of the 35th Annual Hawaii International Conference on System Sciences.

[43]  Nga-man Wong,et al.  Quality evaluation of Geriatric Health Information on Yahoo! Answers: a cross-cultural comparative study , 2013 .

[44]  Shirlee-ann Knight,et al.  User perceptions of information quality in world wide web information retrieval behaviour , 2007 .

[45]  Jeffrey Pomerantz,et al.  Evaluating and predicting answer quality in community QA , 2010, SIGIR.

[46]  Ephraim R. McLean,et al.  Information Systems Success: The Quest for the Dependent Variable , 1992, Inf. Syst. Res..

[47]  Alton Yeow-Kuan Chua,et al.  Predictors of High-Quality Answers , 2012, Online Inf. Rev..

[48]  Peter A. Gloor,et al.  Understanding the effect of social networks on user behaviors in community-driven knowledge services , 2011, J. Assoc. Inf. Sci. Technol..

[49]  Adam Worrall,et al.  Quality evaluation of health answers in social Q&A: Socio-emotional support and evaluation criteria , 2012, ASIST.

[50]  Pnina Shachaf.  Social reference: Toward a unifying theory , 2010 .

[51]  Josu Takala,et al.  Factor Retention Decisions in Exploratory Factor Analysis Results: A Study Type of Knowledge Management Process at Malaysian University Libraries , 2013 .

[52]  Diane M. Strong,et al.  Beyond Accuracy: What Data Quality Means to Data Consumers , 1996, J. Manag. Inf. Syst..

[53]  Snehasish Banerjee,et al.  So fast so good: An analysis of answer quality and answer speed in community Question-answering sites , 2013, J. Assoc. Inf. Sci. Technol..

[54]  Chung-Tzer Liu,et al.  Loyalty of Web 2.0 Sites: The Role of Sense of Belonging , 2010, PACIS.

[55]  Sheizaf Rafaeli,et al.  Predictors of answer quality in online Q&A sites , 2008, CHI.

[56]  Rich Gazan.  Specialists and synthesists in a question answering community , 2006, ASIST.

[57]  Roman Lukyanenko,et al.  The IQ of the Crowd: Understanding and Improving Information Quality in Structured User-Generated Content , 2014, Inf. Syst. Res..

[58]  Anita Sarma,et al.  Perceptions of answer quality in an online technical question and answer forum , 2014, CHASE.

[59]  Roman Lukyanenko,et al.  Information Quality Research Challenge: Adapting Information Quality Principles to User-Generated Content , 2015, JDIQ.

[60]  Ed H. Chi,et al.  Towards a model of understanding social search , 2008, SSM '08.

[61]  Michael Vitale,et al.  The Wisdom of Crowds , 2015, Cell.

[62]  J. Oh,et al.  Research agenda for social Q&A , 2009 .

[63]  Soo Young Rieh,et al.  Answers from the crowd: How credible are strangers in social Q&A? , 2014 .

[64]  Alton Yeow-Kuan Chua,et al.  A predictive framework for retrieving the best answer , 2008, SAC '08.

[65]  E. Hargittai,et al.  Toward a Social Framework for Information Seeking , 2006 .

[66]  H. Kaiser.  A second generation little jiffy , 1970 .

[67]  T. Saracevic.  Relevance: A Review of the Literature and a Framework for Thinking on the Notion in Information Science. Part II , 2006 .

[68]  Peter Ingwersen,et al.  Dimensions of relevance , 2000, Inf. Process. Manag..

[69]  Susan Gauch,et al.  Incorporating quality metrics in centralized/distributed information retrieval on the World Wide Web , 2000, SIGIR '00.

[70]  Fatemeh Zahedi,et al.  The Measurement of Web-Customer Satisfaction: An Expectation and Disconfirmation Approach , 2002, Inf. Syst. Res..

[71]  Rich Gazan,et al.  Microcollaborations in a social Q&A community , 2010, Inf. Process. Manag..