Knowledge Market Design: A Field Experiment at Google Answers

In a field experiment at Google Answers, we investigate the performance of price-based online knowledge markets by systematically manipulating prices. Specifically, we study the effects of price, tip, and a reputation system on both an answerer's effort and answer quality by posting real reference questions from the Internet Public Library on Google Answers under different pricing schemes. We find that a higher price leads to a significantly longer, but not better, answer, while an answerer with a higher reputation provides significantly better answers. Our results highlight the limitation of monetary incentives and the importance of reputation systems in knowledge market design. Copyright © 2010 Wiley Periodicals, Inc..

[1]  J. Heckman.  Sample selection bias as a specification error , 1979 .

[2]  J. Fleiss,et al.  Intraclass correlations: uses in assessing rater reliability. , 1979, Psychological bulletin.

[3]  J F McDermott,et al.  An intervention to improve the reliability of manuscript reviews for the Journal of the American Academy of Child and Adolescent Psychiatry. , 1993, The American journal of psychiatry.

[4]  D. Hamermesh,et al.  Beauty, Productivity and Discrimination: Lawyers&Apos; Looks and Lucre , 1995 .

[5]  N. Black,et al.  Development of the review quality instrument (RQI) for assessing peer reviews of manuscripts. , 1999, Journal of clinical epidemiology.

[6]  D. Weiss,et al.  Interrater reliability and agreement. , 2000 .

[7]  Catherine C. Eckel,et al.  The Value of a Smile: Game Theory with a Human Face , 2001 .

[8]  Paul Resnick,et al.  The value of reputation on eBay: A controlled experiment , 2002 .

[9]  Richard McCrea.  Evaluation of two library‐based and one expert reference service on the Web , 2004 .

[10]  Barbara Howell,et al.  The Reliability of Peer Reviews of Papers on Information Systems , 2004, J. Inf. Sci..

[11]  Tobias Regner,et al.  Why Voluntary Contributions ? Google Answers ! , 2004 .

[12]  Rich Gazan.  Specialists and synthesists in a question answering community , 2006, ASIST.

[13]  Gilad Ravid,et al.  How social motivation enhances economic activity and incentives in the Google Answers knowledge sharing market , 2007, Int. J. Knowl. Learn..

[14]  Mark S. Ackerman,et al.  Competing to Share Expertise: The Taskcn Knowledge Sharing Community , 2021, ICWSM.

[15]  James Andreoni,et al.  Beauty, gender and stereotypes : Evidence from laboratory experiments , 2008 .

[16]  Darren Gergle,et al.  Emotion rating from short blog texts , 2008, CHI.

[17]  Lada A. Adamic,et al.  Knowledge sharing and yahoo answers: everyone knows something , 2008, WWW.

[18]  Chirag Shah,et al.  Exploring Characteristics and Effects of User Participation in Online Social Q&A Sites , 2008, First Monday.

[19]  Sheizaf Rafaeli,et al.  Predictors of answer quality in online Q&A sites , 2008, CHI.

[20]  Mark S. Ackerman,et al.  Questions in, knowledge in?: a study of naver's question answering community , 2009, CHI.

[21]  Yan Chen,et al.  Re-examining price as a predictor of answer quality in an online q&a site , 2010, CHI.

[22]  Roman M. Sheremeta,et al.  Visibility of Contributions and Cost of Information: An Experiment on Public Goods , 2010 .

[23]  Benjamin Edelman,et al.  Earnings and Ratings at Google Answers , 2012 .