Ensuring quality in crowdsourced search relevance evaluation: The effects of training question distribution
暂无分享,去创建一个
John Le | Andrew Edmonds | Vaughn Hester | Lukas A. Biewald | Lukas Biewald | A. Edmonds | V. Hester | John Le
[1] Daphne Koller,et al. Support Vector Machine Active Learning with Applications to Text Classification , 2000, J. Mach. Learn. Res..
[2] Benjamin Piwowarski,et al. Web Search Engine Evaluation Using Clickthrough Data and a User Model , 2007 .
[3] Panagiotis G. Ipeirotis,et al. Quality management on Amazon Mechanical Turk , 2010, HCOMP '10.
[4] Brendan T. O'Connor,et al. Cheap and Fast – But is it Good? Evaluating Non-Expert Annotations for Natural Language Tasks , 2008, EMNLP.
[5] Omar Alonso,et al. Crowdsourcing for relevance evaluation , 2008, SIGF.
[6] Omar Alonso,et al. Relevance criteria for e-commerce: a crowdsourcing-based experimental analysis , 2009, SIGIR.
[7] Lorrie Faith Cranor,et al. Are your participants gaming the system?: screening mechanical turk workers , 2010, CHI.
[8] David Maxwell Chickering,et al. Here or there: preference judgments for relevance , 2008 .
[9] Gabriella Kazai,et al. On the Evaluation of the Quality of Relevance Assessments Collected through Crowdsourcing , 2009 .
[10] Carol Peters,et al. Report on the SIGIR 2009 workshop on the future of IR evaluation , 2009, SIGF.