Fraud Detection by Human Agents: A Pilot Study

Fraud is a constant problem for online auction sites. Besides failures in detecting fraudsters, the currently employed methods yield many false positives: bona fide sellers that end up harassed by the auction site as suspects. We advocate the use of human computation (also called crowdsourcing) to improve precision and recall of current fraud detection techniques. To examine the feasibility of our proposal, we did a pilot study with a set of human subjects, testing whether they could distinguish fraudsters from common sellers before negative feedback arrived and looking just at a snapshot of seller profiles. Here we present the methodology used and the obtained results, in terms of precision and recall of human classifiers, showing positive evidence that detecting fraudsters with human computation is viable.