Distilling the wisdom of crowds: weighted aggregation of decisions on multiple issues

Given the judgments of multiple voters regarding some issue, it is generally assumed that the best way to arrive at some collective judgment is by following the majority. We consider here the now common case in which each voter expresses some (binary) judgment regarding each of a multiplicity of independent issues and assume that each voter has some fixed (unknown) probability of making a correct judgment for any given issue. We leverage the fact that multiple votes by each voter are known in order to demonstrate, both analytically and empirically, that a method based on maximum likelihood estimation is superior to the simple majority rule for arriving at true collective judgments.

[1]  D. Berend,et al.  When is Condorcet's Jury Theorem valid? , 1998 .

[2]  Nicolas de Condorcet.  Essai Sur L'Application de L'Analyse a la Probabilite Des Decisions Rendues a la Pluralite Des Voix , 2009 .

[3]  New York Dover,et al.  ON THE CONVERGENCE PROPERTIES OF THE EM ALGORITHM , 1983 .

[4]  Ruth Ben-Yashar,et al.  A nonasymptotic Condorcet jury theorem , 2000, Soc. Choice Welf..

[5]  Drora Karotkin,et al.  Justification of the simple majority and chairman rules , 1996 .

[6]  C. List.  On the Significance of the Absolute Margin , 2002, The British Journal for the Philosophy of Science.

[7]  C. List,et al.  Aggregating Sets of Judgments: An Impossibility Result , 2002, Economics and Philosophy.

[8]  L. Shapley,et al.  Optimizing group judgmental accuracy in the presence of interdependencies , 1984 .

[9]  Vincent Conitzer,et al.  Common Voting Rules as Maximum Likelihood Estimators , 2005, UAI.

[10]  Shmuel Nitzan,et al.  Optimal Decision Rules in Uncertain Dichotomous Choice Situations , 1982 .

[11]  Shmuel Nitzan,et al.  A general theorem and eight corollaries in search of correct decision , 1994 .

[12]  Franz Dietrich,et al.  General Representation of Epistemically Optimal Procedures , 2006, Soc. Choice Welf..

[13]  Ron Holzman,et al.  Aggregation of binary evaluations for truth-functional agendas , 2009, Soc. Choice Welf..

[14]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .