Corralling crowd power

retainer, enabling time-sensitive applications such as helping blind users navigate their surroundings. The quality of crowd work has increased by orders of magnitude due to research ranging from improved task design (for example, using Bayesian Truth Serum where workers predict others’ answers), to leveraging workers’ behavioral traces (for example, looking at the way workers do their work instead of their output), to inferring worker quality across tasks and reweighting their influence accordingly. Perhaps the most important question for the future of crowd work is whether it is capable of scaling up to the highly complex and creative tasks embodying the pinnacle of human cognition, such as science, art, and innovation. As the authors, myself, and others have argued (for example, in The Future of Crowd Work), doing so may be critical to enabling crowd workers to engage in the kinds of fulfilling, impactful work we would desire for our own children. Realizing this future will require highly interdisciplinary research into fundamental challenges ranging from incentive design to reputation systems to managing interdependent workflows. Such research will be complicated by but ultimately more impactful for grappling with the shifting landscape and ethical issues surrounding global trends towards decentralized work. Promisingly, there have been a number of recent examples of research using crowds to accomplish complex creative work including journalism, film animation, design critique, and even inventing new products. However, the best (or the worst) may be yet to come: we stand now at an inflection point where, with a concerted effort, computing research could tip us toward a positive future of crowd-powered systems.