A Human Computation Framework for Boosting Combinatorial Solvers

We propose a general framework for boosting combinatorial solvers through human computation. Our framework combines insights from human workers with the power of combinatorial optimization. The combinatorial solver is also used to guide requests for the workers, and thereby obtain the most useful human feedback quickly. Our approach also incorporates a problem decomposition approach with a general strategy for discarding incorrect human input. We apply this framework in the domain of materials discovery, and demonstrate a speedup of over an order of magnitude.

[1]  Joe Marks,et al.  Human-Guided Simple Search , 2000, AAAI/IAAI.

[2]  Ashley A. White.  The Materials Genome Initiative: One year on , 2012 .

[3]  Jinfeng Yi,et al.  Supplementary Documents for “ Semi-Crowdsource Clustering : Generalizing Crowd Labeling by Robust Distance Metric Learning ” , 2012 .

[4]  Laura A. Dabbish,et al.  Labeling images with a computer game , 2004, AAAI Spring Symposium: Knowledge Collection from Volunteer Contributors.

[5]  Ronan Le Bras,et al.  Crowdsourcing Backdoor Identification for Combinatorial Optimization , 2013, IJCAI.

[6]  Ronan Le Bras,et al.  A computational challenge problem in materials discovery: synthetic problem generator and real-world datasets , 2014, AAAI 2014.

[7]  H. Lieberman.  Common Consensus : a web-based game for collecting commonsense goals , 2007 .

[8]  Prachi Patel.  Materials Genome Initiative and energy , 2011 .

[9]  Carlos Ansótegui,et al.  On the hardness of solving edge matching puzzles as SAT or CSP problems , 2012, Constraints.

[10]  Toby Walsh,et al.  Handbook of Constraint Programming , 2006, Handbook of Constraint Programming.

[11]  Jinfeng Yi,et al.  Semi-Crowdsourced Clustering: Generalizing Crowd Labeling by Robust Distance Metric Learning , 2012, NIPS.

[12]  Matteo Fischetti,et al.  Backdoor Branching , 2011, INFORMS J. Comput..

[13]  John C. Platt,et al.  Learning from the Wisdom of Crowds by Minimax Entropy , 2012, NIPS.

[14]  Yuri Malitsky,et al.  Backdoors to Combinatorial Optimization: Feasibility and Optimality , 2009, CPAIOR.

[15]  Pietro Perona,et al.  Crowdclustering , 2011, NIPS.

[16]  Hideomi Koinuma,et al.  Combinatorial Synthesis and Evaluation of Functional Inorganic Materials Using Thin-Film Techniques , 2002 .

[17]  Stefano Ermon,et al.  SMT-Aided Combinatorial Materials Discovery , 2012, SAT.

[18]  Alexander Kazimirov,et al.  High energy x-ray diffraction/x-ray fluorescence spectroscopy for high-throughput analysis of composition spread thin films. , 2009, The Review of scientific instruments.

[19]  Matthew Chalmers,et al.  EyeSpy: supporting navigation through play , 2009, CHI.

[20]  Changshui Zhang,et al.  What if the irresponsible teachers are dominating? a method of training on samples and clustering on teachers , 2010, AAAI 2010.

[21]  Erik D. Demaine,et al.  Jigsaw Puzzles, Edge Matching, and Polyomino Packing: Connections and Complexity , 2007, Graphs Comb..

[22]  Manuel Blum,et al.  Peekaboom: a game for locating objects in images , 2006, CHI.

[23]  Toby Walsh,et al.  Backbones and Backdoors in Satisfiability , 2005, AAAI.

[24]  Stefan Szeider,et al.  Backdoor Sets for DLL Subsolvers , 2005, Journal of Automated Reasoning.

[25]  Joe Marks,et al.  Human-guided tabu search , 2002, AAAI/IAAI.

[26]  Manuel Blum,et al.  Verbosity: a game for collecting common-sense facts , 2006, CHI.

[27]  Víctor Parada,et al.  People Efficiently Explore the Solution Space of the Computationally Intractable Traveling Salesman Problem to Find Near-Optimal Tours , 2010, PloS one.

[28]  Pietro Perona,et al.  The Multidimensional Wisdom of Crowds , 2010, NIPS.

[29]  Adrien Treuille,et al.  Predicting protein structures with a multiplayer online game , 2010, Nature.

[30]  Gert R. G. Lanckriet,et al.  A Game-Based Approach for Collecting Semantic Annotations of Music , 2007, ISMIR.

[31]  Edith Law,et al.  Input-agreement: a new mechanism for collecting data using human computation games , 2009, CHI.

[32]  Stefan Szeider,et al.  Backdoors to Satisfaction , 2011, The Multivariate Algorithmic Revolution and Beyond.