Solving Pictorial Jigsaw Puzzle by Stigmergy-inspired Internet-based Human Collective Intelligence

The pictorial jigsaw (PJ) puzzle is a well-known leisure game for humans. Usually, a PJ puzzle game is played by one or several human players face-to-face in the physical space. In this paper, we focus on how to solve PJ puzzles in the cyberspace by a group of physically distributed human players. We propose an approach to solving PJ puzzle by stigmergy-inspired Internet-based human collective intelligence. The core of the approach is a continuously executing loop, named the EIF loop, which consists of three activities: exploration, integration, and feedback. In exploration, each player tries to solve the PJ puzzle alone, without direct interactions with other players. At any time, the result of a player's exploration is a partial solution to the PJ puzzle, and a set of rejected neighboring relation between pieces. The results of all players' exploration are integrated in real time through integration, with the output of a continuously updated collective opinion graph (COG). And through feedback, each player is provided with personalized feedback information based on the current COG and the player's exploration result, in order to accelerate his/her puzzle-solving process. Exploratory experiments show that: (1) supported by this approach, the time to solve PJ puzzle is nearly linear to the reciprocal of the number of players, and shows better scalability to puzzle size than that of face-to-face collaboration for 10-player groups; (2) for groups with 2 to 10 players, the puzzle-solving time decreases 31.36%-64.57% on average, compared with the best single players in the experiments.

[1]  Tom Ziemke,et al.  Social cognition, artefacts, and stigmergy: A comparative analysis of theoretical frameworks for the understanding of artefact-mediated collaborative activity , 2001, Cognitive Systems Research.

[2]  Nathan S. Netanyahu,et al.  A Genetic Algorithm-Based Solver for Very Large Jigsaw Puzzles , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Aniket Kittur,et al.  Crowdsourcing user studies with Mechanical Turk , 2008, CHI.

[4]  Adrien Treuille,et al.  Predicting protein structures with a multiplayer online game , 2010, Nature.

[5]  Andrew C. Gallagher Jigsaw puzzles with pieces of unknown orientation , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Marco Dorigo,et al.  Swarm intelligence: from natural to artificial systems , 1999 .

[7]  Peter J. Fleming,et al.  The MATLAB genetic algorithm toolbox , 1995 .

[8]  Yunhao Liu,et al.  Incentives for Mobile Crowd Sensing: A Survey , 2016, IEEE Communications Surveys & Tutorials.

[9]  Timothy C. Bates,et al.  Smart groups of smart people: Evidence for IQ as the origin of collective intelligence in the performance of human groups , 2017 .

[10]  Nikos Papamarkos,et al.  A New Technique for Solving a Jigsaw Puzzle , 2006, 2006 International Conference on Image Processing.

[11]  David B. Cooper,et al.  Solving Square Jigsaw Puzzles with Loop Constraints , 2014, ECCV.

[12]  Ohad Ben-Shahar,et al.  A fully automated greedy square jigsaw puzzle solver , 2011, CVPR 2011.

[13]  R. Brimacombe,et al.  The structure of ribosomal RNA: a three-dimensional jigsaw puzzle. , 1995, European journal of biochemistry.

[14]  Louis Rosenberg,et al.  Artificial Swarm Intelligence, a Human-in-the-Loop Approach to A.I , 2016, AAAI.

[15]  Ngoc Thanh Nguyen,et al.  Integration Computing and Collective Intelligence , 2013, IDC.

[16]  Mario García Valdez,et al.  Using a Graph Based Database to Support Collaborative Interactive Evolutionary Systems , 2014, Recent Advances on Hybrid Approaches for Designing Intelligent Systems.

[17]  M. Thompson,et al.  Clumsy Solutions for a Complex World: Governance, Politics and Plural Perceptions , 2006 .

[18]  Li Yi,et al.  Feature-oriented stigmergy-based collaborative requirements modeling: an exploratory approach for requirements elicitation and evolution based on web-enabled collective intelligence , 2013, Science China Information Sciences.

[19]  Paolo Favaro,et al.  Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles , 2016, ECCV.

[20]  R. Sternberg,et al.  Complex Problem Solving : Principles and Mechanisms , 1992 .

[21]  H. Van Dyke Parunak,et al.  A Survey of Environments and Mechanisms for Human-Human Stigmergy , 2005, E4MAS.

[22]  Liming Wang,et al.  A Novel Image Based CAPTCHA Using Jigsaw Puzzle , 2010, 2010 13th IEEE International Conference on Computational Science and Engineering.

[23]  H. Wolfson,et al.  Solving jigsaw puzzles by computer , 1988 .

[24]  Erik D. Demaine,et al.  Jigsaw Puzzles, Edge Matching, and Polyomino Packing: Connections and Complexity , 2007, Graphs Comb..

[25]  Marshall W. Bern,et al.  A global approach to automatic solution of jigsaw puzzles , 2002, SCG '02.

[26]  Nathan S. Netanyahu,et al.  An automatic solver for very large jigsaw puzzles using genetic algorithms , 2016, Genetic Programming and Evolvable Machines.

[27]  Longin Jan Latecki,et al.  Particle filter with state permutations for solving image jigsaw puzzles , 2011, CVPR 2011.

[28]  William T. Freeman,et al.  A probabilistic image jigsaw puzzle solver , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[29]  Robert L. Berger The undecidability of the domino problem , 1966 .

[30]  Steven L. Goldman Reinventing Discovery: The New Era of Networked Science , 2014 .

[31]  Dewen Hu,et al.  Globally Consistent Reconstruction of Ripped-Up Documents , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Mu-Chun Su,et al.  A puzzle solver and its application in speech descrambling , 2007 .

[33]  Francis Heylighen,et al.  Collective Intelligence and its Implementation on the Web: Algorithms to Develop a Collective Mental Map , 1999, Comput. Math. Organ. Theory.

[34]  Michael S. Bernstein,et al.  The future of crowd work , 2013, CSCW.

[35]  Aniket Kittur,et al.  Alloy: Clustering with Crowds and Computation , 2016, CHI.

[36]  Denis Burkitt,et al.  Large-Bowel Cancer: An Epidemiologic Jigsaw Puzzle , 1975 .

[37]  Fenghui Yao,et al.  A shape and image merging technique to solve jigsaw puzzles , 2003, Pattern Recognit. Lett..

[38]  M. Kearns,et al.  An Experimental Study of the Coloring Problem on Human Subject Networks , 2006, Science.

[39]  W. Marande,et al.  Mitochondrial DNA as a Genomic Jigsaw Puzzle , 2007, Science.

[40]  R. Levins Decentralized Control of Construction Behavior in Paper Wasps: An Overview of the Stigmergy Approach , 1999 .

[41]  Minjae Lee,et al.  RNA design rules from a massive open laboratory , 2014, Proceedings of the National Academy of Sciences.

[42]  H. Freeman,et al.  Apictorial Jigsaw Puzzles: The Computer Solution of a Problem in Pattern Recognition , 1964, IEEE Trans. Electron. Comput..

[43]  Xingsi Xue,et al.  Collaborative ontology matching based on compact interactive evolutionary algorithm , 2017, Knowl. Based Syst..

[44]  D. Meyer,et al.  Supporting Online Material Materials and Methods Som Text Figs. S1 to S6 References Evidence for a Collective Intelligence Factor in the Performance of Human Groups , 2022 .

[45]  Masataka Tokumaru,et al.  Interactive Evolutionary Computation Using Multiple Users' Gaze Information , 2017, HCI.

[46]  Klaus Hansen,et al.  Solving jigsaw puzzles using image features , 2008, Pattern Recognit. Lett..

[47]  Louis B. Rosenberg Human Swarms, a real-time method for collective intelligence , 2015, ECAL.

[48]  Hideyuki Takagi,et al.  Interactive evolutionary computation: fusion of the capabilities of EC optimization and human evaluation , 2001, Proc. IEEE.

[49]  Jun Wang,et al.  Gamers, citizen scientists, and data: Exploring participant contributions in two games with a purpose , 2017, Comput. Hum. Behav..

[50]  Pierre Lévy,et al.  Collective Intelligence: Mankind's Emerging World in Cyberspace , 1997 .

[51]  Yuan Zhang,et al.  A stigmergy-based aggregation method for swarm robotic system , 2017, 2017 IEEE Symposium Series on Computational Intelligence (SSCI).

[52]  Juan Julián Merelo Guervós,et al.  Gamification techniques in collaborative interactive evolutionary computation , 2017, GECCO.

[53]  Daren C. Brabham Crowdsourcing as a Model for Problem Solving , 2008 .

[54]  Juho Hamari,et al.  Gamification in Crowdsourcing: A Review , 2016, 2016 49th Hawaii International Conference on System Sciences (HICSS).

[55]  Benjamin B. Kimia,et al.  On solving 2D and 3D puzzles using curve matching , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[56]  D. Dörner,et al.  Complex Problem Solving: What It Is and What It Is Not , 2017, Front. Psychol..

[57]  Tim Weyrich,et al.  A system for high-volume acquisition and matching of fresco fragments: reassembling Theran wall paintings , 2008, SIGGRAPH 2008.

[58]  Trond H. Torsvik,et al.  The Rodinia Jigsaw Puzzle , 2003, Science.

[59]  Tim Kraska,et al.  CrowdER: Crowdsourcing Entity Resolution , 2012, Proc. VLDB Endow..

[60]  Aniket Kittur,et al.  The Knowledge Accelerator: Big Picture Thinking in Small Pieces , 2016, CHI.

[61]  Fernando González-Ladrón-de-Guevara,et al.  Towards an integrated crowdsourcing definition , 2012, J. Inf. Sci..

[62]  Norman I. Badler,et al.  Jigsaw puzzle matching using a boundary-centered polar encoding , 1982, Comput. Graph. Image Process..

[63]  Carmen Gervet,et al.  On Refining Ill-Defined Constraint Problems: A Case Study in Iterative Prototyping , 1999 .

[64]  Michael S. Bernstein,et al.  Flash Organizations: Crowdsourcing Complex Work by Structuring Crowds As Organizations , 2017, CHI.

[65]  Marco Dorigo,et al.  Ant algorithms and stigmergy , 2000, Future Gener. Comput. Syst..

[66]  Daniel J. Simon,et al.  Evolutionary optimization algorithms : biologically-Inspired and population-based approaches to computer intelligence , 2013 .

[67]  Tarak Gandhi,et al.  An automatic jigsaw puzzle solver , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[68]  Nicholas A. Christakis,et al.  Locally noisy autonomous agents improve global human coordination in network experiments , 2017, Nature.

[69]  P.-P. Grasse La reconstruction du nid et les coordinations interindividuelles chezBellicositermes natalensis etCubitermes sp. la théorie de la stigmergie: Essai d'interprétation du comportement des termites constructeurs , 1959, Insectes Sociaux.

[70]  Laura A. Dabbish,et al.  Deep Structures of Collaboration: Physiological Correlates of Collective Intelligence and Group Satisfaction , 2017, CSCW.

[71]  Guy Theraulaz,et al.  A Brief History of Stigmergy , 1999, Artificial Life.

[72]  Nayat Sánchez Pi,et al.  Collective preferences in evolutionary multi-objective optimization: techniques and potential contributions of collective intelligence , 2015, SAC.