Playing the System: Can Puzzle Players Teach us How to Solve Hard Problems?

With nearly three billion players, video games are more popular than ever. Casual puzzle games are among the most played categories. These games capitalize on the players’ analytical and problem-solving skills. Can we leverage these abilities to teach ourselves how to solve complex combinatorial problems? In this study, we harness the collective wisdom of millions of players to tackle the classical NP-hard problem of multiple sequence alignment, relevant to many areas of biology and medicine. We show that Borderlands Science players propose solutions to multiple sequence alignment tasks that perform as well or better than standard approaches, while exploring a much larger area of the Pareto-optimal solution space. We also show the strategies of the players, although highly heterogeneous, follow a collective logic that can be mimicked with Behavioral Cloning with minimal performance loss, allowing the players’ collective wisdom to be leveraged for alignment of any sequences.

[1]  M. Dianati,et al.  A Survey on Imitation Learning Techniques for End-to-End Autonomous Vehicles , 2022, IEEE Transactions on Intelligent Transportation Systems.

[2]  J. Waldispühl,et al.  Human-supervised clustering of multidimensional data using crowdsourcing , 2022, Royal Society Open Science.

[3]  Shravan Veerapaneni,et al.  Behavioral Cloning in Atari Games Using a Combined Variational Autoencoder and Predictor Model , 2021, 2021 IEEE Congress on Evolutionary Computation (CEC).

[4]  Scott R. Klemmer,et al.  Galileo: Citizen-led Experimentation Using a Social Computing System , 2021, CHI.

[5]  Tom Sercu,et al.  Transformer protein language models are unsupervised structure learners , 2020, bioRxiv.

[6]  Yong Liu,et al.  DLPAlign: A Deep Learning based Progressive Alignment Method for Multiple Protein Sequences , 2020, CSBio '20: Proceedings of the Eleventh International Conference on Computational Systems-Biology and Bioinformatics.

[7]  Rob Knight,et al.  Leveling up citizen science , 2020, Nature Biotechnology.

[8]  Antonios Liapis,et al.  Dungeons & Replicants: Automated Game Balancing via Deep Player Behavior Modeling , 2020, 2020 IEEE Conference on Games (CoG).

[9]  Ville Hautamäki,et al.  Benchmarking End-to-End Behavioural Cloning on Video Games , 2020, 2020 IEEE Conference on Games (CoG).

[10]  Wojciech M. Czarnecki,et al.  Grandmaster level in StarCraft II using multi-agent reinforcement learning , 2019, Nature.

[11]  Rainer Malaka,et al.  Can You Rely on Human Computation?: A Large-scale Analysis of Disruptive Behavior in Games With A Purpose , 2019, CHI PLAY.

[12]  Che Wang,et al.  BAIL: Best-Action Imitation Learning for Batch Deep Reinforcement Learning , 2019, NeurIPS.

[13]  Tim Salimans,et al.  Axial Attention in Multidimensional Transformers , 2019, ArXiv.

[14]  Ruslan Salakhutdinov,et al.  MineRL: A Large-Scale Dataset of Minecraft Demonstrations , 2019, IJCAI.

[15]  Reza Jafari,et al.  Using deep reinforcement learning approach for solving the multiple sequence alignment problem , 2019, SN Applied Sciences.

[16]  Eder Santana,et al.  Exploring the Limitations of Behavior Cloning for Autonomous Driving , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[17]  Luxin Zhang,et al.  Atari-HEAD: Atari Human Eye-Tracking and Demonstration Dataset , 2019, ArXiv.

[18]  Wael Farag,et al.  Cloning Safe Driving Behavior for Self-Driving Cars using Convolutional Neural Networks , 2019, Recent Patents on Computer Science.

[19]  Junjie Yan,et al.  Grid R-CNN , 2018, 1811.12030.

[20]  Alex Sherstinsky,et al.  Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Network , 2018, Physica D: Nonlinear Phenomena.

[21]  J. Grafman,et al.  The effects of expected reward on creative problem solving , 2018, Cognitive, Affective, & Behavioral Neuroscience.

[22]  Rob Knight,et al.  American Gut: an Open Platform for Citizen Science Microbiome Research , 2018, mSystems.

[23]  Jiwen Lu,et al.  Attention-Aware Deep Reinforcement Learning for Video Face Recognition , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[24]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[25]  Scott R. Klemmer,et al.  Gut Instinct: Creating Scientific Theories with Online Learners , 2017, CHI.

[26]  Tom Schaul,et al.  Deep Q-learning From Demonstrations , 2017, AAAI.

[27]  Mohamed Medhat Gaber,et al.  Imitation Learning , 2017, ACM Comput. Surv..

[28]  Olga Shabalina,et al.  Some Video Games Can Increase the Player's Creativity , 2017, Int. J. Game Based Learn..

[29]  Cédric Notredame,et al.  Multiple sequence alignment modeling: methods and applications , 2016, Briefings Bioinform..

[30]  Martial Hebert,et al.  Learning Transferable Policies for Monocular Reactive MAV Control , 2016, ISER.

[31]  Tian Xia,et al.  Vehicle Detection from 3D Lidar Using Fully Convolutional Network , 2016, Robotics: Science and Systems.

[32]  Morten Nielsen,et al.  Gapped sequence alignment using artificial neural networks: application to the MHC class I system , 2016, Bioinform..

[33]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Seungmoon Song,et al.  A neural circuitry that emphasizes spinal feedback generates diverse behaviours of human locomotion , 2015, The Journal of physiology.

[35]  Roman V. Yampolskiy,et al.  Solving NP-hard number matrix games with Wisdom of Artificial Crowds , 2015, 2015 Computer Games: AI, Animation, Mobile, Multimedia, Educational and Serious Games (CGAMES).

[36]  Li Zhang,et al.  Learning similarity with cosine similarity ensemble , 2015, Inf. Sci..

[37]  Tandy J. Warnow,et al.  PASTA: Ultra-Large Multiple Sequence Alignment for Nucleotide and Amino-Acid Sequences , 2015, J. Comput. Biol..

[38]  Minjae Lee,et al.  RNA design rules from a massive open laboratory , 2014, Proceedings of the National Academy of Sciences.

[39]  Rafik Amir Salama,et al.  A non-independent energy-based multiple sequence alignment improves prediction of transcription factor binding sites , 2013, Bioinform..

[40]  Liping Han,et al.  Distance Weighted Cosine Similarity Measure for Text Classification , 2013, IDEAL.

[41]  Michael D. Lee,et al.  The Wisdom of the Crowd in Combinatorial Problems , 2012, Cogn. Sci..

[42]  M. Blanchette,et al.  Phylo: A Citizen Science Approach for Improving Multiple Sequence Alignment , 2012, PloS one.

[43]  Z. Popovic,et al.  Increased Diels-Alderase activity through backbone remodeling guided by Foldit players , 2012, Nature Biotechnology.

[44]  Roman V. Yampolskiy,et al.  Wisdom of artificial crowds algorithm for solving NP-hard problems , 2011, Int. J. Bio Inspired Comput..

[45]  Li Bai,et al.  Cosine Similarity Metric Learning for Face Verification , 2010, ACCV.

[46]  Geoffrey J. Gordon,et al.  A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning , 2010, AISTATS.

[47]  Adrien Treuille,et al.  Predicting protein structures with a multiplayer online game , 2010, Nature.

[48]  Víctor Parada,et al.  People Efficiently Explore the Solution Space of the Computationally Intractable Traveling Salesman Problem to Find Near-Optimal Tours , 2010, PloS one.

[49]  Luis von Ahn.  Games with a Purpose , 2006, Computer.

[50]  Sonja J. Prohaska,et al.  Multiple sequence alignment with user-defined constraints at GOBICS , 2005, Bioinform..

[51]  Chuong B. Do,et al.  ProbCons: Probabilistic consistency-based multiple sequence alignment. , 2005, Genome research.

[52]  Flemming Topsøe,et al.  Jensen-Shannon divergence and Hilbert space embedding , 2004, International Symposium onInformation Theory, 2004. ISIT 2004. Proceedings..

[53]  Roland L Dunbrack,et al.  Scoring profile‐to‐profile sequence alignments , 2004, Protein science : a publication of the Protein Society.

[54]  Laura A. Dabbish,et al.  Labeling images with a computer game , 2004, AAAI Spring Symposium: Knowledge Collection from Volunteer Contributors.

[55]  D. Haussler,et al.  Aligning multiple genomic sequences with the threaded blockset aligner. , 2004, Genome research.

[56]  I. Grosse,et al.  Analysis of symbolic sequences using the Jensen-Shannon divergence. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[57]  Lukas Wagner,et al.  A Greedy Algorithm for Aligning DNA Sequences , 2000, J. Comput. Biol..

[58]  D. Brooks,et al.  Evidence for striatal dopamine release during a video game , 1998, Nature.

[59]  T. Ormerod,et al.  Human performance on the traveling salesman problem , 1996, Perception & psychophysics.

[60]  J. Thompson,et al.  CLUSTAL W: improving the sensitivity of progressive multiple sequence alignment through sequence weighting, position-specific gap penalties and weight matrix choice. , 1994, Nucleic acids research.

[61]  M S Waterman,et al.  Sequence alignment and penalty choice. Review of concepts, case studies and implications. , 1994, Journal of molecular biology.

[62]  Lawrence D. Jackel,et al.  Backpropagation Applied to Handwritten Zip Code Recognition , 1989, Neural Computation.

[63]  D. Sankoff,et al.  Evolution of 5S RNA and the non-randomness of base replacement. , 1973, Nature: New biology.

[64]  S. B. Needleman,et al.  A general method applicable to the search for similarities in the amino acid sequence of two proteins. , 1970, Journal of molecular biology.

[65]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[66]  Attila Szantner,et al.  Massively Multiplayer Online Science , 2016 .

[67]  Michael Vitale,et al.  The wisdom of crowds , 2016, The Lancet.

[68]  Ismael Rodríguez,et al.  Comparing Problem Solving Strategies for NP-hard Optimization Problems , 2013, Fundam. Informaticae.

[69]  Desney S. Tan,et al.  Proceedings of the 2008 Conference on Human Factors in Computing Systems, CHI 2008, 2008, Florence, Italy, April 5-10, 2008 , 2005, CHI 2008.

[70]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[71]  Claude Sammut,et al.  A Framework for Behavioural Cloning , 1995, Machine Intelligence 15.

[72]  Tao Jiang,et al.  On the Complexity of Multiple Sequence Alignment , 1994, J. Comput. Biol..

[73]  Dean Pomerleau,et al.  ALVINN, an autonomous land vehicle in a neural network , 2015 .