Verification games: making verification fun

Program verification is the only way to be certain that a given piece of software is free of (certain types of) errors --- errors that could otherwise disrupt operations in the field. To date, formal verification has been done by specially-trained engineers. Labor costs have heretofore made formal verification too costly to apply beyond small, critical software components. Our goal is to make verification more cost-effective by reducing the skill set required for program verification and increasing the pool of people capable of performing program verification. Our approach is to transform the verification task (a program and a goal property) into a visual puzzle task --- a game --- that gets solved by people. The solution of the puzzle is then translated back into a proof of correctness. The puzzle is engaging and intuitive enough that ordinary people can through game-play become experts. This paper presents a status report on the Verification Games project and our Pipe Jam prototype game.

[1]  Patrick Cousot,et al.  Abstract interpretation: a unified lattice model for static analysis of programs by construction or approximation of fixpoints , 1977, POPL.

[2]  Per Martin-Löf,et al.  Intuitionistic type theory , 1984, Studies in proof theory.

[3]  Thierry Coquand,et al.  The Calculus of Constructions , 1988, Inf. Comput..

[4]  Patrick Cousot,et al.  Types as abstract interpretations , 1997, POPL '97.

[5]  Patrick Cousot,et al.  Temporal abstract interpretation , 2000, POPL '00.

[6]  Tobias Nipkow,et al.  A Proof Assistant for Higher-Order Logic , 2002 .

[7]  Michael D. Ernst.  Static and dynamic analysis: synergy and duality , 2003 .

[8]  Gilad Bracha.  Pluggable Type Systems , 2004 .

[9]  Pierre Castéran,et al.  Interactive Theorem Proving and Program Development , 2004, Texts in Theoretical Computer Science An EATCS Series.

[10]  Michael D. Ernst,et al.  A practical type system and language for reference immutability , 2004, OOPSLA '04.

[11]  Michael D. Ernst,et al.  Javari: adding reference immutability to Java , 2005, OOPSLA '05.

[12]  Patrick Cousot,et al.  The Verification Grand Challenge and Abstract Interpretation , 2005, VSTTE.

[13]  David Greenfieldboyce,et al.  Type Qualifiers for Java , 2005 .

[14]  Isil Dillig,et al.  An overview of the saturn project , 2007, PASTE '07.

[15]  Jeffrey S. Foster,et al.  Type qualifier inference for java , 2007, OOPSLA.

[16]  Michael D. Ernst,et al.  Practical pluggable types for java , 2008, ISSTA '08.

[17]  Jens Palsberg,et al.  A type system equivalent to a model checker , 2008, TOPL.

[18]  Michael D. Ernst,et al.  Inference of Reference Immutability , 2008, ECOOP.

[19]  Fausto Spoto.  Nullness Analysis in Boolean Form , 2008, 2008 Sixth IEEE International Conference on Software Engineering and Formal Methods.

[20]  Jamie Quinonez.  Javarifier : inference of reference immutability in Java , 2008 .

[21]  Xavier Leroy,et al.  Formal verification of a realistic compiler , 2009, CACM.

[22]  Fausto Spoto.  Precise null-pointer analysis , 2009, Software & Systems Modeling.

[23]  Werner Dietl,et al.  Universe Types - Topology, Encapsulation, Genericity, and Tools , 2009 .

[24]  Thomas Ball,et al.  SLAM2: Static driver verification with under 4% false alarms , 2010, Formal Methods in Computer Aided Design.

[25]  Fausto Spoto.  The Nullness Analyser of julia , 2010, LPAR.

[26]  Kai Engelhardt,et al.  seL4: formal verification of an operating-system kernel , 2010, Communications of the ACM.

[27]  Adrien Treuille,et al.  Predicting protein structures with a multiplayer online game , 2010, Nature.

[28]  David Salesin,et al.  The challenge of designing scientific discovery games , 2010, FDG.

[29]  Yves Bertot,et al.  Interactive Theorem Proving and Program Development: Coq'Art The Calculus of Inductive Constructions , 2010 .

[30]  Michael D. Ernst,et al.  Building and using pluggable type-checkers , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[31]  Michael D. Ernst,et al.  Tunable Static Inference for Generic Universe Types , 2011, ECOOP.

[32]  Hao Lü,et al.  Analysis of social gameplay macros in the Foldit cookbook , 2011, FDG.

[33]  Michael D. Ernst,et al.  Inference of field initialization , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[34]  S. Rajamani,et al.  A decade of software model checking with SLAM , 2011, Commun. ACM.

[35]  Todd Millstein,et al.  Automatic predicate abstraction of C programs , 2001, PLDI '01.

[36]  M. Sørensen,et al.  Lectures on the Curry-Howard Isomorphism , 2013 .

[37]  Seth Cooper,et al.  A Framework for Scientific Discovery through Video Games (dissertation, updated version) , 2014, ACM Books.