Understanding and maintaining tactics graphically OR how we are learning that a diagram can be worth more than 10K LoC

The use of a functional language to implement proof strategies as proof tactics in interactive theorem provers, often provides short, concise and elegant implementations. Whilst being elegant, the use of higher order features and combinator languages often results in a very procedural view of a strategy, which may deviate significantly from the high-level ideas behind it. This can make a tactic hard to understand and hence difficult to to debug and maintain for experts and non-experts alike: one often has to tear apart complex combinations of lower level tactics manually in order to analyse a failure in the overall strategy. In an industrial technology transfer project, we have been working on porting a very large and complex proof tactic into PSGraph, a graphical language for representing proof strategies. The goal of this work is to improve understandability and maintainability of tactics. Motivated by some initial successes with this, we here extend PSGraph with additional features for development and debugging. Through the re-implementation and refactoring of several existing tactics, we demonstrates the advantages of PSGraph compared with a typical sentential tactic language with respect to debugging, readability and maintenance. In order to act as guidance for others, we give a fairly detailed comparison of the user experience with the two approaches. The paper is supported by a web page providing further details about the implementation as well as interactive illustrations of the examples.

[1]  Gerwin Klein,et al.  Proof Engineering Considered Essential , 2014, FM.

[2]  Eric Rosenbaum,et al.  Scratch: programming for all , 2009, Commun. ACM.

[3]  Cezary Kaliszyk,et al.  Towards Formal Proof Metrics , 2016, FASE.

[4]  Alan Bundy,et al.  Typed meta-interpretive learning for proof strategies , 2015, ILP.

[5]  Gudmund Grov,et al.  'The Tinker' for Rodin , 2016, ABZ.

[6]  Gudmund Grov,et al.  Developing and Debugging Proof Strategies by Tinkering , 2016, TACAS.

[7]  Gudmund Grov,et al.  A Super Industrial Application of PSGraph , 2016, ABZ.

[8]  David Delahaye,et al.  A Proof Dedicated Meta-Language , 2002, LFM.

[9]  W. Feurzeig,et al.  Programming-languages as a conceptual framework for teaching mathematics , 1969, SCOU.

[10]  Gérard P. Huet,et al.  The Zipper , 1997, Journal of Functional Programming.

[11]  Aleks Kissinger,et al.  Open-graphs and monoidal theories† , 2010, Mathematical Structures in Computer Science.

[12]  Alan Robinson,et al.  Computational Logic - Essays in Honor of Alan Robinson , 1991, Computational Logic - Essays in Honor of Alan Robinson.

[13]  David Weintrop,et al.  To block or not to block, that is the question: students' perceptions of blocks-based programming , 2015, IDC.

[14]  Gudmund Grov,et al.  Towards Formal Proof Script Refactoring , 2011, Calculemus/MKM.

[15]  Randy Pausch,et al.  Alice: a 3-D tool for introductory programming concepts , 2000 .

[16]  J. Michael Spivey,et al.  The Z notation - a reference manual , 1992, Prentice Hall International Series in Computer Science.

[17]  Brigitte Pientka A type-theoretic foundation for programming with higher-order abstract syntax and first-class substitutions , 2008, POPL '08.

[18]  P. Mahadevan,et al.  An overview , 2007, Journal of Biosciences.

[19]  Zhong Shao,et al.  VeriML: typed computation of logical terms inside a language with effects , 2010, ICFP '10.

[20]  Maris A. Ozols,et al.  XIsabelle: A System Description , 1997, CADE.

[21]  Standard Glossary of Software Engineering Terminology , 1990 .

[22]  Tobias Nipkow,et al.  A FORMAL PROOF OF THE KEPLER CONJECTURE , 2015, Forum of Mathematics, Pi.

[23]  Richard Bornat,et al.  Jape: A Calculator for Animating Proof-on-Paper , 1997, CADE.

[24]  Rod M. Burstall ProveEasy: helping people learn to do proofs , 2000, Electron. Notes Theor. Comput. Sci..

[25]  Alan Bundy,et al.  A Science of Reasoning , 1991, Computational Logic - Essays in Honor of Alan Robinson.

[26]  Maurice H. Halstead,et al.  Elements of software science , 1977 .

[27]  Shari Lawrence Pfleeger,et al.  Software Metrics : A Rigorous and Practical Approach , 1998 .

[28]  Jacques D. Fleuriot,et al.  IsaPlanner: A Prototype Proof Planner in Isabelle , 2003, CADE.

[29]  Markus Wenzel,et al.  An Isabelle Proof Method Language , 2014, ITP.

[30]  Alexander K. Petrenko,et al.  Electronic Notes in Theoretical Computer Science , 2009 .

[31]  Joe Hurd,et al.  OpenTheory: Package Management for Higher Order Logic Theories , 2009 .

[32]  Jim Grundy,et al.  Window Inference In The HOL System , 1991, 1991., International Workshop on the HOL Theorem Proving System and Its Applications.

[33]  John Harrison,et al.  A Mizar Mode for HOL , 1996, TPHOLs.

[34]  Hartmut Ehrig,et al.  Fundamentals of Algebraic Graph Transformation (Monographs in Theoretical Computer Science. An EATCS Series) , 1992 .

[35]  Michael Norrish,et al.  A Brief Overview of HOL4 , 2008, TPHOLs.

[36]  D. Ross Jeffery,et al.  Productivity for proof engineering , 2014, ESEM '14.

[37]  Jean-Francois Girard,et al.  An Activity-Based Quality Model for Maintainability , 2007, 2007 IEEE International Conference on Software Maintenance.

[38]  Gudmund Grov,et al.  A Graphical Language for Proof Strategies , 2013, LPAR.

[39]  Gudmund Grov,et al.  Tinker, tailor, solver, proof , 2014, UITP.

[40]  Aleks Kissinger,et al.  Quantomatic: A proof assistant for diagrammatic reasoning , 2015, CADE.

[41]  Hartmut Ehrig,et al.  Fundamentals of Algebraic Graph Transformation , 2006, Monographs in Theoretical Computer Science. An EATCS Series.

[42]  Carsten Schürmann,et al.  System Description: Delphin - A Functional Programming Language for Deductive Systems , 2008, LFMTP@LICS.

[43]  Serge Autexier,et al.  A Tactic Language for Declarative Proofs , 2010, ITP.

[44]  Lawrence C. Paulson,et al.  Logic and computation - interactive proof with Cambridge LCF , 1987, Cambridge tracts in theoretical computer science.

[45]  Andrea Asperti,et al.  A new type for tactics , 2009 .

[46]  Colin O'Halloran Automated verification of code automatically generated from Simulink® , 2012, Automated Software Engineering.

[47]  Karol Pak THE ALGORITHMS FOR IMPROVING AND REORGANIZING NATURAL DEDUCTION PROOFS , 2010 .

[48]  Jan Madey,et al.  The Z notation: A reference manual , 1990 .

[49]  Georges Gonthier A computer-checked proof of the Four Colour Theorem , 2005 .

[50]  Michael J. C. Gordon,et al.  From LCF to HOL: a short history , 2000, Proof, Language, and Interaction.

[51]  Mark Adams,et al.  Refactoring Proofs with Tactician , 2015, SEFM Workshops.

[52]  Robin Denis Arthan,et al.  Now f is continuous (exercise!) , 2016, J. Formaliz. Reason..

[53]  Iain Whiteside Refactoring proofs , 2013 .

[54]  Herbert A. Simon,et al.  Why a Diagram is (Sometimes) Worth Ten Thousand Words , 1987, Cogn. Sci..

[55]  Lawrence C. Paulson,et al.  A Higher-Order Implementation of Rewriting , 1983, Sci. Comput. Program..

[56]  Tomer Libal,et al.  Advanced Proof Viewing in ProofTool , 2014, UITP.

[57]  Viktor Vafeiadis,et al.  Mtac: a monad for typed tactic programming in Coq , 2013, ICFP.

[58]  Gernot Heiser,et al.  Comprehensive formal verification of an OS microkernel , 2014, TOCS.

[59]  John Harrison,et al.  HOL Light: An Overview , 2009, TPHOLs.

[60]  Freek Wiedijk,et al.  MMode, a Mizar Mode for the proof assistant Coq , 2003 .

[61]  Lars Hupel Interactive Simplifier Tracing and Debugging in Isabelle , 2014, CICM.

[62]  Ewen Denney,et al.  A Tactic Language for Hiproofs , 2008, AISC/MKM/Calculemus.

[63]  Per Runeson,et al.  Guidelines for conducting and reporting case study research in software engineering , 2009, Empirical Software Engineering.