An analysis of errors in interactive proof attempts

The practical utility of interactive, user-guided, theorem proving depends on the design of good interaction environments, the study of which should be grounded in methods of research into human‐computer interaction (HCI). This paper discusses the relevance of classifications of programming errors developed by the HCI community to the problem of interactive theorem proving. A new taxonomy of errors is proposed for interaction with theorem provers and its adequacy as a usability metric is assessed experimentally. q 2000 Elsevier Science B.V. All rights reserved.

[1]  Richard Davis,et al.  Task Analysis and User Errors: A Methodology for Assessing Interactions , 1983, Int. J. Man Mach. Stud..

[2]  Laurent Théry,et al.  A proof development system for the HOL theorem prover , 1993 .

[3]  Henry Lieberman,et al.  Bridging the gulf between code and behavior in programming , 1995, CHI '95.

[4]  Jochen Prümper,et al.  Errors in working with office computers: A first validation of a taxonomy for observed errors in a field setting , 1992, Int. J. Hum. Comput. Interact..

[5]  John Domingue,et al.  Software visualization : programming as a multimedia experience , 1998 .

[6]  Paul A. Booth,et al.  Identifying and interpreting design errors , 1990, Int. J. Hum. Comput. Interact..

[7]  Yves Bertot,et al.  Implementing Proof by Pointing without a Structure Editor , 1997 .

[8]  Helen Lowe,et al.  XBarnacle: Making Theorem Provers More Accessible , 1997, CADE.

[9]  Michael J. C. Gordon,et al.  Edinburgh LCF: A mechanised logic of computation , 1979 .

[10]  Edmund M. Clarke,et al.  Formal Methods: State of the Art and Future Directions Working Group Members , 1996 .

[11]  Mandayam K. Srivas,et al.  Formal verification of the AAMP5 microprocessor: a case study in the industrial use of formal methods , 1995, Proceedings of 1995 IEEE Workshop on Industrial-Strength Formal Specification Techniques.

[12]  Muffy Calder,et al.  Interactive Theorem Proving: An Empirical Study of User Activity , 1998, J. Symb. Comput..

[13]  Don Syme,et al.  A New Interface for HOL - Ideas, Issues and Implementation , 1995, TPHOLs.

[14]  M. A. Ozols.  Xisabelle: a Graphical User Interface to the Isabelle Theorem Prover Xisabelle: a Graphical User Interface to the Isabelle Theorem Prover , 1999 .

[15]  Elsa L. Gunter,et al.  Adding External Decision Procedures to HOL90 Securely , 1998, TPHOLs.

[16]  D. Norman.  Categorization of action slips. , 1981 .

[17]  Stephen J. Garland,et al.  An Overview of LP, The Larch Power , 1989, RTA.

[18]  Thomas P. Moran,et al.  The Command Language Grammar: A Representation for the User Interface of Interactive Computer Systems , 1981, Int. J. Man Mach. Stud..

[19]  Natarajan Shankar,et al.  PVS: A Prototype Verification System , 1992, CADE.

[20]  Richard Davis.  User Error or Computer Error? Observations on a Statistics Package , 1983, Int. J. Man Mach. Stud..

[21]  Matthew W. Lewis,et al.  ERRORS IN AN INTERACTIVE PROGRAMMING ENVIRONMENT: CAUSES AND CURES , 1992 .

[22]  Jakob Nielsen.  A Virtual Protocol Model for Computer-Human Interaction , 1984 .

[23]  Edward A. Youngs.  Human Errors in Programming , 1974, Int. J. Man Mach. Stud..

[24]  Richard Bornat.  Jape's quiet interface , 1996 .

[25]  Lawrence Charles Paulson,et al.  Isabelle: A Generic Theorem Prover , 1994 .

[26]  D. Norman.  The psychology of everyday things", Basic Books Inc , 1988 .

[27]  Thomas R. G. Green,et al.  Reducing Programming Errors in Nested Conditionals by Prescribing a Writing Procedure , 1977, Int. J. Man Mach. Stud..

[28]  Marc Eisenstadt,et al.  Adding Data and Procedure Abstraction to the Transparent Prolog Machine TPM , 1988, ICLP/SLP.

[29]  James D. Hollan,et al.  Direct Manipulation Interfaces , 1985, Hum. Comput. Interact..

[30]  Marc Eisenstadt,et al.  The Transparent Prolog Machine (TPM): An Execution Model and Graphical Debugger for Logic Programming , 1988, J. Log. Program..

[31]  Tom Melham.  ITP Project Anthology , 1997 .

[32]  Yves Bertot,et al.  The Ctcoq Experience , 1996 .

[33]  Tom Schubert,et al.  A Tree-based, Graphical Interface for Large Proof Development , 1994 .

[34]  Yves Bertot,et al.  A Generic Approach to Building User Interfaces for Theorem Provers , 1998, J. Symb. Comput..

[35]  Kjeld Schmidt,et al.  Taxonomy for Cognitive Work Analysis , 1990 .

[36]  M. Gordon,et al.  Introduction to HOL: a theorem proving environment for higher order logic , 1993 .

[37]  Jakob Nielsen,et al.  A Virtual Protocol Model for Computer-Human Interaction , 1984, Int. J. Man Mach. Stud..

[38]  Albert John Camilleri,et al.  A Hybrid Approach to Verifying Liveness in a Symmetric Multi-Processor , 1997, TPHOLs.