The TPTP Problem Library

This paper provides a detailed description of the CNF part of the TPTP Problem Library for automated theorem-proving systems. The library is available via the Internet and forms a common basis for development and experimentation with automated theorem provers. This paper explains the motivations and reasoning behind the development of the TPTP (thus implicitly explaining the design decisions made) and describes the TPTP contents and organization. It also provides guidelines for obtaining and using the library, summary statistics about release v1.2.1, and an overview of the tptp2X utility program. References for all the sources of TPTP problems are provided.

[1]  W LovelandDonald.  Mechanical Theorem-Proving by Model Elimination , 1968 .

[2]  Leon Sterling,et al.  The Art of Prolog , 1987, IEEE Expert.

[3]  Alonzo Church,et al.  Review: Jan Lukasiewicz, The Shortest Axiom of the Implicational Calculus of Propositions , 1948 .

[4]  Ross A. Overbeek,et al.  Complexity and related enhancements for automated theorem-proving programs , 1976 .

[5]  VincentelliDepartment,et al.  Formal Design Veri cation of Digital SystemsThomas , 1993 .

[6]  S. D. Wilson,et al.  Incorporating Relevancy Testing in SATCHMO , 1989 .

[7]  Laurent Siklóssy,et al.  Breadth-First Search: Some Surprising Results , 1973, Artif. Intell..

[8]  Mark E. Stickel.  Schubert's Steamroller problem: Formulations and solutions , 2004, Journal of Automated Reasoning.

[9]  Chin-Liang Chang.  The Unit Proof and the Input Proof in Theorem Proving , 1970, JACM.

[10]  Donald A. Martin,et al.  Set Theory and Its Logic. , 1966 .

[11]  Francis Jeffry Pelletier,et al.  Seventy-five problems for testing automatic theorem provers , 1986, Journal of Automated Reasoning.

[12]  Richard C. T. Lee,et al.  Symbolic logic and mechanical theorem proving , 1973, Computer science classics.

[13]  Larry Wos,et al.  Automated Reasoning: Introduction and Applications , 1984 .

[14]  A. G. Howson.  A handbook of terms used in algebra and analysis: Sets and functions , 1972 .

[15]  A. G. Howson.  A handbook of terms used in algebra and analysis: Measure and Lebesgue integration , 1972 .

[16]  M. R. Genesereth,et al.  Knowledge Interchange Format Version 3.0 Reference Manual , 1992, LICS 1992.

[17]  Gordon I. McCalla,et al.  The knowledge frontier: essays in the representation of knowledge , 1987 .

[18]  Gerald J. Sussman,et al.  A Computational Model of Skill Acquisition , 1973 .

[19]  William McCune,et al.  Single axioms for groups and Abelian groups with various operations , 1993, Journal of Automated Reasoning.

[20]  Larry Wos,et al.  Set theory in first-order logic: Clauses for Gödel's axioms , 1986, Journal of Automated Reasoning.

[21]  Hugues Leblanc,et al.  Alternatives to Standard First-Order Semantics , 1989 .

[22]  John L. Pollock,et al.  Interest driven suppositional reasoning , 1990, Journal of Automated Reasoning.

[23]  Melvil Dewey.  Decimal classification and relative index , 1942 .

[24]  William McCune,et al.  Experiments with ROO: A Parallel Automated Deduction System , 1990, Dagstuhl Seminar on Parallelization in Inference Systems.

[25]  Allen Newell,et al.  Human Problem Solving. , 1973 .

[26]  Christoph Weidenbach,et al.  Common Syntax of the DFG-Schwerpunktprogramm ''Deduktion'' , 1996 .

[27]  K. Hensel.  Journal für die reine und angewandte Mathematik , 1892 .

[28]  Andrei Voronkov,et al.  The Inverse Method , 2001, Handbook of Automated Reasoning.

[29]  William C. Frederick,et al.  A Combinatory Logic , 1995 .

[30]  Jieh Hsiang,et al.  Automated proofs of the moufang identities in alternative rings , 1990, Journal of Automated Reasoning.

[31]  G Sutcliffe,et al.  The practice of clausification in automatic theorem proving , 1996 .

[32]  Dan Benanav.  Recognizing unnecessary clauses in resolution based systems , 2004, Journal of Automated Reasoning.

[33]  Donald W. Loveland,et al.  Mechanical Theorem-Proving by Model Elimination , 1968, JACM.

[34]  Alan Bundy.  Proceedings of the 12th International Conference on Automated Deduction , 1994 .

[35]  S. Lane.  Categories for the Working Mathematician , 1971 .

[36]  Witold Lukaszewicz,et al.  Reasoning about Plans , 1997, IJCAI.

[37]  A. Tarski,et al.  Metamathematische Methoden in der Geometrie , 1983 .

[38]  Leon Henkin,et al.  Completeness in the theory of types , 1950, Journal of Symbolic Logic.

[39]  B. H. Neumann.  Another single law for groups , 1981, Bulletin of the Australian Mathematical Society.

[40]  William McCune.  Challenge Equality Problems in Lattice Theory , 1988, CADE.

[41]  Mark E. Stickel,et al.  Complete Sets of Reductions for Some Equational Theories , 1981, JACM.

[42]  Steven K. Winker.  Generation and Verification of Finite Models and Counterexamples Using an Automated Theorem Prover Answering Two Open Questions , 1982, JACM.

[43]  Rick L. Stevens,et al.  Challenge Problems from Nonassociative Rings for Theorem Provers , 1988, CADE.

[44]  Rick Stevens,et al.  Automated Reasoning Contributed to Mathematics and Logic , 1990, CADE.

[45]  Jeffrey D. Ullman,et al.  Introduction to Automata Theory, Languages and Computation , 1979 .

[46]  Frances G. Livingston.  Dewey Decimal Classification and Relative Index. , 1966 .

[47]  Richard Montague.  On the Paradox of Grounded Classes , 1955, J. Symb. Log..

[48]  W. W. Bledsoe,et al.  Non-Resolution Theorem Proving , 1977, Artif. Intell..

[49]  A. Church.  Review: Carew A. Meredith, Single Axioms for the Systems $(C, N), (C, 0)$ and $(A, N)$ of the Two- Valued Propositional Calculus , 1954 .

[50]  Larry Wos,et al.  Challenge Problems Focusing on Equality and Combinatory Logic: Evaluating Automated Theorem-Proving Programs , 1988, CADE.

[51]  Matthias Fuchs,et al.  The application of goal-oriented heuristics for prooving equational theorems via the unfailing Knuth-Bendix completion procedure A case study: lattice ordered groups , 1999 .

[52]  Peter B. Andrews.  An introduction to mathematical logic and type theory - to truth through proof , 1986, Computer science and applied mathematics.

[53]  James R. Slagle,et al.  Automatic Theorem Proving With Renamable and Semantic Resolution , 1967, JACM.

[54]  Alan Bundy,et al.  The Computer Modelling of Mathematical Reasoning , 1983 .

[55]  Mark E. Stickel,et al.  Upside-down meta-interpretation of the model elimination theorem-proving procedure for deduction and abduction , 1994, Journal of Automated Reasoning.

[56]  G. Boolos,et al.  What is the name of this book? : the riddle of Dracula and other logical puzzles , 1979 .

[57]  Frank Pfenning.  Single Axioms in the Implicational Propositional Calculus , 1988, CADE.

[58]  Geoff Sutcliffe,et al.  The TPTP Problem Library , 1994, CADE.

[59]  Garrett Birkhoff,et al.  A survey of modern algebra , 1942 .

[60]  W. W. Bledsoe,et al.  Hierarchical deduction , 1987, Journal of Automated Reasoning.

[61]  Stephen A. Cook,et al.  The Relative Efficiency of Propositional Proof Systems , 1979, Journal of Symbolic Logic.

[62]  David A. Plaisted,et al.  Semantically Guided First-Order Theorem Proving using Hyper-Linking , 1994, CADE.

[63]  William McCune,et al.  OTTER 3.0 Reference Manual and Guide , 1994 .

[64]  Steven K. Winker,et al.  Robbins algebra: Conditions that make a near-boolean algebra boolean , 1990, Journal of Automated Reasoning.

[65]  William McCune,et al.  OTTER 1.0 Users' Guide , 1990 .

[66]  Elliott Mendelson,et al.  Introduction to Mathematical Logic , 1979 .

[67]  Hantao Zhang,et al.  Automated proofs of equality problems in Overbeek's competition , 1993, Journal of Automated Reasoning.

[68]  Hans Jürgen Ohlbach.  Predicate logic hacker tricks , 2004, Journal of Automated Reasoning.

[69]  A. N. Prior,et al.  Equational logic , 1968, Notre Dame J. Formal Log..

[70]  Robert E. Shostak.  Refutation Graphs , 1976, Artif. Intell..

[71]  Larry Wos,et al.  Experiments in Automated Deduction with Condensed Detachment , 1992, CADE.

[72]  Christoph Goller,et al.  Controlled integration of the cut rule into connection tableau calculi , 2004, Journal of Automated Reasoning.

[73]  D. Scott.  Identity and existence in intuitionistic logic , 1979 .

[74]  Donald W. Loveland.  Theorem-Provers Combining Model Elimination and Resolution , 1983 .

[75]  François Bry,et al.  SATCHMO: A Theorem Prover Implemented in Prolog , 1988, CADE.

[76]  M. Stickel,et al.  Automated reasoning and exhaustive search: Quasigroup existence problems☆ , 1995 .

[77]  K. Ross.  Elementary Analysis: The Theory of Calculus , 1980 .

[78]  Joyce Friedman.  A Semi-Decision Procedure for the Functional Calculus , 1963, JACM.

[79]  Maria Paola Bonacina,et al.  An Application of the Theorem Prover SBR 3 to Many-valued Logic , .

[80]  Henk Barendregt,et al.  The Lambda Calculus: Its Syntax and Semantics , 1985 .

[81]  Christian G. Fermüller,et al.  Resolution Methods for the Decision Problem , 1993, Lecture Notes in Computer Science.

[82]  Larry Wos,et al.  Automated reasoning - 33 basic research problems , 1988 .

[83]  Art Quaife,et al.  Automated development of Tarski's geometry , 1989, Journal of Automated Reasoning.

[84]  Ross A. Overbeek,et al.  A foray into combinatory logic , 1986, Journal of Automated Reasoning.

[85]  W. W. Bledsoe,et al.  Challenge problems in elementary calculus , 1990, Journal of Automated Reasoning.

[86]  Masayuki Fujita,et al.  Model Generation Theorem Provers on a Parallel Inference Machine , 1992, FGCS.

[87]  Robert S. Boyer,et al.  Computer Proofs of Limit Theorems , 1971, IJCAI.

[88]  Hans Hermes,et al.  Introduction to mathematical logic , 1973, Universitext.

[89]  Ross A. Overbeek.  The CADE-11 competitions: A personal view , 2004, Journal of Automated Reasoning.

[90]  Larry Wos.  Searching for Fixed Point Combinators by Using Automated Theorem Proving : a Preliminary Report , 1988 .

[91]  E. Wright,et al.  An Introduction to the Theory of Numbers , 1939 .

[92]  Anthony S. Wojcik,et al.  Formal Design Verification of Digital Systems , 1983, 20th Design Automation Conference Proceedings.

[93]  I. S. Gradshtein,et al.  THE ELEMENTS OF MATHEMATICAL LOGIC , 1963 .

[94]  Proceedings of the Glasgow Mathematical Association , 1952, Nature.

[95]  Allen Newell,et al.  Empirical explorations with the logic theory machine: a case study in heuristics , 1995 .

[96]  Tie-Cheng Wang.  Designing Examples for Semantically Guided Hierarchical Deduction , 1985, IJCAI.

[97]  Larry Wos,et al.  A case study in automated theorem proving: Finding sages in combinatory logic , 1987, Journal of Automated Reasoning.

[98]  Charles Elkan,et al.  A High-Performance Explanation-Based Learning Algorithm , 1994, Artif. Intell..

[99]  William McCune,et al.  Automated reasoning about elementary point-set topology , 2004, Journal of Automated Reasoning.

[100]  Dennis de Champeaux.  Sub-problem Finder and Instance Checker: Two Cooperating Preprocessors for Theorem Provers , 1979, IJCAI.

[101]  Larry Wos,et al.  Benchmark Problems in Which Equality Plays the Major Role , 1992, CADE.

[102]  David A. Plaisted,et al.  Eliminating duplication with the hyper-linking strategy , 1992, Journal of Automated Reasoning.

[103]  Rolf Socher-Ambrosius.  How to avoid the derivation of redundant clauses in reasoning systems , 2004, Journal of Automated Reasoning.

[104]  G. B. M..  Principia Mathematica , 1911, Nature.

[105]  Larry Wos,et al.  Problems and Experiments for and with Automated Theorem-Proving Programs , 1976, IEEE Transactions on Computers.

[106]  J. A. Robinson,et al.  Theorem-Proving on the Computer , 1963, JACM.

[107]  A. Tarski.  A Decision Method for Elementary Algebra and Geometry , 2023 .

[108]  J. Neumann.  Eine Axiomatisierung der Mengenlehre. , 1925 .

[109]  Larry Wos,et al.  The kernel strategy and its use for the study of combinatory logic , 1993, Journal of Automated Reasoning.

[110]  B. H. Neumann.  Yet another single law for groups , 1986 .

[111]  Thomas Jech.  OTTER experiments in a system of combinatory logic , 2004, Journal of Automated Reasoning.

[112]  Peter L. Balise,et al.  Boolean Algebra and Its Applications , 1995 .

[113]  N. Bourbaki.  Algebra I: Chapters 1-3 , 1989 .

[114]  Ewing L. Lusk,et al.  Non-horn problems , 2004, Journal of Automated Reasoning.

[115]  William McCune,et al.  Automated discovery of new axiomatizations of the left group and right group calculi , 1992, Journal of Automated Reasoning.

[116]  Masayuki Fujita,et al.  Automatic Generation of Some Results in Finite Algebra , 1993, IJCAI.

[117]  JOHN F. Young.  Machine Intelligence , 1971, Nature.

[118]  David A. Plaisted,et al.  Theorem Proving with Abstraction , 1981, Artif. Intell..

[119]  L Burkholder,et al.  The halting problem , 1987, SIGA.

[120]  R. Ramanujam,et al.  Forever undecided: A puzzle guide to Gödel , 2001 .

[121]  Thomas Hillenbrand,et al.  On Gaining Efficiency in Completion-Based Theorem Proving , 1996, RTA.

[122]  Ricardo Caferra,et al.  A Method for Building Models Automatically. Experiments with an Extension of OTTER , 1994, CADE.

[123]  David A. Plaisted.  The Search Efficiency of Theorem Proving Strategies , 1994, CADE.

[124]  J. Denbigh Starkey,et al.  Experimental tests of resolution-based theorem-proving strategies , 1976, Inf. Sci..

[125]  Anthony S. Wojcik,et al.  Automated Design of Multiple-Valued Logic Circuits by Automatic Theorem Proving Techniques , 1983, IEEE Transactions on Computers.

[126]  Rick L. Stevens,et al.  Some experiments in nonassociative ring theory with an automated theorem prover , 1987, Journal of Automated Reasoning.

[127]  Owen L. Astrachan.  METEOR: Exploring model elimination theorem proving , 2004, Journal of Automated Reasoning.

[128]  Wolfgang Bibel,et al.  SETHEO: A high-performance theorem prover , 1992, Journal of Automated Reasoning.

[129]  Kenneth Kunen.  Single axioms for groups , 2004, Journal of Automated Reasoning.

[130]  J. W. Addison,et al.  Automated development of fundamental mathematical theories , 1992 .

[131]  Alfred Tarski.  Ein Beitrag zur Axiomatik der Abelschen Gruppen , 1938 .

[132]  Bernhard Beckert,et al.  leanTAP: Lean tableau-based deduction , 1995, Journal of Automated Reasoning.

[133]  A. Tarski.  What is Elementary Geometry , 1959 .

[134]  Frank Harary,et al.  Graph Theory , 2016 .

[135]  David A. Plaisted,et al.  A Simplified Problem Reduction Format , 1982, Artif. Intell..

[136]  Art Quaife,et al.  Automated deduction in von Neumann-Bernays-Gödel set theory , 1992, Journal of Automated Reasoning.

[137]  Geoff Sutcliffe,et al.  The CADE-13 ATP System Competition , 2004, Journal of Automated Reasoning.

[138]  Donald W. Loveland,et al.  An Implementation of the Model Elimination Proof Procedure , 1974, JACM.

[139]  Geoff Sutcliffe,et al.  ATP System Results for the TPTP Problem Library , 1995 .

[140]  Thomas C. Bartee,et al.  Modern Applied Algebra , 1999 .

[141]  Olgierd Wojtasiewicz,et al.  Elements of mathematical logic , 1964 .

[142]  Ewing L. Lusk,et al.  Reasoning about equality , 2004, Journal of Automated Reasoning.

[143]  Alasdair Urquhart,et al.  Formal Languages]: Mathematical Logic--mechanical theorem proving , 2022 .

[144]  Larry Wos,et al.  What Is Automated Reasoning? , 1987, J. Autom. Reason..

[145]  Lewis Carroll,et al.  Lewis Carroll's Symbolic Logic , 1977 .

[146]  Henry A. Kautz,et al.  Reasoning about plans , 1991, Morgan Kaufmann series in representation and reasoning.

[147]  Jack Minker,et al.  Resolution, Refinements, and Search Strategies: A Comparative Study , 1976, IEEE Transactions on Computers.

[148]  William McCune,et al.  Uniform strategies: The CADE-11 theorem proving contest , 2004, Journal of Automated Reasoning.

[149]  Hans Jürgen Ohlbach,et al.  The lion and the unicorn , 2004, Journal of Automated Reasoning.

[150]  W. T. Harwood,et al.  Problem Set , 2018, Friction, Wear, Lubrication.

[151]  James R. Munkres,et al.  Topology; a first course , 1974 .

[152]  David A. Plaisted,et al.  Non-Horn clause logic programming without contrapositives , 1988, Journal of Automated Reasoning.