Case studies and tools for contract specifications

Contracts are a popular tool for specifying the functional behavior of software. This paper characterizes the contracts that developers write, the contracts that developers could write, and how a developer reacts when shown the difference. This paper makes three research contributions based on an investigation of open-source projects' use of Code Contracts. First, we characterize Code Contract usage in practice. For example, approximately three-fourths of the Code Contracts are basic checks for the presence of data. We discuss similarities and differences in usage across the projects, and we identify annotation burden, tool support, and training as possible explanations based on developer interviews. Second, based on contracts automatically inferred for four of the projects, we find that developers underutilize contracts for expressing state updates, object state indicators, and conditional properties. Third, we performed user studies to learn how developers decide which contracts to enforce. The developers used contract suggestions to support their existing use cases with more expressive contracts. However, the suggestions did not lead them to experiment with other use cases for which contracts are better-suited. In support of the research contributions, the paper presents two engineering contributions: (1) Celeriac, a tool for generating traces of .NET programs compatible with the Daikon invariant detection tool, and (2) Contract Inserter, a Visual Studio add-in for discovering and inserting likely invariants as Code Contracts.

[1]  Michael D. Ernst,et al.  Invariant inference for static checking: an empirical evaluation , 2002, SOEN.

[2]  Bertrand Meyer,et al.  Contract driven development = test driven development - writing test cases , 2007, ESEC-FSE '07.

[3]  Robert W. Bowdidge,et al.  Why don't software developers use static analysis tools to find bugs? , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[4]  Nikolai Tillmann,et al.  Exploiting the synergy between automated-test-generation and programming-by-contract , 2009, 2009 31st International Conference on Software Engineering - Companion Volume.

[5]  Michael D. Ernst,et al.  Invariant inference for static checking: , 2002, SIGSOFT '02/FSE-10.

[6]  Bertrand Meyer,et al.  What good are strong specifications? , 2012, 2013 35th International Conference on Software Engineering (ICSE).

[7]  Gary T. Leavens,et al.  A Simple and Practical Approach to Unit Testing: The JML and JUnit Way , 2002, ECOOP.

[8]  Yannis Smaragdakis,et al.  Dynamically discovering likely interface invariants , 2006, ICSE '06.

[9]  Patrick Cousot,et al.  An abstract interpretation framework for refactoring with application to extract methods with contracts , 2012, OOPSLA '12.

[10]  Michael D. Ernst,et al.  Practical pluggable types for java , 2008, ISSTA '08.

[11]  Michael D. Ernst,et al.  Reducing the barriers to writing verified specifications , 2012, OOPSLA '12.

[12]  Robert O'Callahan,et al.  Generalized aliasing as a basis for program analysis tools , 2001 .

[13]  Robert O'Callahan,et al.  Lackwit: A Program Understanding Tool Based on Type Inference , 1997, Proceedings of the (19th) International Conference on Software Engineering.

[14]  Thomas Ball,et al.  Modular and verified automatic program repair , 2012, OOPSLA '12.

[15]  Bertrand Meyer.  Attached Types and Their Application to Three Open Problems of Object-Oriented Programming , 2005, ECOOP.

[16]  Philip J. Guo.  A Scalable Mixed-Level Approach to Dynamic Analysis of C and C++ Programs , 2006 .

[17]  Patrick Cousot,et al.  A semantic integrated development environment , 2012, SPLASH '12.

[18]  Bertrand Meyer,et al.  Contracts in Practice , 2012, FM.

[19]  Michael D. Ernst,et al.  An overview of JML tools and applications , 2003, International Journal on Software Tools for Technology Transfer.

[20]  Stephen McCamant,et al.  The Daikon system for dynamic detection of likely invariants , 2007, Sci. Comput. Program..

[21]  Greg Nelson,et al.  Extended static checking for Java , 2002, PLDI '02.

[22]  Mira Mezini,et al.  Ieee Transactions on Software Engineering 1 Automated Api Property Inference Techniques , 2022 .

[23]  Patrice Chalin,et al.  Non-null References by Default in Java: Alleviating the Nullity Annotation Burden , 2007, ECOOP.

[24]  Patrice Chalin,et al.  JML Runtime Assertion Checking: Improved Error Reporting and Efficiency Using Strong Validity , 2008, FM.

[25]  Michael D. Ernst,et al.  Javari: adding reference immutability to Java , 2005, OOPSLA '05.

[26]  Bertrand Meyer,et al.  Automatic Verification of Advanced Object-Oriented Features: The AutoProof Approach , 2011, LASER Summer School.

[27]  Sebastián Uchitel,et al.  Contractor.NET: inferring typestate properties to enrich code contracts , 2011, TOPI '11.

[28]  Manuel Fähndrich,et al.  Static Contract Checking with Abstract Interpretation , 2010, FoVeOOS.

[29]  Robert Andrew Rudd.  An Improved Scalable Mixed-Level Approach to Dynamic Analysis of C and C++ Programs , 2010 .

[30]  Daan Leijen,et al.  Integrating a set of contract checking tools into Visual Studio , 2012, 2012 Second International Workshop on Developing Tools as Plug-Ins (TOPI).

[31]  Nikolai Tillmann,et al.  Parameterized unit tests , 2005, ESEC/FSE-13.

[32]  Stephen McCamant,et al.  Inference and enforcement of data structure consistency specifications , 2006, ISSTA '06.

[33]  Patrice Chalin,et al.  Non-null references by default in the Java modeling language , 2006, SAVCBS '05.

[34]  K. Rustan M. Leino,et al.  The Spec# Programming System: An Overview , 2004, CASSIS.

[35]  Henry G. Baker,et al.  Unify and conquer , 1990, LISP and Functional Programming.

[36]  Patrice Chalin,et al.  Are Practitioners Writing Contracts? , 2006, RODIN Book.

[37]  MeyerBertrand,et al.  Design by Contract , 1997 .

[38]  William G. Griswold,et al.  Dynamically discovering likely program invariants to support program evolution , 1999, Proceedings of the 1999 International Conference on Software Engineering (IEEE Cat. No.99CB37002).

[39]  Michael D. Ernst,et al.  Automatic generation of program specifications , 2002, ISSTA '02.

[40]  Nikolai Tillmann,et al.  Pex-White Box Test Generation for .NET , 2008, TAP.

[41]  Gary T. Leavens,et al.  How the design of JML accommodates both runtime assertion checking and formal verification , 2003, Sci. Comput. Program..

[42]  Patrick Cousot,et al.  Automatic Inference of Necessary Preconditions , 2013, VMCAI.

[43]  William G. Griswold,et al.  Automated support for program refactoring using invariants , 2001, Proceedings IEEE International Conference on Software Maintenance. ICSM 2001.

[44]  Nikolai Tillmann,et al.  DySy: dynamic symbolic execution for invariant inference , 2008, ICSE.

[45]  Michael D. Ernst,et al.  Automatically patching errors in deployed software , 2009, SOSP '09.

[46]  Stephen McCamant,et al.  Dynamic inference of abstract types , 2006, ISSTA '06.

[47]  Michael D. Ernst,et al.  Building and using pluggable type-checkers , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[48]  Gregg Rothermel,et al.  Understanding user understanding: determining correctness of generated program invariants , 2012, ISSTA 2012.

[49]  Bertrand Meyer,et al.  A comparative study of programmer-written and automatically inferred contracts , 2009, ISSTA.

[50]  Bertrand Meyer,et al.  Applying 'design by contract' , 1992, Computer.