Testing mined specifications

Specifications are necessary for nearly every software engineering task, but they are often missing or incomplete. "Specification mining" is a line of research promising to solve this problem through automated tools that infer specifications directly from existing programs. The standard practice is one of inductive learning: mining tools make observations about software and inductively generalize them into specifications. Inductive reasoning is unsound, however, and existing tools commonly grapple with the problem of inferring "false" specifications, which must be manually checked. In this work, we introduce a new technique for automatically validating mined specifications that lessens this manual burden. Our technique is not based on heuristics; it rather uses a general, semantic definition of a "true" specification. We perform systematic, targeted program transformations to test a mined specification's necessity for overall correctness. If a "violating" program is correct, the specification is false. We have implemented our technique in a prototype tool that validates temporal properties of Java programs, and we demonstrate it to be effective through a large-scale case study on the DaCapo benchmarks.