Usable Verification: Balancing Thinking and Automating

Here are five ideas on how to make verification more usable by and useful for people. They touch on education, techniques, and process. A common theme is the need to balance the need for human to think and the ability for machines to automate. Education Some understanding of formal logic is good. It is necessary and possible. Verification is based on formal logic. We know from experience that the kind of mathematical thinking needed to specify and verify systems takes facility with logic. Many developers do not have this facility. So to make verification usable, at one extreme we could try to hide the formalism. I don't think that is a good idea. While we may want push‐button verification tools, we don't want push‐button thinking. We want to make sure that people have enough understanding of formalism to do a sanity check on the results of verification tools and techniques. Achieving this goal means making sure developers are trained and educated in the basics of mathematical logic, including proof techniques and abstract modeling. The challenge is striking a balance between how much people should know and how much we can safely hide from them in our tools. A good start for usable verification is usable specification. At Carnegie Mellon, we have recently revamped our introductory classes 1. The revision includes a first‐semester first‐year class, 15‐122 Principles of Imperative Computation 2 , that teaches specification as part of the programming process, with a goal of learning how to write reliable software. In the context of a subset of C amenable to verification, students learn to write pre‐/post‐conditions, assertions, loop invariants, and representation invariants for abstract data types. The specification language is inspired by JML and Spec#. Right now, the compiler causes the specifications to be checked dynamically, but the research goal is to integrate the compiler with a back‐end tool, e.g., Boogie, to do checks statically. By mid‐October, the instructor (Frank Pfenning) was having the 100+ students in lecture interactively write code to test the representation invariants of red‐black trees. [Aside: The second‐semester first‐year class, 15‐150 Principles of Functional Programming, will be a companion course offered for the first time Spring 2011.] Techniques Make machines do what they can do best. Provide " Think Now " or " Do Now " guideposts for humans to do what they do best and what verification tools cannot yet do.