Preliminary Summary: FM89 Assessment of Formal Methods for Trustworthy Computer Systems

ion is a prerequisite to design; extensive review procedures require detailed, well structured, and precise documentation; specialized training and high skill levels substitute for professional certification; 154 l the loss of overall intellectual control of a project is sufficient to question the viability of the project. A variety of Formal Methods address these situations to varying degrees. Each is characterized by notations for expressing specifications and programs, logical systems to support reasoning about these, and prescriptions for specifying, implementing, and analysing. Some use strongly communicative notations and work best at high levels of abstraction, while others capture the detail of hardware and software implementations with great precision. Certain methods are geared toward different kinds of analyses, e.g. division into cases corresponding to modes of a system or inductive proofs over a data space. Methods are mature to different levels in their theoretical underpinnings, in the instruction materials available to teach them, and in the understanding of their best and worst domains of applicability. Dozens of variations of Formal Methods have been investigated over the past 20 years and a few are now gaining industrial acceptance through attractive tool support, intensive technology transfers, or demonstrations of merit. Formal Methods face a variety of challenges. The intrinsic limits of Formal Methods are the same as for any engineering discipline: the faithfulness of the mapping between models and the physical world, the ability to capture intent and assumptions in a requirements statement, and the degree of approximation required during the engineering process. A great deal can be done today with current methods and toolsets, as shown in the case studies and documented in a survey at the workshop. Much more will be possible as the methods become better understood, as toolsets improve through better managed investment and through industrial engineering use, and as theory pinpoints the key problems and offers new solutions. However, the need for more rigorous methods is not widely recognized in the computer industry. Nor are they appreciated by consumers now accustomed to low quality and costly software projects, even to frequent total failures. The trustworthy system area highlights this need because standard commercial practice fails miserably. Formal Methods repeatedly face social backlash from misunderstanding of terminology, e.g. a proof is not a “guarantee” but instead “an argument of some identified level of formality relative to a number of assumptions about the operating environment”. The path toward proficiency in Formal Methods requires a mathematical background, level of specific method training, and guided experience that, although measured in months not years, is beyond the standard expected of most industrial so-called software engineers. An additional cultural distinction was drawn at the workshop. Most U.K.-based Formal Methods are oriented toward application of mathematical specification early in the software life cycle. The U.K. research on mathematical specification is to be supported eventually by toolsets and to be gradually applied further into the implementation phases of software development. In contrast, most North American-based research approaches are geared toward formal verification tools to support proofs and they emphasize greater control of implementation details later in the development cycle. North American design methods are expected to build upon the foundations of verifiable languages and verified environments. (Some projects on each side of the Atlantic exhibit characteristics of the other side.) These differing approaches frequently lead to confusion on the part of both researchers and users of Formal Methods in terms of what can be achieved and what kinds of applications are best. However, this ‘tram-Atlantic Formal Methods gap’ is viewed as a strength of the field, e.g. (1) the limits of relying solely on readilytransferred mathematical specifications have been exposed by the ultra-formal security verification community probing into the weaknesses of implementations and models, and (2) the value of mathematical specifications as arbiters of contracts and as intellectual clarifiers of system purpose and behavior has been recognized independent of verification strategy. Several steps were recommended to address the cultural and technical challenges: l A set of scrupulously documented benchmarks and real life experiences across a variety of application areas is badly needed. One working group specified the qualities of these and suggested some interesting and worthy problems . The field needs a taxonomy of methods, problems they address, modes of use, and results to be expected. Further study is required to understand the informal/formal and physical/model boundaries that determine how Formal Methods should be used. Cross-fertilization of techniques and ideas across the Atlantic is encouraged. Efforts must be made to improve the educational systems that feed into software and system engineering, providing the not unreasonably high but still unmet mathematical background required for appreciating and applying Formal Methods. Formal Methods that work must be integrated with other software engineering techniques and taught with an engineering flavor within computer science and computer engineering curricula. Finally, the ultimate forcing factor may be called upon, namely certification of professionals and curricula, especially for those providing trustworthy systems. Eventually the tools and methods will become standardized and certified as in other engineering fields.