Pinnacles of software engineering: 25 years of formal methods

In this invited paper we review 25 years of propagating formal specification in software engineering. We will do so through outlining a paradigmatic approach to the practice of software engineering. For the sake of contrasting argument we shall claim that this approach stands in sharp contrast to classical engineering – and that there is little help to be obtained from classical engineering in securing the quality of the most important facets of software engineering! We shall be outlining a software engineering* practice in which formal techniques are applied in capturing the application domain void of any reference to requirements let alone software; and in then capturing requirements: Domain requirements (projected, instantiated, possibly extended and usually initialised from domain descriptions), interface requirements and machine requirements. The software engineering practice then goes on to design the software: First the architecture, then the program structure, etc. Throughout abstraction and modelling, hand‐in‐hand, are used in applicative (functional), imperative and process oriented descriptions, from loose specifications towards concrete, instantiated descriptions, using hierarchical as well as configurational modelling, denotational as well as computational modelling, and in structuring even small scale descriptions using appropriate modularisation concepts: Schemes, classes and objects. All the concepts spelled in this font are software engineering “program” description notions that have been honed over the years, starting in 1973 with VDM [Bekić et al. 1974; Bjørner and Jones 1978, 1982b] and continuing with RAISE [Group 1992, 1995]. The current status of our approach to software engineering, based on extensive, but not exclusive use of formal techniques, developed significantly during my years as UN Director of the UN University's International Institute for Software Technology (UNU/IIST) in Macau, 1992–1997. Many large scale software developments based on the domain/requirements/software design paradigm outlined here were systematically applied to the experimental development of software designs for the computing support of a number of diverse infrastructure components**. Special boxes, scattered throughout the text, highlight ‘pinnacle’ contribution by named computer and computing scientists as well as by specific R&D projects.*The text high‐lighted in ‘this font’ identifies important software development principles and techniques – the main contribution and subject of this quarter century overview.**Railways [Bjørner et al. 1997; George 1995], financial service institutions [Bjørner 1997; Bjørner et al. 1998], manufacturing [Bjørner 1996; Janowski 1996; Janowski and Acebedo 1996; Janowski and Atienza 1997], ministry of finance [Dung et al. 1996; George et al. 1995], airlines [Anh and More 1996], air traffic [Bjørner 1995], etc.

[1]  Andrew William Roscoe,et al.  The Theory and Practice of Concurrency , 1997 .

[2]  Joseph A. Goguen,et al.  Initial Algebra Semantics and Continuous Algebras , 1977, J. ACM.

[3]  Michael A. Jackson,et al.  Software requirements and specifications - a lexicon of practice, principles and prejudices , 1995 .

[4]  Niklaus Wirth,et al.  The oberon system , 1989, Softw. Pract. Exp..

[5]  Martin Reiser Oberon system - user guide and programmer's manual , 1991 .

[6]  Christopher Strachey,et al.  A theory of programming language semantics , 1976 .

[7]  Chris George A Theory of Distributing Train Rescheduling , 1996, FME.

[8]  Dana S. Scott,et al.  Some Domain Theory and Denotational Semantics in Coq , 2009, TPHOLs.

[9]  Jim Woodcock,et al.  Software engineering mathematics : formal methods demystified , 1988 .

[10]  Niklaus Wirth The Programming Language Pascal (Reprint) , 2002, Software Pioneers.

[11]  Dines Bjo.KC rner Project graphs and meta-programs. Towards a theory of software development , 1987 .

[12]  Peter D. Mosses CoFI: The Common Framework Initiative for Algebraic Specification , 1996, Bull. EATCS.

[13]  Dana S. Scott,et al.  Some Ordered Sets in Computer Science , 1982 .

[14]  Christopher Strachey,et al.  The varieties of programming language , 1972 .

[15]  Patrick Cousot,et al.  Abstract interpretation: a unified lattice model for static analysis of programs by construction or approximation of fixpoints , 1977, POPL.

[16]  P. J. Landin,et al.  Correspondence between ALGOL 60 and Church's Lambda-notation , 1965, Commun. ACM.

[17]  Anders P. Ravn,et al.  Specifying and Verifying Requirements of Real-Time Systems , 1993, IEEE Trans. Software Eng..

[18]  Yoshihiko Futamura,et al.  Partial Evaluation of Computation Process--An Approach to a Compiler-Compiler , 1999, High. Order Symb. Comput..

[19]  A. Ravn,et al.  A Probabilistic Duration Calculus , 1992 .

[20]  Dana S. Scott,et al.  The lattice of flow diagrams , 1971, Symposium on Semantics of Algorithmic Languages.

[21]  C. A. R. Hoare,et al.  A contribution to the development of ALGOL , 1966, CACM.

[22]  Martín Abadi,et al.  A Theory of Objects , 1996, Monographs in Computer Science.

[23]  Niklaus Wirth,et al.  The programming language oberon , 1988, Softw. Pract. Exp..

[24]  Hartmut Ehrig,et al.  Fundamentals of Algebraic Specification 1: Equations and Initial Semantics , 1985 .

[25]  John McCarthy,et al.  A BASIS FOR A MATHEMATICAL THEORY OF COMPUTATION 1) , 2018 .

[26]  Mary Shaw,et al.  An Introduction to Software Architecture , 1993, Advances in Software Engineering and Knowledge Engineering.

[27]  Michael Jackson,et al.  The meaning of requirements , 1997, Ann. Softw. Eng..

[28]  Ian J. Hayes,et al.  Specification case studies , 1987 .

[29]  David Garlan,et al.  A case study in architectural modeling: the AEGIS system , 1996, Proceedings of the 8th International Workshop on Software Specification and Design.

[30]  Gregory D. Abowd,et al.  Using style to understand descriptions of software architecture , 1993, SIGSOFT '93.

[31]  Jim Woodcock,et al.  FME'96: Industrial Benefit and Advances in Formal Methods , 1996, Lecture Notes in Computer Science.

[32]  Dines Bjørner,et al.  Formal specification and software development , 1982 .

[33]  P. J. Landin The Mechanical Evaluation of Expressions , 1964, Comput. J..

[34]  Cliff B. Jones,et al.  Software development - a rigorous approach , 1980, Prentice Hall international series in computer science.

[35]  C. A. R. Hoare,et al.  Laws of programming , 1987, CACM.

[36]  Kurt Jensen Coloured Petri Nets , 1992, EATCS Monographs in Theoretical Computer Science.

[37]  Zhou Chaochen,et al.  A mean value calculus of durations , 1994 .

[38]  Robin Milner,et al.  A Calculus of Communicating Systems , 1980, Lecture Notes in Computer Science.

[39]  Peter Sestoft,et al.  Partial evaluation and automatic program generation , 1993, Prentice Hall international series in computer science.

[40]  Joseph A. Goguen,et al.  Putting Theories Together to Make Specifications , 1977, IJCAI.

[41]  Dines Bjørner,et al.  Domains as a Prerequisite for Requirements and Software Domain Perspectives & Facets, Requirements Aspects and Software Views , 1997, Requirements Targeting Software and Systems Engineering.

[42]  Edsger W. Dijkstra,et al.  A Discipline of Programming , 1976 .

[43]  D. Bjoerner,et al.  Towards a Formal Description of Ada , 1980, Lecture Notes in Computer Science.

[44]  Uwe Schmidt,et al.  Experience with VDM in Norsk Data , 1987, VDM Europe.

[45]  Zhou Chaochen,et al.  Duration Calculi: An overview , 1993 .

[46]  Thomas I. M. Ho Review of On programming: an interim report on the SETL project, intallment II: the SETL language and examples of its use by J. T. Schwartz. New York University, 1973. , 1974, SIGP.

[47]  David Garlan,et al.  Formalizing architectural connection , 1994, Proceedings of 16th International Conference on Software Engineering.

[48]  Dines Bjørner,et al.  Abstract Software Specifications , 1982, Lecture Notes in Computer Science.

[49]  Dines Bjørner,et al.  A formal definition of a PL/I subset , 1984 .

[50]  Hans-Jörg Kreowski,et al.  Algebraic system specification and development , 1991, Lecture Notes in Computer Science.

[51]  Luca Cardelli,et al.  On understanding types, data abstraction, and polymorphism , 1985, CSUR.

[52]  Miquel Bertran,et al.  FME '94 : industrial benefit of formal methods : Second International Symposium of Formal Methods Europe, Barcelona, Spain, October 24-28, 1994 : proceedings , 1994, FME 1994.

[53]  Dines Bjørner,et al.  The Vienna Development Method: The Meta-Language , 1978, Lecture Notes in Computer Science.

[54]  Edsger W. Dijkstra,et al.  Structured programming , 1972, A.P.I.C. Studies in data processing.

[55]  David Garlan,et al.  The role of software architecture in requirements engineering , 1994, Proceedings of IEEE International Conference on Requirements Engineering.

[56]  Johan Lewi,et al.  Algebraic Specifications in Software Engineering , 1989, Springer Berlin Heidelberg.

[57]  Chaochen Zhou Duration Calculi: An Overview (Invited Paper) , 1993, Formal Methods in Programming and Their Applications.

[58]  Dines Bjørner Project Graphs and Meta-Programs. Towards a Theory of Software Development , 1986, System Development and Ada.

[59]  Hartmut Ehrig,et al.  Fundamentals of Algebraic Specification 2: Module Specifications and Constraints , 1990 .

[60]  Xu Qiwen,et al.  Hierarchical design of a chemical concentration control system , 1996 .

[61]  J. Schwartz Mathematical Aspects of Computer Science , 1967 .

[62]  Michel Bidoit,et al.  Algebraic System Specification and Development: A Survey and Annotated Bibliography , 1991 .

[63]  Michael Jackson,et al.  Requirements for telecommunications services: an attack on complexity , 1997, Proceedings of ISRE '97: 3rd IEEE International Symposium on Requirements Engineering.

[64]  John McCarthy,et al.  Correctness of a compiler for arithmetic expressions , 1966 .

[65]  Anders P. Ravn,et al.  An Extended Duration Calculus for Hybrid Real-Time Systems , 1992, Hybrid Systems.

[66]  Jean-Raymond Abrial,et al.  The B-book - assigning programs to meanings , 1996 .

[67]  Niklaus Wirth,et al.  Pascal User Manual and Report , 1991, Springer New York.

[68]  Mícheál Mac an Airchinnigh Tutorial on the Irish School of the VDM , 1991, VDM Europe.

[69]  David Garlan,et al.  Research directions in software architecture , 1995, CSUR.

[70]  Yoshihiko Futamura,et al.  Essence of Generalized Partial Computation , 1991, Theor. Comput. Sci..

[71]  Niklaus Wirth,et al.  Project Oberon - the design of an operating system and compiler , 1992 .

[72]  Tony Hoare,et al.  Notes on Data Structuring , 1972 .

[73]  D Bjorner Where do software architectures come from? Systematic development from domains and requirements. A re-assessment of software engineering? , 1998 .

[74]  Cliff B. Jones,et al.  Programming Languages and Their Definition , 1984, Lecture Notes in Computer Science.

[75]  David Garlan,et al.  A Formal Approach to Software Architectures , 1992, IFIP Congress.

[76]  Dines Bjørner,et al.  Software Engineering Education: R^ Oles of Formal Speciication and Design Calculi , 1998 .

[77]  Irène Guessarian,et al.  Algebraic semantics , 1981, Lecture Notes in Computer Science.

[78]  Jan A. Bergstra,et al.  Algebraic specification , 1989 .

[79]  J. Girard,et al.  Proofs and types , 1989 .

[80]  Kees Middelburg The VIP VDM Specification Language , 1988, VDM Europe.

[81]  Patrick Cousot,et al.  Abstract interpretation , 1996, CSUR.

[82]  C. A. R. Hoare,et al.  A Calculus of Durations , 1991, Inf. Process. Lett..

[83]  Michael Jackson,et al.  Problems, methods and specialisation , 1994, Softw. Eng. J..

[84]  Stephen N. Zilles,et al.  Programming with abstract data types , 1974 .

[85]  Dines Bjørner The Stepwise Development of Software Development Graphs: Meta-Programming VDM Developments , 1987, VDM Europe.

[86]  Anders Olsen,et al.  Use of VDM within CCITT , 1987, VDM Europe.

[87]  Dana S. Scott,et al.  Outline of a Mathematical Theory of Computation , 1970 .

[88]  Chaochen Zhou,et al.  A Duration Calculus with Infinite Intervals , 1995, FCT.

[89]  Anne Elisabeth Haxthausen,et al.  The Raise Specification Language , 1992 .

[90]  Edsger W. Dijkstra,et al.  Guarded commands, nondeterminacy and formal derivation of programs , 1975, Commun. ACM.

[91]  Robin Milner,et al.  Definition of standard ML , 1990 .

[92]  Niklaus Wirth,et al.  Programming in Modula-2 , 1985, Texts and Monographs in Computer Science.

[93]  J. Van Leeuwen,et al.  Handbook of theoretical computer science - Part A: Algorithms and complexity; Part B: Formal models and semantics , 1990 .

[94]  E. Engeler Symposium on Semantics of Algorithmic Languages , 1971, Symposium on Semantics of Algorithmic Languages.

[95]  José Meseguer,et al.  Principles of OBJ2 , 1985, POPL.

[96]  Niklaus Wirth,et al.  EULER: a generalization of ALGOL, and its formal definition: Part II , 1965, CACM.

[97]  Jim Woodcock,et al.  Software engineering mathematics , 1988 .

[98]  Niklaus Wirth,et al.  Algorithms + Data Structures = Programs , 1976 .

[99]  Bengt Nordström,et al.  Programming in Martin-Lo¨f's type theory: an introduction , 1990 .

[100]  Dave Roberts,et al.  Developing Software Using OVID , 1997, IEEE Softw..

[101]  Hartmut Ehrig,et al.  Fundamentals of Algebraic Specification 1 , 1985, EATCS Monographs on Theoretical Computer Science.

[102]  Dana S. Scott,et al.  Lectures on a Mathematical Theory of Computation , 1982 .

[103]  Anne Elisabeth Haxthausen,et al.  CASL - The CoFI Algebraic Specification Language (Tentative Design, version 0.95) - Language Summary, with annotations concerning the semantics of constructs , 1997 .

[104]  Joseph A. Goguen,et al.  The Semantics of CLEAR, A Specification Language , 1979, Abstract Software Specifications.

[105]  John McCarthy,et al.  Recursive functions of symbolic expressions and their computation by machine, Part I , 1960, Commun. ACM.

[106]  Miquel Bertran,et al.  FME '94: Industrial Benefit of Formal Methods , 1994, Lecture Notes in Computer Science.

[107]  Niklaus Wirth,et al.  EULER: A generalization of ALGOL and its formal definition: Part 1 , 1966, Commun. ACM.

[108]  Niklaus Wirth,et al.  From modula to oberon , 1988, Softw. Pract. Exp..

[109]  Dines Bjørner,et al.  The DDC Ada Compiler Development Project , 1980, Towards a Formal Description of Ada.

[110]  Anders P. Ravn,et al.  Specification Of Embedded, Real-time Systems , 1992, Fourth Euromicro workshop on Real-Time Systems.

[111]  Patrick Cousot,et al.  Constructive design of a hierarchy of semantics of a transition system by abstract interpretation , 2002, MFPS.

[112]  Robin Milner,et al.  Communication and concurrency , 1989, PHI Series in computer science.

[113]  John McCarthy,et al.  Towards a Mathematical Science of Computation , 1962, IFIP Congress.

[114]  Ivo Van Horebeek,et al.  Algebraic Specifications in Software Engineering: An Introduction , 1989 .

[115]  Jan van Leeuwen,et al.  Handbook of Theoretical Computer Science, Vol. B: Formal Models and Semantics , 1994 .

[116]  Cliff B. Jones,et al.  Systematic software development using VDM , 1986, Prentice Hall International Series in Computer Science.

[117]  Michael Jackson,et al.  Four dark corners of requirements engineering , 1997, TSEM.

[118]  Ataru T. Nakagawa,et al.  An overview of CAFE specification environment-an algebraic approach for creating, verifying, and maintaining formal specifications over networks , 1997, First IEEE International Conference on Formal Engineering Methods.

[119]  Dines Bjørner,et al.  Algorithmic & Knowledge Based Methods - Do they "Unify" ? With some Programme Remarks for UNU/IIST , 1992, FGCS.

[120]  David Garlan,et al.  Formal Approaches to Software Architecture , 1993, ICSE Workshop on Studies of Software Design.

[121]  C. Böhm,et al.  λ-Calculus and Computer Science Theory , 1975, Lecture Notes in Computer Science.

[122]  Luca Cardelli,et al.  Basic Polymorphic Typechecking , 1987, Sci. Comput. Program..

[123]  Gregory D. Abowd,et al.  Formalizing style to understand descriptions of software architecture , 1995, TSEM.

[124]  Michael R. Hansen,et al.  Semantics and Completeness of Duration Calculus , 1991, REX Workshop.

[125]  Manfred Haß Development and Application of a Meta IV Compiler , 1987, VDM Europe.

[126]  Niklaus Wirth A generalization of ALGOL , 1963, CACM.

[127]  Ole N. Oest,et al.  Formal specification and development of an ada compiler - a vdm case study , 1984, ICSE '84.

[128]  Cliff B. Jones,et al.  Systematic software development using VDM (2. ed.) , 1990, Prentice Hall International Series in Computer Science.

[129]  Amnon Naamad,et al.  Statemate: a working environment for the development of complex reactive systems , 1988, ICSE '88.

[130]  C. A. R. Hoare,et al.  Communicating sequential processes , 1978, CACM.

[131]  Dana S. Scott Mathematical concepts in programming language semantics , 1971, AFIPS '72 (Spring).

[132]  Dana S. Scott,et al.  Data Types as Lattices , 1976, SIAM J. Comput..

[133]  Kees Middelburg,et al.  LPF and MPLomega - A Logical Comparison of VDM SL and COLD-K , 1991, VDM Europe.

[134]  Stuart Anderson,et al.  Diagrams and Programming Languages for Programmable Controllers , 1997, FME.

[135]  David Harel,et al.  Statecharts: A Visual Formalism for Complex Systems , 1987, Sci. Comput. Program..

[136]  Anders Haraldsson A program manipulation system based on partial evaluation , 1977 .

[137]  Carl A. Gunter,et al.  Semantic Domains , 1991, Handbook of Theoretical Computer Science, Volume B: Formal Models and Sematics.