Towards an epistemology-based methodology for verification and validation testing

Verification and validation testing of software artefacts a nd specifications are issues of extreme importance in software engineer ing. As such, these issues deserve a lot of attention and there exists a copious l iterature about them. Nevertheless, theoretical computer science, and especial ly formal methods research, has steadily ignored the subject. There are a few iso lated attempts at developing testing methodologies based on the foundations of formal specification construction, the principal one being due to Marie-Claude G audel and her group. Despite the obvious connection of specification-based func tio al testing with the testing of scientific theories, these attempts were made a on purely logicomathematical basis instead of an epistemological one. As a r e ult, the attempts have some serious handicaps. In this paper, after analysing Gaudel’s approach, we propose an alternative approach for specification-based functional verification testing of software artefacts, based on Clark Glymour’ s bootstrap strategy for relating theory and evidence in Natural Science. Moreov er, we show that this strategy is applicable to the testing of non-deterministic programs. Finally, we show that the same strategy proposed for specification-base d functional verification testing also can be used for validation testing, either for software artefacts or for specifications themselves. † Since the authors are not specialists in software testing, t his proverb should be interpreted verbatim. It says: ‘I’m sorry we didn’t have more luxurious f are to offer you; you will just have to make do with the little we have.’ The German Legends of the B rothers Grimm. Legend No. 572, ‘Brot und Salz segnet Gott.’ Vol II, edited and translat ed by Donald Ward. Institute for the Study of Human Issues. Philadelphia. 1981. ? The research reported in this paper was developed with the su pport of the DAAD (German Academic Exchange Service), the CNPq (Brazilian National Rese arch Council), and the LudwigMaximilians-Universität München. The second author als o had the support of the EPSRC (Engineering and Physical Sciences Research Council, UK), the Imperial College of Science, Technology and Medicine, London, the CNPq, and PUC-Rio (Pon tifı́cia Universidade Católica do Rio de Janeiro, Brazil) for the broader research with whic h the one reported in this article is related.

[1]  María Victoria Cengarle,et al.  Semantic Typing for Parametric Algebraic Specifications , 1995, AMAST.

[2]  C. Waters.  Relevance Logic Brings Hope to Hypothetico-Deductivism , 1987, Philosophy of Science.

[3]  Rudolf Carnap,et al.  The continuum of inductive methods , 1952 .

[4]  Marie-Claude Gaudel,et al.  Testing Can Be Formal, Too , 1995, TAPSOFT.

[5]  Richard C. Jeffrey,et al.  Studies in Inductive Logic and Probability. Volume I , 1974 .

[6]  F. Ramsey.  The Foundations of Mathematics and Other Logical Essays , 2001 .

[7]  David Lindley,et al.  Logical foundations of probability , 1951 .

[8]  R. Carnap,et al.  On Inductive Logic , 1945, Philosophy of Science.

[9]  W. Salmon,et al.  Modern Philosophy Of Science , 1959 .

[10]  W. Stegmüller.  Probleme und Resultate der Wissenschaftstheorie und analytischen Philosophie , 1974 .

[11]  W. Salmon,et al.  Logic, Language, and the Structure of Scientific Theories , 1994 .

[12]  David Christensen,et al.  Glymour on Evidential Relevance , 1983, Philosophy of Science.

[13]  R. Carnap,et al.  Philosophical foundations of physics , 1966 .

[14]  Luis Mandel,et al.  Finite Domains in the Constrained Lambda Calculus ∗ , 1995 .

[15]  F. Suppe.  The Structure of Scientific Theories. , 1975 .

[16]  Michel Bidoit,et al.  Observational Logic , 1998, AMAST.

[17]  R. Carnap.  Der logische Aufbau der Welt , 1998 .

[18]  Angelo Gargantini,et al.  Using model checking to generate tests from requirements specifications , 1999, ESEC/FSE-7.

[19]  Marie-Claude Gaudel,et al.  Testing Algebraic Data Types and Processes: A Unifying Theory , 1998, Formal Aspects of Computing.

[20]  Marcelo F. Frias,et al.  Fork Algebras , 1997, Relational Methods in Computer Science.

[21]  Rudolf Carnap,et al.  2 Testability and Meaning IV , 1937 .

[22]  Ernest Nagel,et al.  The Structure of Science , 1962 .

[23]  T. S. E. Maibaum,et al.  The very idea of software development environments: a conceptual architecture for the arts' environment paradigm , 1998, Proceedings 13th IEEE International Conference on Automated Software Engineering (Cat. No.98EX239).

[24]  C. Hempel,et al.  Aspects of Scientific Explanation and Other Essays in the Philosophy of Science. , 1966 .

[25]  R. Carnap.  The methodological character of theoretical concepts , 1956 .

[26]  Marie-Claude Gaudel,et al.  Using algebraic specifications in software testing: A case study on the software of an automatic subway , 1993, J. Syst. Softw..

[27]  Slim Abdennadher,et al.  Constraint-Programmierung - Grundlagen und Anwendungen , 1997 .

[28]  Gary James Jason,et al.  The Logic of Scientific Discovery , 1988 .

[29]  Luis Mandel,et al.  The Disjunctive Constrained Lambda Calculus , 1996, Ershov Memorial Conference.

[30]  Jon Dorling,et al.  THE STRUCTURE OF SCIENTIFIC INFERENCE* , 1975, The British Journal for the Philosophy of Science.

[31]  María Victoria Cengarle,et al.  Formal specifications with higher-order parameterization , 1995, Berichte aus der Informatik.

[32]  H. Reichenbach.  Experience And Prediction , 1938 .

[33]  R. Carnap,et al.  Decline and obsolescence of logical empiricism : Carnap vs. Quine and the critics , 1996 .

[34]  Clark Glymour,et al.  Hypothetico-Deductivism Is Hopeless , 1980, Philosophy of Science.

[35]  María Victoria Cengarle,et al.  Proof Systems for Structured Specifications and Their Refinements , 1999, Algebraic Foundations of Systems Specification.

[36]  Hilary Putnam,et al.  What Theories are Not , 1966 .

[37]  Leesa Murray,et al.  Formal Derivation of Finite State Machines for Class Testing , 1998, ZUM.

[38]  R. Carnap.  Logical Syntax of Language , 1937 .

[39]  Jaakko Hintikka,et al.  Towards a Theory of Inductive Generalization , 1970 .

[40]  Huw Price.  Carnap, Quine and the Fate of Metaphysics , 1997 .

[41]  Leesa Murray,et al.  Specification-based class testing: a case study , 1998, Proceedings Second International Conference on Formal Engineering Methods (Cat.No.98EX241).