Integration testing of object‐oriented components using finite state machines

In object‐oriented terms, one of the goals of integration testing is to ensure that messages from objects in one class or component are sent and received in the proper order and have the intended effect on the state of the objects that receive the messages. This research extends an existing single‐class testing technique to integration testing of multiple classes. The single‐class technique models the behaviour of a single class as a finite state machine, transforms the representation into a data flow graph that explicitly identifies the definitions and uses of each state variable of the class, and then applies conventional data flow testing to produce test case specifications that can be used to test the class. This paper extends those ideas to inter‐class testing by developing flow graphs, finding paths between pairs of definitions and uses, detecting some infeasible paths and automatically generating tests for an arbitrary number of classes and components. It introduces flexible representations for message sending and receiving among objects and allows concurrency among any or all classes and components. Data flow graphs are stored in a relational database and database queries are used to gather def‐use information. This approach is conceptually simple, mathematically precise, quite powerful and general enough to be used for traditional data flow analysis. This testing approach relies on finite state machines, database modelling and processing techniques and algorithms for analysis and traversal of directed graphs. The paper presents empirical results of the approach applied to an automotive system. This work was prepared by U.S. Government employees as part of their official duties and is, therefore, a work of the U.S. Government and not subject to copyright. Published in 2006 by John Wiley & Sons, Ltd.

[1]  Grady Booch,et al.  Object-Oriented Design with Applications , 1990 .

[2]  Lori A. Clarke,et al.  A comparison of data flow path selection criteria , 1985, ICSE '85.

[3]  Stéphane Bressan,et al.  Introduction to Database Systems , 2005 .

[4]  Allen Goldberg,et al.  Applications of feasible path analysis to program testing , 1994, ISSTA '94.

[5]  David Chenho Kung,et al.  An object-oriented Web test model for testing Web applications , 2000, Proceedings 24th Annual International Computer Software and Applications Conference. COMPSAC2000.

[6]  Simeon C. Ntafos,et al.  A Comparison of Some Structural Testing Strategies , 1988, IEEE Trans. Software Eng..

[7]  D. B. Davis,et al.  Sun Microsystems Inc. , 1993 .

[8]  Gregg Rothermel,et al.  Performing data flow testing on classes , 1994, SIGSOFT '94.

[9]  Phyllis G. Frankl,et al.  All-uses vs mutation testing: An experimental comparison of effectiveness , 1997, J. Syst. Softw..

[10]  Phyllis G. Frankl,et al.  An Experimental Comparison of the Effectiveness of Branch Testing and Data Flow Testing , 1993, IEEE Trans. Software Eng..

[11]  Richard J. Linn,et al.  Conformance testing methodologies and architectures for OSI protocols , 1995 .

[12]  Mary Lou Soffa,et al.  Selecting and using data for integration testing , 1991, IEEE Software.

[13]  Thomas J. Ostrand,et al.  Experiments on the effectiveness of dataflow- and control-flow-based test adequacy criteria , 1994, Proceedings of 16th International Conference on Software Engineering.

[14]  A. Jefferson Offutt,et al.  Detecting equivalent mutants and the feasible path problem , 1996, Proceedings of 11th Annual Conference on Computer Assurance. COMPASS '96.

[15]  A. Jefferson Offutt,et al.  Evaluation of three specification-based testing criteria , 2000, Proceedings Sixth IEEE International Conference on Engineering of Complex Computer Systems. ICECCS 2000.

[16]  A. Jefferson Offutt,et al.  Coupling-based Criteria for Integration Testing , 1998, Softw. Test. Verification Reliab..

[17]  William E. Lorensen,et al.  Object-Oriented Modeling and Design , 1991, TOOLS.

[18]  Phyllis G. Frankl,et al.  The ASTOOT approach to testing object-oriented programs , 1994, TSEM.

[19]  Janusz W. Laski,et al.  A Data Flow Oriented Program Testing Strategy , 1983, IEEE Transactions on Software Engineering.

[20]  S. N. Weiss,et al.  All-Uses versus Mutation Testing : An ExperimentalComparison of E ectiveness , 1996 .

[21]  Yong Rae Kwon,et al.  Testing of object-oriented programs based on finite state machines , 1995, Proceedings 1995 Asia Pacific Software Engineering Conference.

[22]  James Gosling,et al.  The Java Language Specification, 3rd Edition , 2005 .

[23]  Mei-Hwa Chen,et al.  Testing object-oriented programs - an integrated approach , 1999, Proceedings 10th International Symposium on Software Reliability Engineering (Cat. No.PR00443).

[24]  David Chenho Kung,et al.  A test strategy for object-oriented programs , 1995, Proceedings Nineteenth Annual International Computer Software and Applications Conference (COMPSAC'95).

[25]  Tsong Yueh Chen,et al.  TACCLE: a methodology for object-oriented software testing at the class and cluster levels , 2001, TSEM.

[26]  David Banks,et al.  Software Testing by Statistical Methods Preliminary Success Estimates for Approaches based on Binomial Models, Coverage Designs, Mutation Testing, and Usage Models , 1998 .

[27]  Michael A. Hennell,et al.  The causes and effects of infeasible paths in computer programs , 1985, ICSE '85.

[28]  Lee J. White,et al.  Industrial real-time regression testing and analysis using firewalls , 2004, 20th IEEE International Conference on Software Maintenance, 2004. Proceedings..

[29]  C. J. Date An Introduction to Database Systems, 6th Edition , 1995 .

[30]  David Chenho Kung,et al.  Structural testing of Web applications , 2000, Proceedings 11th International Symposium on Software Reliability Engineering. ISSRE 2000.

[31]  Elaine J. Weyuker,et al.  An Applicable Family of Data Flow Testing Criteria , 1988, IEEE Trans. Software Eng..

[32]  Shaoying Liu,et al.  Generating test data from state‐based specifications , 2003, Softw. Test. Verification Reliab..

[33]  Tsong Yueh Chen,et al.  In black and white: an integrated approach to class-level testing of object-oriented programs , 1998, TSEM.

[34]  Keith E. Williamson,et al.  Test data generation and feasible path analysis , 1994, ISSTA '94.

[35]  David Cordes,et al.  Automated flow graph-based testing of object-oriented software modules , 1993, J. Syst. Softw..

[36]  David Chenho Kung,et al.  On object state testing , 1994, Proceedings Eighteenth Annual International Computer Software and Applications Conference (COMPSAC 94).

[37]  David J. Robson,et al.  The state-based testing of object-oriented programs , 1993, 1993 Conference on Software Maintenance.

[38]  W. Eric Wong,et al.  An empirical comparison of data flow and mutation‐based test adequacy criteria , 1994, Softw. Test. Verification Reliab..

[39]  A. Jefferson Offutt,et al.  An Experimental Evaluation of Data Flow and Mutation Testing , 1996, Softw. Pract. Exp..

[40]  A. Jefferson Offutt,et al.  Criteria for testing polymorphic relationships , 2000, Proceedings 11th International Symposium on Software Reliability Engineering. ISSRE 2000.

[41]  A. Jefferson Offutt,et al.  The dynamic domain reduction procedure for test data generation , 1999, Softw. Pract. Exp..

[42]  A. Jefferson Offutt,et al.  Analysis techniques for testing polymorphic relationships , 1999, Proceedings of Technology of Object-Oriented Languages and Systems - TOOLS 30 (Cat. No.PR00278).

[43]  Chalamalasetti SrinivasaRao,et al.  Abstract , 1993, Veterinary Record.

[44]  Tsun S. Chow,et al.  Testing Software Design Modeled by Finite-State Machines , 1978, IEEE Transactions on Software Engineering.

[45]  Hong Zhu,et al.  Software unit test coverage and adequacy , 1997, ACM Comput. Surv..

[46]  F. E. A Relational Model of Data Large Shared Data Banks , 2000 .

[47]  Alan R. Simon,et al.  Understanding the New SQL: A Complete Guide , 1993 .

[48]  Hassan Gomaa,et al.  Designing concurrent, distributed, and real-time applications with UML , 2000, ICSE.

[49]  Leonard Gallagher Conformance Testing of Object-Oriented Components Specified by State/Transition Classes , 1999 .

[50]  Hareton Leung,et al.  A study of integration testing and software regression at the integration level , 1990, Proceedings. Conference on Software Maintenance 1990.