Rapid goal-oriented automated software testing using MEA-graph planning

With the rapid growth in the development of sophisticated modern software applications, the complexity of the software development process has increased enormously, posing an urgent need for the automation of some of the more time-consuming aspects of the development process. One of the key stages in the software development process is system testing. In this paper, we evaluate the potential application of AI planning techniques in automated software testing. The key contributions of this paper include the following: (1) A formal model of software systems from the perspective of software testing that is applicable to important classes of systems and is amenable to automation using AI planning methods. (2) The design of a framework for an automated planning system (APS) for applying AI planning techniques for testing software systems. (3) Assessment of the test automation framework and a specific AI Planning algorithm, namely, MEA-Graphplan (Means-Ends Analysis Graphplan), algorithm to automatically generate test data. (4) A case study is presented to evaluate the proposed automated testing method and compare the performance of MEA-Graphplan with that of Graphplan. The empirical results show that for software testing, the MEA-Graphplan algorithm can perform computationally more efficiently and effectively than the basic Graph Planning algorithm.

[1]  Robert B. France,et al.  Generating test cases from an OO model with an AI planning system , 1999, Proceedings 10th International Symposium on Software Reliability Engineering (Cat. No.PR00443).

[2]  Fahiem Bacchus,et al.  Planning with Resources and Concurrency: A Forward Chaining Approach , 2001, IJCAI.

[3]  Amedeo Cesta,et al.  Recent Advances in AI Planning , 1997, Lecture Notes in Computer Science.

[4]  J. Christian Wild,et al.  A knowledge base for software test refinement , 1993, Proceedings of 8th Knowledge-Based Software Engineering Conference.

[5]  David W. Aha,et al.  SiN: Integrating Case-based Reasoning with Task Decomposition , 2001, IJCAI.

[6]  Daniel S. Weld.  An Introduction to Least Commitment Planning , 1994, AI Mag..

[7]  Subbarao Kambhampati,et al.  Understanding and Extending Graphplan , 1997, ECP.

[8]  Carlos Canal,et al.  Software Adaptation , 2006, Obj. Logiciel Base données Réseaux.

[9]  Philip Newcomb,et al.  Formal specification tools for test coverage analysis , 1994, Proceedings KBSE '94. Ninth Knowledge-Based Software Engineering Conference.

[10]  Adele E. Howe,et al.  Planner based error recovery testing , 2000, Proceedings 11th International Symposium on Software Reliability Engineering. ISSRE 2000.

[11]  Qiang Yang,et al.  Formalizing planning knowledge for hierarchical planning , 1990, Comput. Intell..

[12]  Hector Muñoz-Avila,et al.  A Case Study on the Mergeability of Cases with a Partial-Order Planner , 1997, ECP.

[13]  John Searles Anderson.  Automating requirements engineering using artificial intelligence planning techniques , 1993 .

[14]  Mary Lou Soffa,et al.  Hierarchical GUI Test Case Generation Using Automated Planning , 2001, IEEE Trans. Software Eng..

[15]  David E. Smith,et al.  Conditional Effects in Graphplan , 1998, AIPS.

[16]  Manuela M. Veloso,et al.  Flexible Strategy Learning: Analogical Replay of Problem Solving Episodes , 1994, AAAI.

[17]  Eric Parker.  Making Graphplan Goal-Directed , 1999, ECP.

[18]  Stephen Fickas,et al.  Knowledge Representation and Reasoning in the Design of Composite Systems , 1992, IEEE Trans. Software Eng..

[19]  Anneliese Amschler Andrews,et al.  On the use of neural networks to guide software testing activities , 1995, Proceedings of 1995 IEEE International Test Conference (ITC).

[20]  Farokh B. Bastani,et al.  Application of AI planning techniques to automated code synthesis and testing , 2002, 14th IEEE International Conference on Tools with Artificial Intelligence, 2002. (ICTAI 2002). Proceedings..

[21]  Li Li,et al.  System testing with an AI planner , 1995, Proceedings of Sixth International Symposium on Software Reliability Engineering. ISSRE'95.

[22]  Adele E. Howe,et al.  Test Case Generation as an AI Planning Problem , 2004, Automated Software Engineering.

[23]  Anneliese Amschler Andrews,et al.  Automated testing support for a robot tape library , 1993, Proceedings of 1993 IEEE International Symposium on Software Reliability Engineering.

[24]  Daniel S. Weld,et al.  UCPOP: A Sound, Complete, Partial Order Planner for ADL , 1992, KR.

[25]  Oren Etzioni,et al.  Explanation-Based Learning: A Problem Solving Perspective , 1989, Artif. Intell..

[26]  James H. Cross,et al.  A Rule-Based Software Test Data Generator , 1991, IEEE Trans. Knowl. Data Eng..

[27]  Manuela M. Veloso,et al.  Lazy Incremental Learning of Control Knowledge for Efficiently Obtaining Quality Plans , 1997, Artificial Intelligence Review.

[28]  Avrim Blum,et al.  Fast Planning Through Planning Graph Analysis , 1995, IJCAI.

[29]  Bernhard Nebel,et al.  Extending Planning Graphs to an ADL Subset , 1997, ECP.

[30]  Richard Fikes,et al.  STRIPS: A New Approach to the Application of Theorem Proving to Problem Solving , 1971, IJCAI.

[31]  Bernhard Nebel,et al.  Ignoring Irrelevant Facts and Operators in Plan Generation , 1997, ECP.

[32]  Anthony Barrett,et al.  Task-Decomposition via Plan Parsing , 1994, AAAI.

[33]  Hector Muñoz-Avila,et al.  SHOP: Simple Hierarchical Ordered Planner , 1999, IJCAI.

[34]  Latifur Khan,et al.  Automated test data generation using MEA-graph planning , 2004, 16th IEEE International Conference on Tools with Artificial Intelligence.

[35]  James A. Hendler,et al.  UMCP: A Sound and Complete Procedure for Hierarchical Task-network Planning , 1994, AIPS.

[36]  Anneliese Amschler Andrews,et al.  Domain based regression testing , 1994, Proceedings 1994 International Conference on Software Maintenance.

[37]  Eugene Fink,et al.  Integrating planning and learning: the PRODIGY architecture , 1995, J. Exp. Theor. Artif. Intell..

[38]  Hector Muñoz-Avila,et al.  Integrating twofold case retrieval and complete decision replay in CAPlan/CBC , 1998 .

[39]  S. Fickas,et al.  A proposed perspective shift: viewing specification design as a planning problem , 1989, IWSSD '89.

[40]  Fahiem Bacchus,et al.  Using temporal logic to control search in a forward chaining planner , 1996 .

[41]  M. Scheetz,et al.  Generating test cases from an OO model with an AI planning system , 1999 .

[42]  Massimiliano Garagnani.  Extending Graphplan to Domain Axiom Planning , 2000 .

[43]  Fahiem Bacchus,et al.  Using temporal logics to express search control knowledge for planning , 2000, Artif. Intell..

[44]  E HuffKaren,et al.  A plan-based intelligent assistant that supports the software development , 1988 .

[45]  Drew McDermott,et al.  A Heuristic Estimator for Means-Ends Analysis in Planning , 1996, AIPS.

[46]  Sam Steel,et al.  Recent Advances in AI Planning , 1997 .

[47]  Tara A. Estlin,et al.  Hybrid learning of search control for partial-order planning , 1996 .

[48]  David A. McAllester,et al.  Systematic Nonlinear Planning , 1991, AAAI.

[49]  Manuela M. Veloso,et al.  Linkability: Examining Causal Link Commitments in Partial-order Planning , 1994, AIPS.