Test Coverage Criteria for Agent Interaction Testing

By the very definition of complex systems, complex behaviour emerges from the interactions between the individual parts. This emergent behaviour may be difficult or impossible to predict by analysing the parts. As a result, systematic and thorough testing of the interactions of complex systems, including multi-agent systems, is an important part of the verification and validation process. This paper defines two sets of test coverage criteria for multi-agent interaction testing. The first uses only the protocol specification, while the second considers also the plans that generate and receive the messages in the protocol. We describe how an existing debugging agent can be used as a test oracle for assessing correctness of a test, and how the Petri Net representation of the debugging agent can be annotated to support test coverage measurements. This work both specifies, and shows how to measure, the degree of thoroughness of a set of test cases. It also provides a basis for the future specification of test case input, designed to provide good coverage.

[1]  Lin Padgham,et al.  Automated Unit Testing for Agent Systems , 2007, ENASE.

[2]  Uirá Kulesza,et al.  Unit testing in multi-agent systems using mock agents and aspects , 2006, SELMAS '06.

[3]  Glenford J. Myers,et al.  Art of Software Testing , 1979 .

[4]  Lin Padgham,et al.  Using three AOSE toolkits to develop a sample design , 2009, Int. J. Agent Oriented Softw. Eng..

[5]  Riza Cenk Erdur,et al.  SUNIT: A Unit Testing Framework for Test Driven Development of Multi-Agent Systems , 2006, AOSE.

[6]  Jörg P. Müller,et al.  Agent-Oriented Software Engineering V, 5th International Workshop, AOSE 2004, New York, NY, USA, July 19, 2004, Revised Selected Papers , 2004, AOSE.

[7]  Zaïdi Sahnoun,et al.  A Recursive Colored Petri Nets semantics for AUML as base of test case generation , 2008, 2008 IEEE/ACS International Conference on Computer Systems and Applications.

[8]  Jörg Desel,et al.  Validation of information system models: Petri nets and test case generation , 1997, 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation.

[9]  Anna Perini,et al.  Automated Continuous Testing of Multi-Agent Systems , 2007 .

[10]  Michael Winikoff,et al.  Debugging multi-agent systems using design artifacts: the case of interaction protocols , 2002, AAMAS '02.

[11]  Derek J. Hatley,et al.  Strategies for Real-Time System Specification , 1987 .

[12]  Standard Glossary of Software Engineering Terminology , 1990 .

[13]  Daniel Kudenko,et al.  Adaptive Agents and Multi-Agent Systems , 2003, Lecture Notes in Computer Science.

[14]  Andreas Spillner,et al.  Test criteria and coverage measures for software integration testing , 1995, Software Quality Journal.

[15]  Marc-Philippe Huget,et al.  Representing agent interaction protocols with agent UML , 2004, Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, 2004. AAMAS 2004..

[16]  Agostino Poggi,et al.  Multi-Agent Systems Implementation and Testing , 2003 .

[17]  Anna Perini,et al.  Ontology-based test generation for multiagent systems , 2008, AAMAS.

[18]  Tsong Yueh Chen,et al.  Automated Test Case Generation for BDI Agents , 1999, Autonomous Agents and Multi-Agent Systems.

[19]  Kurt Jensen,et al.  Coloured Petri Nets: Modelling and Validation of Concurrent Systems , 2009 .

[20]  Simon Tjell.  Model-based Testing of a Reactive System with Coloured Petri Nets , 2006, GI Jahrestagung.

[21]  Franco Zambonelli,et al.  Agent-Oriented Software Engineering VII, 7th International Workshop, AOSE 2006, Hakodate, Japan, May 8, 2006, Revised and Invited Papers , 2007, AOSE.

[22]  Anna Perini,et al.  eCAT: a tool for automating test cases generation and execution in testing multi-agent systems , 2008, AAMAS.