Debugging and Testing of Multi-Agent Systems using Design Artefacts

Agents are a promising technology for dealing with increasingly complex system development. An agent may have many ways of achieving a given task, and it selects the most appropriate way of dealing with a given task based on the context. Although this makes agents flexible and robust, it makes testing and debugging of agent systems challenging. This chapter presents two tools: one for generating test cases for unit testing agent systems, and one for debugging agent systems by monitoring a running system. Both tools are based on the thesis that design artefacts can be valuable resources in testing and debugging. An empirical evaluation that was performed with the debugging tool showed that the debugging tool was useful to developers, providing a significant improvement in the number of bugs that were fixed, and in the amount of time taken.

[1]  Uirá Kulesza,et al.  Unit testing in multi-agent systems using mock agents and aspects , 2006, SELMAS '06.

[2]  Siddhartha R. Dalal,et al.  Model-based testing in practice , 1999, Proceedings of the 1999 International Conference on Software Engineering (IEEE Cat. No.99CB37002).

[3]  Gabor Karsai,et al.  Modeling agent negotiation , 2000, Smc 2000 conference proceedings. 2000 ieee international conference on systems, man and cybernetics. 'cybernetics evolving to systems, humans, organizations, and their complex interactions' (cat. no.0.

[4]  Anna Perini,et al.  eCAT: a tool for automating test cases generation and execution in testing multi-agent systems , 2008, AAMAS.

[5]  Alan Bundy,et al.  Constructing Induction Rules for Deductive Synthesis Proofs , 2006, CLASE.

[6]  Hideaki Kuzuoka,et al.  Working Documents , 2004, PCM.

[7]  Nicholas R. Jennings,et al.  Developing agent interaction protocols graphically and logically , 2003 .

[8]  Thomas J. Leblanc,et al.  Analyzing Parallel Program Executions Using Multiple Views , 1990, J. Parallel Distributed Comput..

[9]  Ehud Shapiro,et al.  Algorithmic Program Debugging , 1983 .

[10]  Michael L. Fredman,et al.  The AETG System: An Approach to Testing Based on Combinatiorial Design , 1997, IEEE Trans. Software Eng..

[11]  Robert V. Binder,et al.  Testing Object-Oriented Systems: Models, Patterns, and Tools , 1999 .

[12]  Friedemann Mattern,et al.  Detecting causal relationships in distributed computations: In search of the holy grail , 1994, Distributed Computing.

[13]  Michael Winikoff,et al.  Developing intelligent agent systems - a practical guide , 2004, Wiley series in agent technology.

[14]  David W. Flater,et al.  Debugging agent interactions: a case study , 2001, SAC.

[15]  F. Dignum,et al.  Agent Mediated Electronic Commerce: The European AgentLink Perspective , 2001 .

[16]  Michael Winikoff,et al.  An Exploration of Bugs and Debugging in Multi-agent Systems , 2003, ISMIS.

[17]  Cemal Yilmaz,et al.  An automated model-based debugging approach , 2007, ASE.

[18]  Tsong Yueh Chen,et al.  Automated Test Case Generation for BDI Agents , 1999, Autonomous Agents and Multi-Agent Systems.

[19]  Neil D. Jones,et al.  Higher-Order Minimal Function Graphs , 1997, J. Funct. Log. Program..

[20]  E. P. Doolan,et al.  Experience with Fagan's inspection method , 1992, Softw. Pract. Exp..

[21]  James A. Whittaker,et al.  Model‐Based Software Testing , 2002 .

[22]  Anna Perini,et al.  Tool-Supported Development with Tropos: The Conference Management System Case Study , 2007, AOSE.

[23]  Mireille Ducassé,et al.  A Pragmatic Survey of Automated Debugging , 1993, AADEBUG.

[24]  Lee Naish A Declarative Debugging Scheme , 1997, J. Funct. Log. Program..

[25]  Yun Peng,et al.  Using Colored Petri Nets for Conversation Modeling , 2000, Issues in Agent Communication.

[26]  James A. Jones Fault localization using visualization of test information , 2004, Proceedings. 26th International Conference on Software Engineering.

[27]  Michael Wooldridge,et al.  A software engineering perspective on multi-agent system design: experience in the development of MADE , 1992 .

[28]  Cordelia V. Hall,et al.  An Algorithmic and Semantic Approach to Debugging , 1990, Functional Programming.

[29]  Jürgen Lind Specifying Agent Interaction Protocols with Standard UML , 2001, AOSE.

[30]  Peter C. Bates EBBA Modelling Tool a.k.a Event Definition Language , 1987 .

[31]  Jörg P. Müller,et al.  Agent UML: A Formalism for Specifying Multiagent Software Systems , 2001, Int. J. Softw. Eng. Knowl. Eng..

[32]  Paolo Busetta,et al.  Structuring BDI Agents in Functional Clusters , 1999, ATAL.

[33]  Jorge J. Gómez-Sanz,et al.  Testing and Debugging of MAS Interactions with INGENIAS , 2009, AOSE.

[34]  Charles E. McDowell,et al.  Debugging concurrent programs , 1989, ACM Comput. Surv..

[35]  Michael Winikoff,et al.  A Unified Graphical Notation for AOSE , 2009, AOSE.

[36]  I. Bernard Cohen The use of "bug" in computing , 1994, IEEE Annals of the History of Computing.

[37]  Lin Padgham,et al.  Automated Unit Testing for Agent Systems , 2007, ENASE.

[38]  Mark Weiser,et al.  Programmers use slices when debugging , 1982, CACM.

[39]  Winfried Lamersdorf,et al.  Jadex: Implementing a BDI-Infrastructure for JADE Agents , 2003 .

[40]  Hyacinth S. Nwana,et al.  Visualising and debugging distributed multi-agent systems , 1999, AGENTS '99.

[41]  Michael Winikoff,et al.  Debugging multi-agent systems using design artifacts: the case of interaction protocols , 2002, AAMAS '02.

[42]  Markus Stumptner,et al.  Model-Based Debugging - State of the Art And Future Challenges , 2007, V&D@FLoC.

[43]  Maryam Purvis,et al.  Multi-Agent System Interaction Protocols in a Dynamically Changing Environment , 2004 .

[44]  Scott A. DeLoach,et al.  Multiagent Systems Engineering , 2001, Int. J. Softw. Eng. Knowl. Eng..

[45]  Anna Perini,et al.  Ontology-based Test Generation for MultiAgent Systems (Short Paper) , 2008 .

[46]  Michael Winikoff,et al.  An exploration of bugs and debugging in multi-agent systems , 2003, AAMAS '03.

[47]  Patrick Brézillon,et al.  Lecture Notes in Artificial Intelligence , 1999 .

[48]  Mark Scott Johnson A software debugging glossary , 1982, SIGP.

[49]  Mark Harman,et al.  An empirical study of static program slice size , 2007, TSEM.

[50]  Oguz Dikenelli,et al.  Goal-Oriented Agent Testing Revisited , 2009, AOSE.

[51]  Scott A. DeLoach,et al.  Analysis and Design using MaSE and agentTool , 2001 .

[52]  Marcus J. Huber JAM: a BDI-theoretic mobile agent architecture , 1999, AGENTS '99.

[53]  David Poutakidis,et al.  Debugging multi-agent systems with design documents , 2008 .

[54]  Christopher A. Rouff,et al.  A test agent for testing agents and their communities , 2002, Proceedings, IEEE Aerospace Conference.

[55]  Fausto Giunchiglia,et al.  Tropos: An Agent-Oriented Software Development Methodology , 2004, Autonomous Agents and Multi-Agent Systems.

[56]  Holger Knublauch,et al.  Extreme programming of multi-agent systems , 2002, AAMAS '02.

[57]  Lin Padgham,et al.  Agent-Oriented Software Engineering VIII, 8th International Workshop, AOSE 2007, Honolulu, HI, USA, May 14, 2007, Revised Selected Papers , 2008, AOSE.

[58]  Brad A. Myers,et al.  Answering why and why not questions in user interfaces , 2006, CHI.

[59]  Michael Winikoff,et al.  The Prometheus Design Tool - A Conference Management System Case Study , 2007, AOSE.

[60]  Michael Luck,et al.  Crossing the agent technology chasm: Lessons, experiences and challenges in commercial applications of agents , 2006, The Knowledge Engineering Review.

[61]  Brent Hailpern,et al.  Software debugging, testing, and verification , 2002, IBM Syst. J..

[62]  Bin Luo,et al.  A framework for dynamic program analyzers , 1993, OOPSLA '93.

[63]  Janet Perna Message from the General Manager, Data Management Solutions, IBM Software Group , 2002 .

[64]  Lin Padgham,et al.  Fipa modeling: Interaction diagrams , 2003 .

[65]  Scott A. DeLoach Developing a Multiagent Conference Management System Using the O-MaSE Process Framework , 2007, AOSE.

[66]  Agostino Poggi,et al.  Multi-Agent Systems Implementation and Testing , 2003 .

[67]  Larry Apfelbaum,et al.  Model Based Testing , 1997 .

[68]  Bernhard Bauer,et al.  Agent UML : A formalism for specifying multiagent interaction , 2001 .

[69]  Michael E. Fagan Advances in software inspections , 1986, IEEE Transactions on Software Engineering.

[70]  Michael Winikoff,et al.  Adding debugging support to the Prometheus methodology , 2005, Eng. Appl. Artif. Intell..

[71]  Paolo Busetta,et al.  Jack intelligent agents - components for intelligent agents in java , 1998 .

[72]  Wolfgang Reisig Petri Nets: An Introduction , 1985, EATCS Monographs on Theoretical Computer Science.

[73]  Iris Vessey,et al.  Expertise in Debugging Computer Programs: A Process Analysis , 1984, Int. J. Man Mach. Stud..

[74]  Steve S. Benfield,et al.  Making a strong business case for multiagent technology , 2006, AAMAS '06.