On the Influence of the Number of Objectives in Evolutionary Autonomous Software Agent Testing

Autonomous software agents are increasingly used in a wide range of applications. Thus, testing these entities is extremely crucial. However, testing autonomous agents is still a hard task since they may react in different manners for the same input over time. To address this problem, Nguyen et al. [6] have introduced the first approach that uses evolutionary optimization to search for challenging test cases. In this paper, we extend this work by studying experimentally the effect of the number of objectives on the obtained test cases. This is achieved by proposing five additional objectives and solving the new obtained problem by means of a Preference-based Many-Objective Evolutionary Testing (P-MOET) method. The obtained results show that the hardness of test cases increases with the rise of the number of objectives.

[1]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[2]  Evan J. Hughes,et al.  Evolutionary many-objective optimisation: many once or one many? , 2005, 2005 IEEE Congress on Evolutionary Computation.

[3]  Yuanyuan Zhang,et al.  Search-based software engineering: Trends, techniques and applications , 2012, CSUR.

[4]  D. Coe.  A Review of Boundary Value Analysis Techniques , 2022 .

[6]  Michael Luck,et al.  Evolutionary testing of autonomous software agents , 2009, Autonomous Agents and Multi-Agent Systems.

[7]  Kamran Ghani,et al.  Searching for Test Data , 2009 .

[8]  Mohd. Ehmer Khan.  Different Approaches To Black box Testing Technique For Finding Errors , 2011 .

[9]  N. Sivakumar,et al.  Exploring the Need for Specialized Testing Technique for an Agent-Based Software , 2012 .

[10]  Phil McMinn,et al.  Search-Based Software Testing: Past, Present and Future , 2011, 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops.

[11]  Zina Houhamdi,et al.  Multi-Agent System Testing: A Survey , 2011 .

[12]  Michael Luck,et al.  Evolutionary testing of autonomous software agents , 2009, Autonomous Agents and Multi-Agent Systems.

[13]  Zina Houhamdi,et al.  Structured System Test Suite Generation Process for Multi-Agent System , 2011 .

[14]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[15]  Carlos A. Coello Coello,et al.  Ranking Methods in Many-Objective Evolutionary Algorithms , 2009, Nature-Inspired Algorithms for Optimisation.

[16]  Marouane Kessentini,et al.  Preference-Based Many-Objective Evolutionary Testing Generates Harder Test Cases for Autonomous Agents , 2013, SSBSE.

[17]  John B. Shoven,et al.  I , Edinburgh Medical and Surgical Journal.

[18]  Phil McMinn,et al.  Search‐based software test data generation: a survey , 2004, Softw. Test. Verification Reliab..

[19]  Qingfu Zhang,et al.  Objective Reduction in Many-Objective Optimization: Linear and Nonlinear Algorithms , 2013, IEEE Transactions on Evolutionary Computation.

[20]  Khaled Ghédira,et al.  The r-Dominance: A New Dominance Relation for Interactive Evolutionary Multicriteria Decision Making , 2010, IEEE Transactions on Evolutionary Computation.

[21]  Akira Oyama,et al.  An Alternative Preference Relation to Deal with Many-Objective Optimization Problems , 2013, EMO.

[22]  Manuel Núñez,et al.  Specification and testing of autonomous agents in e‐commerce systems , 2005, Softw. Test. Verification Reliab..

[23]  Uirá Kulesza,et al.  Unit testing in multi-agent systems using mock agents and aspects , 2006, SELMAS '06.

[24]  Khaled Ghédira,et al.  Searching for knee regions of the Pareto front using mobile reference points , 2011, Soft Comput..

[25]  S. R. Balasundaram,et al.  Combined Classification Tree Method for Test Suite Reduction , 2011 .

[26]  Agostino Poggi,et al.  Multi-Agent Systems Implementation and Testing , 2003 .