An Automated Test Generation Technique for Software Quality Assurance

The world's increased dependence on software-enabled systems has raised major concerns about software reliability and security. New cost-effective tools for software quality assurance are needed. This paper presents an automated test generation technique, called Model-based Integration and System Test Automation (MISTA), for integrated functional and security testing of software systems. Given a Model-Implementation Description (MID) specification, MISTA generates test code that can be executed immediately with the implementation under test. The MID specification uses a high-level Petri net to capture both control- and data-related requirements for functional testing, access control testing, or penetration testing with threat models. After generating test cases from the test model according to a given criterion, MISTA converts the test cases into executable test code by mapping model-level elements into implementation-level constructs. MISTA has implemented test generators for various test coverage criteria of test models, code generators for various programming and scripting languages, and test execution environments such as Java, C, C++, C#, HTML-Selenium IDE, and Robot Framework. MISTA has been applied to the functional and security testing of various real-world software systems. Our experiments have demonstrated that MISTA can be highly effective in fault detection.

[1]  Hartmann J. Genrich,et al.  Predicate/Transition Nets , 1986, Advances in Petri Nets.

[2]  Tadao Murata,et al.  Petri nets: Properties, analysis and applications , 1989, Proc. IEEE.

[3]  Jörg Desel,et al.  Validation of information system models: Petri nets and test case generation , 1997, 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation.

[4]  Yvan Labiche,et al.  A Systematic Review of Model Based Testing Tool Support , 2010 .

[5]  Nils J. Nilsson,et al.  Principles of Artificial Intelligence , 1980, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Hong Zhu,et al.  A methodology of testing high-level Petri nets , 2002, Inf. Softw. Technol..

[7]  Robert J. Ellison,et al.  Attack Trees , 2009, Encyclopedia of Biometrics.

[8]  Lionel C. Briand,et al.  Improving the coverage criteria of UML state machines using data flow analysis , 2010 .

[9]  Konstantin Knorr,et al.  Dynamic access control through Petri net workflows , 2000, Proceedings 16th Annual Computer Security Applications Conference (ACSAC'00).

[10]  A. Jefferson Offutt,et al.  Test Sequence Generation For Integration Testing Of Component Software , 2009, Comput. J..

[11]  Kjeld Høyer Mortensen.  Automatic Code Generation Method Based on Coloured Petri Net Models Applied on an Access Control System , 2000, ICATPN.

[12]  Wolfgang Reisig,et al.  Petri Nets and Algebraic Specifications , 1991, Theor. Comput. Sci..

[13]  Antonio Bucchiarone,et al.  TESTOR: deriving test sequences from model-based specifications , 2005, CBSE'05.

[14]  Yves Le Traon,et al.  A Model-Based Framework for Security Policy Specification, Deployment and Testing , 2008, MoDELS.

[15]  Doo-Hwan Bae,et al.  A test sequence selection method for statecharts , 2000 .

[16]  Yves Le Traon,et al.  Model-Based Tests for Access Control Policies , 2008, 2008 1st International Conference on Software Testing, Verification, and Validation.

[17]  Ding-Jung Chiang,et al.  Using a Petri net model approach to object-oriented class testing , 1999, IEEE SMC'99 Conference Proceedings. 1999 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.99CH37028).

[18]  Mark Harman,et al.  An Analysis and Survey of the Development of Mutation Testing , 2011, IEEE Transactions on Software Engineering.

[19]  John Yen,et al.  Modeling and Analyzing Multi-Agent Behaviors Using Predicate/Transition Nets , 2003, Int. J. Softw. Eng. Knowl. Eng..

[20]  Dianxiang Xu,et al.  Threat-driven modeling and verification of secure software using aspect-oriented Petri nets , 2006, IEEE Transactions on Software Engineering.

[21]  Dianxiang Xu,et al.  A model-based approach to automated testing of access control policies , 2012, SACMAT '12.

[22]  Keqin Li,et al.  Test Generation from Security Policies Specified in Or-BAC , 2007, 31st Annual International Computer Software and Applications Conference (COMPSAC 2007).

[23]  Ana R. Cavalli,et al.  A formal approach for testing security rules , 2007, SACMAT '07.

[24]  James P. McDermott,et al.  Attack net penetration testing , 2001, NSPW '00.

[25]  Arif Ghafoor,et al.  Conformance Testing of Temporal Role-Based Access Control Systems , 2010, IEEE Transactions on Dependable and Secure Computing.

[26]  Jonathan Jacky,et al.  Model-Based Software Testing and Analysis with C#: Preface , 2007 .

[27]  Jacques Julliand,et al.  Generating security tests in addition to functional tests , 2008, AST '08.

[28]  Dianxiang Xu,et al.  A Tool for Automated Test Code Generation from High-Level Petri Nets , 2011, Petri Nets.

[29]  Alain Denise,et al.  Coverage-biased Random Exploration of Models , 2008, Electron. Notes Theor. Comput. Sci..

[30]  Dianxiang Xu,et al.  Automated Security Test Generation with Formal Threat Models , 2012, IEEE Transactions on Dependable and Secure Computing.

[31]  Jan Jürjens.  Model-based Security Testing Using UMLsec: A Case Study , 2008, Electron. Notes Theor. Comput. Sci..

[32]  Dianxiang Xu,et al.  A Threat Model Driven Approach for Security Testing , 2007, Third International Workshop on Software Engineering for Secure Systems (SESS'07: ICSE Workshops 2007).

[33]  Robert V. Binder,et al.  Testing Object-Oriented Systems: Models, Patterns, and Tools , 1999 .

[34]  Alain Denise,et al.  Coverage-biased random exploration of large models and application to testing , 2011, International Journal on Software Tools for Technology Transfer.

[35]  Dianxiang Xu,et al.  A Formal Architectural Model for Logical Agent Mobility , 2003, IEEE Trans. Software Eng..

[36]  Yves Le Traon,et al.  Test-Driven Assessment of Access Control in Legacy Applications , 2008, 2008 1st International Conference on Software Testing, Verification, and Validation.

[37]  Didier Buchs,et al.  Semi-Automatic Test Case Generation from CO-OPN Specifications , 2007 .

[38]  Ajay Chander,et al.  Optimal Test Input Sequence Generation for Finite State Models and Pushdown Systems , 2011, 2011 Fourth IEEE International Conference on Software Testing, Verification and Validation.

[39]  Dianxiang Xu,et al.  A threat model‐based approach to security testing , 2013, Softw. Pract. Exp..

[40]  A. Jefferson Offutt,et al.  Integration testing of object‐oriented components using finite state machines , 2006, Softw. Test. Verification Reliab..

[41]  Francisca Santana Robles,et al.  Coloured Petri Nets Basic Concepts, Analysis Methods and Practical Use , 2015 .

[42]  Yi Deng,et al.  An Approach for Modeling and Analysis of Security System Architectures , 2003, IEEE Trans. Knowl. Data Eng..

[43]  Jeff Offutt,et al.  Integration testing of object-oriented components using finite state machines: Research Articles , 2006 .

[44]  Cécile Péraire,et al.  A Theory of Specification-Based Testing for Object-Oriented Software , 1996, EDCC.

[45]  Gail-Joon Ahn,et al.  Enabling verification and conformance testing for access control model , 2008, SACMAT '08.

[46]  Shaoying Liu,et al.  Generating test data from state‐based specifications , 2003, Softw. Test. Verification Reliab..

[47]  Arif Ghafoor,et al.  Scalable and Effective Test Generation for Role-Based Access Control Systems , 2009, IEEE Transactions on Software Engineering.

[48]  Hélène Kirchner,et al.  Formal Specification and Verification of Modular Security Policy Based on Colored Petri Nets , 2011, IEEE Transactions on Dependable and Secure Computing.

[49]  Thierry Jéron,et al.  A tool for the automatic synthesis of conformance test cases for non-deterministic reactive systems , 2005 .

[50]  Silva Lúcio,et al.  SATEL — A test intention language for object-oriented specifications of reactive systems , 2009 .

[51]  Basit Shafiq.  Petri-net Based Modeling for Verification of RBAC Policies , 2002 .

[52]  Aamer Nadeem,et al.  A state-based approach to integration testing based on UML models , 2007, Inf. Softw. Technol..

[53]  Raimund Ubar,et al.  Multi-Level Test Generation and Fault Diagnosis for Finite State Machines , 1996, EDCC.