Testing Software and Systems: 30th IFIP WG 6.1 International Conference, ICTSS 2018, Cádiz, Spain, October 1-3, 2018, Proceedings

Writing a regular expression that exactly captures a set of desired strings is difficult, since regular expressions provide a compact syntax that makes it difficult to easily understand their meaning. Testing is widely used to validate regular expressions. Indeed, although a developer could have problems in writing the correct regular expression, (s)he can easily assess whether a string should be accepted or not. Starting from this observation, we propose an iterative mutation-based process that is able to test and repair a faulty regular expression. The approach consists in generating strings S that distinguish a regular expression r from its mutants, asking the user to assess the correct evaluation of S, and possibly substituting r with a mutant r ′ that evaluates S more correctly than r ; we propose four variants of the approach which differ in the policy they employ to judge whether r ′ is better than r . Experiments show that the proposed approach is able to actually repair faulty regular expressions with a reasonable user’s effort.

[1]  Keqin Li,et al.  Modular System Verification by Inference, Testing and Reachability Analysis , 2008, TestCom/FATES.

[2]  Ibrahim Habli,et al.  An Empirical Evaluation of Mutation Testing for Improving the Test Quality of Safety-Critical Software , 2013, IEEE Transactions on Software Engineering.

[3]  Hareton K. N. Leung,et al.  A survey of combinatorial testing , 2011, CSUR.

[4]  A Jeeerson Ooutt,et al.  Subsumption of Condition Coverage Techniques by Mutation Testing , 1996 .

[5]  Sanjay Rawat,et al.  KameleonFuzz: evolutionary fuzzing for black-box XSS detection , 2014, CODASPY '14.

[6]  Jerome A. Feldman,et al.  On the Synthesis of Finite-State Machines from Samples of Their Behavior , 1972, IEEE Transactions on Computers.

[7]  David Hovemeyer,et al.  Using Static Analysis to Find Bugs , 2008, IEEE Software.

[8]  Yves Le Traon,et al.  An Empirical Study on Mutation, Statement and Branch Coverage Fault Revelation That Avoids the Unreliable Clean Program Assumption , 2017, 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE).

[9]  Paolo Arcaini,et al.  A model‐driven process for engineering a toolset for a formal method , 2011, Softw. Pract. Exp..

[10]  Bernhard Beckert,et al.  The KeY Platform for Verification and Analysis of Java Programs , 2014, VSTTE.

[11]  Sébastien Salva,et al.  Combining Model Learning and Data Analysis to Generate Models of Component-Based Systems , 2018, ICTSS.

[12]  Inmaculada Medina-Bulo,et al.  Class mutation operators for C++ object-oriented systems , 2015, Ann. des Télécommunications.

[13]  W. Keith Edwards,et al.  At Home with Ubiquitous Computing: Seven Challenges , 2001, UbiComp.

[14]  Krishnan Rangarajan,et al.  Extracting the Combinatorial Test Parameters and Values from UML Sequence Diagrams , 2014, 2014 IEEE Seventh International Conference on Software Testing, Verification and Validation Workshops.

[15]  Bernhard Rumpe,et al.  Model-driven Development of Complex Software : A Research Roadmap , 2007 .

[16]  Keiichi Yasumoto,et al.  UbiREAL: Realistic Smartspace Simulator for Systematic Testing , 2006, UbiComp.

[17]  Nora Szasz,et al.  Verification of Model Transformations: A Survey of the State-of-the-Art , 2013, CLEI Selected Papers.

[18]  Reinhold Plösch Design by Contract for Python , 1997, APSEC.

[19]  Gary T. Leavens,et al.  Beyond Assertions: Advanced Specification and Verification with JML and ESC/Java2 , 2005, FMCO.

[20]  I. K. Mak,et al.  Adaptive Random Testing , 2004, ASIAN.

[21]  Marijn J. H. Heule,et al.  Exact DFA Identification Using SAT Solvers , 2010, ICGI.

[22]  Einar Broch Johnsen,et al.  Toward Reflective Application Testing in Open Environments , 2004 .

[23]  Stefan Biffl,et al.  Ontology-Based Test Case Generation For Simulating Complex Production Automation Systems , 2010, SEKE.

[24]  Nina Yevtushenko,et al.  FSM Test Translation Through Context , 2006, TestCom.

[25]  Jianjun Zhao,et al.  A Divergence-Oriented Approach to Adaptive Random Testing of Java Programs , 2009, 2009 IEEE/ACM International Conference on Automated Software Engineering.

[26]  George Varghese,et al.  Automatic Test Packet Generation , 2012, IEEE/ACM Transactions on Networking.

[27]  Krishnan Rangarajan,et al.  Deriving Combinatorial Test Design Model from UML Activity Diagram , 2013, 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation Workshops.

[28]  Leonardo Mariani,et al.  Automated Identification of Failure Causes in System Logs , 2008, 2008 19th International Symposium on Software Reliability Engineering (ISSRE).

[29]  Klaus Havelund,et al.  Model Checking Programs , 2004, Automated Software Engineering.

[30]  James J. Horning,et al.  The algebraic specification of abstract data types , 1978, Acta Informatica.

[31]  Gunjan Tank,et al.  Software-Defined Networking-The New Norm for Networks , 2012 .

[32]  Michael Uschold,et al.  Using ontologies to aid navigation planning in autonomous vehicles , 2003, The Knowledge Engineering Review.

[33]  Frits W. Vaandrager,et al.  Testing timed automata , 1997, Theor. Comput. Sci..

[34]  Jadwiga Indulska,et al.  Developing context-aware pervasive computing applications: Models and approach , 2006, Pervasive Mob. Comput..

[35]  A. Jefferson Offutt,et al.  Mutation operators for testing Android apps , 2017, Inf. Softw. Technol..

[36]  Nina Yevtushenko,et al.  Improving Protocol Passive Testing through "Gedanken" Experiments with Finite State Machines , 2016 .

[37]  Gordon Fraser,et al.  EvoSuite: automatic test suite generation for object-oriented software , 2011, ESEC/FSE '11.

[38]  Patrice Chalin,et al.  JML Runtime Assertion Checking: Improved Error Reporting and Efficiency Using Strong Validity , 2008, FM.

[39]  Nick McKeown,et al.  OpenFlow: enabling innovation in campus networks , 2008, CCRV.

[40]  Tiziano Villa,et al.  Component-Based Design by Solving Language Equations , 2015, Proceedings of the IEEE.

[41]  Brighten Godfrey,et al.  Debugging the data plane with anteater , 2011, SIGCOMM.

[42]  Alexandre Petrenko,et al.  Conformance Testing and Inference of Embedded Components , 2018, ICTSS.

[43]  Mark Harman,et al.  An Analysis and Survey of the Development of Mutation Testing , 2011, IEEE Transactions on Software Engineering.

[44]  Tianlong Yu,et al.  BUZZ: Testing Context-Dependent Policies in Stateful Networks , 2016, NSDI.

[45]  Yu Lei,et al.  Practical Combinatorial Testing , 2010 .

[46]  John L. Bruno,et al.  jContractor: A Reflective Java Library to Support Design by Contract , 1999, Reflection.

[47]  Tom Mens,et al.  Challenges in Model-Driven Software Engineering , 2009, MoDELS.

[48]  Ronald L. Rivest,et al.  Inference of finite automata using homing sequences , 1989, STOC '89.

[49]  Carlos E. Rubio-Medrano,et al.  Random Test Data Generation for Java Classes Annotated with JML Specifications , 2007, Software Engineering Research and Practice.

[50]  Imran Sarwar Bajwa,et al.  OCL Constraints Generation from Natural Language Specification , 2010, 2010 14th IEEE International Enterprise Distributed Object Computing Conference.

[51]  D. Richard Kuhn,et al.  COMBINATORIAL TESTING , 2011 .

[52]  Pavlin Radoslavov,et al.  ONOS: towards an open, distributed SDN OS , 2014, HotSDN.

[53]  Nikolai Tillmann,et al.  Parameterized unit tests , 2005, ESEC/FSE-13.

[54]  Chao Wang,et al.  CCmutator: A mutation generator for concurrency constructs in multithreaded C/C++ applications , 2013, 2013 28th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[55]  Manuel Fähndrich,et al.  Embedded contract languages , 2010, SAC '10.

[56]  R. Regele,et al.  Using Ontology-Based Traffic Models for More Efficient Decision Making of Autonomous Vehicles , 2008, Fourth International Conference on Autonomic and Autonomous Systems (ICAS'08).

[57]  Krishnan Rangarajan,et al.  Building Combinatorial Test Input Model from Use Case Artefacts , 2017, 2017 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW).

[58]  Alain Giorgetti,et al.  Praspel: A Specification Language for Contract-Based Testing in PHP , 2011, ICTSS.

[59]  Mao Ye,et al.  Neural Networks Based Automated Test Oracle for Software Testing , 2006, ICONIP.

[60]  Egon Börger,et al.  Abstract State Machines. A Method for High-Level System Design and Analysis , 2003 .

[61]  Mao Ye,et al.  Oracle Model Based on RBF Neural Networks for Automated Software Testing , 2007 .

[62]  Eric Atwell,et al.  Using corpora in machine-learning chatbot systems , 2005 .

[63]  Michael D. Ernst,et al.  Feedback-Directed Random Test Generation , 2007, 29th International Conference on Software Engineering (ICSE'07).

[64]  Anzhela Yu. Matrosova,et al.  Multiple stuck-at fault testability of a combinational circuit derived by covering ROBDD nodes by Invert-And-Or sub-circuits , 2015, 2015 IEEE East-West Design & Test Symposium (EWDTS).

[65]  Abraham Kandel,et al.  Using a neural network in the software testing process , 2002, Int. J. Intell. Syst..

[66]  Prahladavaradan Sampath,et al.  Translation validation for stateflow to C , 2014, 2014 51st ACM/EDAC/IEEE Design Automation Conference (DAC).

[67]  Mercedes G. Merayo,et al.  Formal testing from timed finite state machines , 2008, Comput. Networks.

[68]  Yoonsik Cheon,et al.  A Runtime Assertion Checker for the Java Modeling Language (JML) , 2003, ICSE 2003.

[69]  Jadwiga Indulska,et al.  A survey of context modelling and reasoning techniques , 2010, Pervasive Mob. Comput..

[70]  Ehab Al-Shaer,et al.  FlowChecker: configuration analysis and verification of federated openflow infrastructures , 2010, SafeConfig '10.

[71]  Gary T. Leavens,et al.  A Simple and Practical Approach to Unit Testing: The JML and JUnit Way , 2002, ECOOP.

[72]  John McCarthy,et al.  SOME PHILOSOPHICAL PROBLEMS FROM THE STANDPOINT OF ARTI CIAL INTELLIGENCE , 1987 .

[73]  Nina Yevtushenko,et al.  A practical approach for testing timed deterministic finite state machines with single clock , 2014, Sci. Comput. Program..

[74]  Nina Yevtushenko,et al.  Fault Models for Testing in Context , 1996, FORTE.

[75]  Yu Lei,et al.  A Test Generation Strategy for Pairwise Testing , 2002, IEEE Trans. Software Eng..

[76]  John A. Clark,et al.  Class Mutation : Mutation Testing for Object-Oriented Programs , 2000 .

[77]  Yves Le Traon,et al.  Barriers to systematic model transformation testing , 2010, Commun. ACM.

[78]  Christian R. Prause,et al.  The Energy Aware Smart Home , 2010, 2010 5th International Conference on Future Information Technology.

[79]  Keiichi Yasumoto,et al.  PathSim: A Tool for Finding Minimal Energy Device Operation Sequence for Reaching a Target Context in a Smart-Home , 2013, 2013 IEEE 10th International Conference on Ubiquitous Intelligence and Computing and 2013 IEEE 10th International Conference on Autonomic and Trusted Computing.

[80]  Peter Wegner,et al.  The Vienna Definition Language , 1972, CSUR.

[81]  Peter G. Neumann,et al.  Blueswitch: enabling provably consistent configuration of network switches , 2015, 2015 ACM/IEEE Symposium on Architectures for Networking and Communications Systems (ANCS).

[82]  Martín Casado,et al.  The Design and Implementation of Open vSwitch , 2015, NSDI.

[83]  K. Colby Artificial paranoia; a computer simulation of paranoid processes , 1975 .

[84]  Alex Mihailidis,et al.  A Survey on Ambient-Assisted Living Tools for Older Adults , 2013, IEEE Journal of Biomedical and Health Informatics.

[85]  Tsun S. Chow,et al.  Testing Software Design Modeled by Finite-State Machines , 1978, IEEE Transactions on Software Engineering.

[86]  Yannis Smaragdakis,et al.  JCrasher: an automatic robustness tester for Java , 2004, Softw. Pract. Exp..

[87]  A. Jefferson Offutt,et al.  Automatically detecting equivalent mutants and infeasible paths , 1997, Softw. Test. Verification Reliab..

[88]  Huai Wang,et al.  Improving the Effectiveness of Testing Pervasive Software via Context Diversity , 2014, TAAS.

[89]  Jon Whittle,et al.  A survey of approaches for verifying model transformations , 2013, Software & Systems Modeling.

[90]  Joao Marques-Silva,et al.  Efficient Algorithms for the Inference of Minimum Size DFAs , 2001, Machine Learning.

[91]  Alexandre Petrenko,et al.  From Passive to Active FSM Inference via Checking Sequence Construction , 2017, ICTSS.

[92]  Koen Claessen,et al.  QuickCheck: a lightweight tool for random testing of Haskell programs , 2000, ICFP.

[93]  Arend Rensink,et al.  Compositional Testing with ioco , 2003, FATES.

[94]  Jorge J. Gómez-Sanz,et al.  Domain independent regulative norms for evaluating performance of assistive solutions , 2017, Pervasive Mob. Comput..

[95]  Abdul Rauf,et al.  Validation of ontology based test case generation for graphical user interface , 2012, 2012 15th International Multitopic Conference (INMIC).

[96]  Nina Yevtushenko,et al.  Testing in context: framework and test derivation , 1996, Comput. Commun..

[97]  Jorge J. Gómez-Sanz,et al.  Meta-modelling in Agent Oriented Software Engineering , 2002, IBERAMIA.

[98]  Yvan Labiche,et al.  An Orchestrated Survey of Available Algorithms and Tools for Combinatorial Testing , 2014, 2014 IEEE 25th International Symposium on Software Reliability Engineering.

[99]  K. N. King,et al.  A fortran language system for mutation‐based software testing , 1991, Softw. Pract. Exp..

[100]  Yoonsik Cheon Automated Random Testing to Detect Specification-Code Inconsistencies , 2007, SETP.

[101]  Angelo Gargantini,et al.  Generation of C++ Unit Tests from Abstract State Machines Specifications , 2018, 2018 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW).

[102]  Han Li,et al.  Using ontology to generate test cases for GUI testing , 2011, Int. J. Comput. Appl. Technol..

[103]  Michael D. Ernst,et al.  Are mutants a valid substitute for real faults in software testing? , 2014, SIGSOFT FSE.

[104]  Nikolai Tillmann,et al.  Pex-White Box Test Generation for .NET , 2008, TAP.

[105]  Andreas Zeller,et al.  The Impact of Equivalent Mutants , 2009, 2009 International Conference on Software Testing, Verification, and Validation Workshops.

[106]  Dirk Beyer,et al.  CPAchecker: A Tool for Configurable Software Verification , 2009, CAV.

[107]  George Varghese,et al.  Header Space Analysis: Static Checking for Networks , 2012, NSDI.

[108]  Johann Marius Zöllner,et al.  Traffic intersection situation description ontology for advanced driver assistance , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[109]  Joelle Pineau,et al.  Towards an Automatic Turing Test: Learning to Evaluate Dialogue Responses , 2017, ACL.

[110]  Manuel Fähndrich,et al.  Static Verification for Code Contracts , 2010, SAS.

[111]  Angelo Gargantini,et al.  Asm2C++: A Tool for Code Generation from Abstract State Machines to Arduino , 2017, NFM.

[112]  John Rushby,et al.  Quality Measures and Assurance for AI Software1 , 1988 .

[113]  Ailton Akira Shinoda,et al.  Using Mininet for emulation and prototyping Software-Defined Networks , 2014, 2014 IEEE Colombian Conference on Communications and Computing (COLCOM).

[114]  Jorge J. Gómez-Sanz,et al.  Virtual Development of a Presence Sensor Network Using 3D Simulations , 2017, Smart-CT.

[115]  Bjarne Stroustrup,et al.  Programming: Principles and Practice Using C++ , 2008 .

[116]  Alessandro Orso,et al.  A Classification of SQL Injection Attacks and Countermeasures , 2006, ISSSE.

[117]  Tien-Dung Cao,et al.  Automated Runtime Verification for Web Services , 2010, 2010 IEEE International Conference on Web Services.

[118]  Jorge J. Gómez-Sanz,et al.  A Framework for Developing Multi-Agent Systems in Ambient Intelligence Scenarios , 2015, AAMAS.

[119]  Wolfram Wöß,et al.  An analysis of ontologies and their success factors for application to business , 2016, Data Knowl. Eng..

[120]  Ingo Stürmer,et al.  Systematic Testing of Model-Based Code Generators , 2007, IEEE Transactions on Software Engineering.

[121]  Bertrand Meyer Design By Contract. The Eiffel Method , 1998, Proceedings. Technology of Object-Oriented Languages. TOOLS 26 (Cat. No.98EX176).

[122]  Jochen Malte Küster,et al.  Validation of model transformations: first experiences using a white box approach , 2006, MoDELS'06.

[123]  Ana R. Cavalli,et al.  FSM-based conformance testing methods: A survey annotated with experimental evaluation , 2010, Inf. Softw. Technol..

[124]  David S. Rosenblum,et al.  Context-Aware Adaptive Applications: Fault Patterns and Their Automated Identification , 2010, IEEE Transactions on Software Engineering.

[125]  Mirko Conrad Testing-based translation validation of generated code in the context of IEC 61508 , 2009, Formal Methods Syst. Des..

[126]  Andrea Polini,et al.  Testing software components for integration: a survey of issues and techniques , 2007, Softw. Test. Verification Reliab..

[127]  Heike Wehrheim,et al.  Jass - Java with Assertions , 2001, RV@CAV.

[128]  Fabiano Cutigi Ferrari,et al.  Mutation Testing for Aspect-Oriented Programs , 2008, 2008 1st International Conference on Software Testing, Verification, and Validation.

[129]  Albert L. Baker,et al.  Preliminary design of JML: a behavioral interface specification language for java , 2006, SOEN.

[130]  Janak H. Patel,et al.  Stuck-at fault: a fault model for the next millennium , 1998, Proceedings International Test Conference 1998 (IEEE Cat. No.98CH36270).

[131]  Neville Churcher,et al.  A Generated Parser of C , 2001 .

[132]  Sarfraz Khurshid,et al.  Korat: automated testing based on Java predicates , 2002, ISSTA '02.

[133]  T. H. Tse,et al.  Testing context-aware middleware-centric programs: a data flow approach and an RFID-based experimentation , 2006, SIGSOFT '06/FSE-14.

[134]  E. Mark Gold,et al.  Complexity of Automaton Identification from Given Data , 1978, Inf. Control..

[135]  Jean-Marc Jézéquel,et al.  Model Driven Engineering , 2017, Encyclopedia of GIS.

[136]  Peter Thiemann,et al.  TreatJS: Higher-Order Contracts for JavaScripts , 2015, ECOOP.

[137]  Marc Pallot,et al.  Living Lab Research Landscape: From User Centred Design and User Experience towards User Cocreation , 2010 .

[138]  Nina Yevtushenko,et al.  Investigation of Correlation of Test Sequences for Reliability Testing of Digital Physical System Components , 2016 .

[139]  Jan Reineke,et al.  Gray-Box Learning of Serial Compositions of Mealy Machines , 2016, NFM.

[140]  J. Dingel,et al.  Mutation Operators for Concurrent Java (J2SE 5.0) , 2006, Second Workshop on Mutation Analysis (Mutation 2006 - ISSRE Workshops 2006).

[141]  Monica S. Lam,et al.  Automatic Generation of XSS and SQL Injection Attacks with Goal-Directed Model Checking , 2008, USENIX Security Symposium.

[142]  Tanja E. J. Vos,et al.  AB=ᗺA: execution equivalence as a new type of testing oracle , 2015, SAC.

[143]  Claudio S. Pinhanez,et al.  Bottester: Testing Conversational Systems with Simulated Users , 2017, IHC.

[144]  I. S. W. B. Prasetya T3i: a tool for generating and querying test suites for Java , 2015, ESEC/SIGSOFT FSE.

[145]  Stephen S. Yau,et al.  Integration Testing of Context-sensitive Middleware-based Applications: a Metamorphic Approach , 2006, Int. J. Softw. Eng. Knowl. Eng..

[146]  Catherine Oriat,et al.  Jartege: A Tool for Random Generation of Unit Tests for Java Classes , 2004, QoSA/SOQUA.

[147]  Alexandre Petrenko,et al.  Test Selection Based on Communicating Nondeterministic Finite-State Machines Using a Generalized WP-Method , 1994, IEEE Trans. Software Eng..

[148]  Mark Weiser,et al.  Some computer science issues in ubiquitous computing , 1993, CACM.

[149]  Shing-Chi Cheung,et al.  Inter-context control-flow and data-flow test adequacy criteria for nesC applications , 2008, SIGSOFT '08/FSE-16.

[150]  Eva Blomqvist,et al.  Ontology Testing - Methodology and Tool , 2012, EKAW.

[151]  Franz Wotawa,et al.  Security Testing Based on Attack Patterns , 2014, 2014 IEEE Seventh International Conference on Software Testing, Verification and Validation Workshops.

[152]  Yves Le Traon,et al.  Chapter Six - Mutation Testing Advances: An Analysis and Survey , 2019, Adv. Comput..

[153]  K. Rustan M. Leino,et al.  The Spec# Programming System: An Overview , 2004, CASSIS.

[154]  Rui Li,et al.  The implementation of user interface autogenerate for spacecraft automatic tests based on ontology , 2015, 2015 12th International Conference on Fuzzy Systems and Knowledge Discovery (FSKD).

[155]  Karl Meinke,et al.  CGE: A Sequential Learning Algorithm for Mealy Automata , 2010, ICGI.

[156]  Graham K. Rand,et al.  Quantitative Applications in the Social Sciences , 1983 .

[157]  Scott Meyers,et al.  Effective modern C++: 42 specific ways to improve your use of C++11 and C++14 , 2014 .

[158]  Anna Perini,et al.  Ontology-based test generation for multiagent systems , 2008, AAMAS.

[159]  Stephen McCamant,et al.  The Daikon system for dynamic detection of likely invariants , 2007, Sci. Comput. Program..

[160]  Inmaculada Medina-Bulo,et al.  Assessment of class mutation operators for C++ with the MuCPP mutation system , 2017, Inf. Softw. Technol..

[161]  Jorge López,et al.  Behavior evaluation for trust management based on formal distributed network monitoring , 2015, World Wide Web.

[162]  Jeffery von Ronne,et al.  Test Suite Minimization an Empirical Investigation an Abstract of the Thesis of Test Suite Minimization an Empirical Investigation Acknowledgment , 2022 .

[163]  Thomas R. Gross,et al.  Automatic testing of sequential and concurrent substitutability , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[164]  Keiichi Yasumoto,et al.  A method for recognizing living activities in homes using positioning sensor and power meters , 2015, 2015 IEEE International Conference on Pervasive Computing and Communication Workshops (PerCom Workshops).

[165]  Gang Liu,et al.  Model-based testing and validation on artificial intelligence systems , 2007, Second International Multi-Symposiums on Computer and Computational Sciences (IMSCCS 2007).

[166]  Olivier Bonaventure,et al.  Towards test-driven software defined networking , 2014, 2014 IEEE Network Operations and Management Symposium (NOMS).

[167]  Guoqing Xu,et al.  JMLAutoTest: A Novel Automated Testing Framework Based on JML and JUnit , 2003, FATES.

[168]  Ana R. Cavalli,et al.  FSM-Based Test Derivation Strategies for Systems with Time-Outs , 2011, 2011 11th International Conference on Quality Software.

[169]  Alberto Fernández-Isabel,et al.  Analysis of Intelligent Transportation Systems Using Model-Driven Simulations , 2015, Sensors.

[170]  Fabrice Bouquet,et al.  Automated Boundary Test Generation from JML Specifications , 2006, FM.

[171]  Ferhat Khendek,et al.  Timed Wp-Method: Testing Real-Time Systems , 2002, IEEE Trans. Software Eng..

[172]  Feng Duan,et al.  Combinatorial Testing: Theory and Practice , 2015, Adv. Comput..

[173]  Bernhard Steffen,et al.  Active Automata Learning: From DFAs to Interface Programs and Beyond , 2012, ICGI.

[174]  Michael L. Fredman,et al.  The AETG System: An Approach to Testing Based on Combinatiorial Design , 1997, IEEE Trans. Software Eng..

[175]  Sakir Sezer,et al.  Queen ' s University Belfast-Research Portal Are We Ready for SDN ? Implementation Challenges for Software-Defined Networks , 2016 .

[176]  Ichiro Satoh,et al.  A Testing Framework for Mobile Computing Software , 2003, IEEE Trans. Software Eng..

[177]  Wolfram Schulte,et al.  VCC: Contract-based modular verification of concurrent C , 2009, 2009 31st International Conference on Software Engineering - Companion Volume.

[178]  Keiichi Yasumoto,et al.  Saving energy in smart homes with minimal comfort level reduction , 2013, 2013 IEEE International Conference on Pervasive Computing and Communications Workshops (PERCOM Workshops).

[179]  Jianping Wu,et al.  Formal Modeling and Systematic Black-Box Testing of SDN Data Plane , 2014, 2014 IEEE 22nd International Conference on Network Protocols.

[180]  Ricardo de Almeida Falbo,et al.  Using Ontology Patterns for Building a Reference Software Testing Ontology , 2013, 2013 17th IEEE International Enterprise Distributed Object Computing Conference Workshops.

[181]  Jing Tian,et al.  An Ontology-Based Knowledge Sharing Portal for Software Testing , 2017, 2017 IEEE International Conference on Software Quality, Reliability and Security Companion (QRS-C).

[182]  Joseph Weizenbaum,et al.  ELIZA—a computer program for the study of natural language communication between man and machine , 1966, CACM.

[183]  Mats Per Erik Heimdahl,et al.  Test-suite reduction for model based tests: effects on test quality and implications for testing , 2004, Proceedings. 19th International Conference on Automated Software Engineering, 2004..

[184]  K. K. Aggarwal,et al.  A neural net based approach to Test Oracle , 2004, SOEN.

[185]  Katerina J. Argyraki,et al.  Toward a verifiable software dataplane , 2013, HotNets.

[186]  Nina Yevtushenko,et al.  Towards Model based Testing for Software Defined Networks , 2018, ENASE.

[187]  Jeff Yu Lei,et al.  ACTS: A Combinatorial Test Generation Tool , 2013, 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation.

[188]  Keqin Li,et al.  Integration testing of communicating systems with unknown components , 2015, Ann. des Télécommunications.

[189]  Robert K. Brayton,et al.  ABC: An Academic Industrial-Strength Verification Tool , 2010, CAV.

[190]  Atif M. Memon,et al.  Definition and evaluation of mutation operators for GUI-level mutation analysis , 2015, 2015 IEEE Eighth International Conference on Software Testing, Verification and Validation Workshops (ICSTW).