A Framework for Continuous Regression and Integration Testing in IoT Systems Based on Deep Learning and Search-Based Techniques

Tremendous systems are rapidly evolving based on the trendy Internet of Things (IoT) in various domains. Different technologies are used for communication between the massive connected devices through all layers of the IoT system, causing many security and performance issues. Regression and integration testing are considered repeatedly, in which the vast costs and efforts associated with the frequent execution of these inflated test suites hinder the adequate testing of such systems. This necessitates the focus on exploring innovative scalable testing approaches for large test suites in IoT-based systems. In this paper, a scalable framework for continuous integration and regression testing in IoT-based systems (IoT-CIRTF) is proposed, based on IoT-related criteria for test case prioritization and selection. The framework utilizes search-based techniques to provide an optimized prioritized set of test cases to select from. The selection is based on a trained prediction model for IoT standard components using supervised deep learning algorithms to continuously ensure the overall reliability of IoT-based systems. The experiments are held on two GSM datasets. The experimental results achieved prioritization accuracy up to 90% and 92% for regression testing and integration testing respectively. This provides an enhanced and efficient framework for continuous testing of IoT-based systems, as per IoT-related criteria for the prioritization and selection purposes.

[1]  Mark Johnson,et al.  Automated test case generation from high-level logic requirements using model transformation techniques , 2017, 2017 9th Computer Science and Electronic Engineering (CEEC).

[2]  Sorin-Aurel Moraru,et al.  Continuous Testing in the Development of IoT Applications , 2019, 2019 International Conference on Sensing and Instrumentation in IoT Era (ISSI).

[3]  Davide Ancona,et al.  An acceptance testing approach for Internet of Things systems , 2018, IET Softw..

[4]  Anna Katrina Gomez,et al.  Challenges of Testing Complex Internet of Things (IoT) Devices and Systems , 2019, 2019 11th International Conference on Knowledge and Systems Engineering (KSE).

[5]  Mehreen Sirshar,et al.  Software Quality Assurance Testing Methodologies in IoT , 2019 .

[6]  Romina Spalazzese,et al.  Characterizing Internet of Things Systems through Taxonomies: A Systematic Mapping Study , 2019, Internet Things.

[7]  Raúl Montoliu,et al.  A study of deep neural networks for human activity recognition , 2020, Comput. Intell..

[8]  Ihtesham ul Haq,et al.  GSM Technology: Architecture, Security and Future Challenges , 2017 .

[9]  Laurie A. Williams,et al.  System test case prioritization of new and regression test cases , 2005, 2005 International Symposium on Empirical Software Engineering, 2005..

[10]  Bruno Lima.  Automated Scenario-Based Integration Testing of Time-Constrained Distributed Systems , 2019, 2019 12th IEEE Conference on Software Testing, Validation and Verification (ICST).

[12]  B. Prakash,et al.  A comparative study of meta-heuristic optimisation techniques for prioritisation of risks in agile software development , 2020 .

[13]  Prasanna Balaprakash,et al.  An Experimental Study of Global and Local Search Algorithms in Empirical Performance Tuning , 2012, VECPAR.

[14]  Miroslav Bures,et al.  Interoperability and Integration Testing Methods for IoT Systems: a Systematic Mapping Study , 2020, SEFM.

[15]  Mariam Lahami,et al.  Towards a Runtime Testing Framework for Dynamically Adaptable Internet of Things Networks in Smart Cities , 2020 .

[16]  Moez Krichen,et al.  Improving Formal Verification and Testing Techniques for Internet of Things and Smart Cities , 2019, Mobile Networks and Applications.

[17]  Xiaolin Wang,et al.  Location-Based Test Case Prioritization for Software Embedded in Mobile Devices Using the Law of Gravitation , 2019, Mob. Inf. Syst..

[18]  Wan Haslina Hassan,et al.  Current research on Internet of Things (IoT) security: A survey , 2019, Comput. Networks.

[19]  John Wu,et al.  Bug Report Classification Using LSTM Architecture for More Accurate Software Defect Locating , 2018, 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA).

[20]  Sung Wook Baik,et al.  Action Recognition in Video Sequences using Deep Bi-Directional LSTM With CNN Features , 2018, IEEE Access.

[21]  Md. Rezaul Karim,et al.  A Scalable and Hybrid Intrusion Detection System Based on the Convolutional-LSTM Network , 2019, Symmetry.

[22]  Mert R. Sabuncu,et al.  Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels , 2018, NeurIPS.

[23]  Di He,et al.  Towards Binary-Valued Gates for Robust LSTM Training , 2018, ICML.

[24]  Raimo Kantola,et al.  An Adaptive Security Data Collection and Composition Recognition method for security measurement over LTE/LTE-A networks , 2020, J. Netw. Comput. Appl..

[25]  Jayashree D. Mallapur,et al.  Remote sensing and controlling of greenhouse agriculture parameters based on IoT , 2017, 2017 International Conference on Big Data, IoT and Data Science (BID).

[26]  Jean-Luc Gauvain,et al.  Spoken Language Identification Using LSTM-Based Angular Proximity , 2017, INTERSPEECH.

[27]  Carlo Gatta,et al.  Unsupervised Deep Feature Extraction for Remote Sensing Image Classification , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[28]  Alex Vakaloudis,et al.  A framework for rapid integration of IoT Systems with industrial environments , 2019, 2019 IEEE 5th World Forum on Internet of Things (WF-IoT).

[29]  Ce Zhang,et al.  A General Architecture of IoT System , 2017, 22017 IEEE International Conference on Computational Science and Engineering (CSE) and IEEE International Conference on Embedded and Ubiquitous Computing (EUC).

[30]  Yu Zheng,et al.  An ensemble framework based on convolutional bi-directional LSTM with multiple time windows for remaining useful life estimation , 2020, Comput. Ind..

[31]  Gregg Rothermel,et al.  Prioritizing test cases for regression testing , 2000, ISSTA '00.

[32]  André Stuhlsatz,et al.  Feature Extraction for Simple Classification , 2010, 2010 20th International Conference on Pattern Recognition.

[33]  Yuefan Deng,et al.  Applying Simulated Annealing and Parallel Computing to the Mobile Sequential Recommendation , 2019, IEEE Transactions on Knowledge and Data Engineering.

[34]  Atif M. Memon,et al.  Fault Detection Probability Analysis for Coverage-Based Test Suite Reduction , 2007, 2007 IEEE International Conference on Software Maintenance.

[35]  Richard Socher,et al.  Regularizing and Optimizing LSTM Language Models , 2017, ICLR.

[36]  Xiaohui Huang,et al.  Make It Directly: Event Extraction Based on Tree-LSTM and Bi-GRU , 2020, IEEE Access.

[37]  Yan Tian,et al.  Traffic flow prediction using LSTM with feature enhancement , 2019, Neurocomputing.

[38]  Stefan Güttel,et al.  Time Series Forecasting Using LSTM Networks: A Symbolic Approach , 2020, ArXiv.

[39]  Sung Wook Baik,et al.  Short-Term Prediction of Residential Power Energy Consumption via CNN and Multi-Layer Bi-Directional LSTM Networks , 2020, IEEE Access.

[40]  Wang Jun,et al.  Test Case Prioritization Technique Based on Genetic Algorithm , 2011, 2011 International Conference on Internet Computing and Information Services.

[41]  R. Uma Maheswari,et al.  Combined Genetic and Simulated Annealing Approach for Test Case Prioritization , 2015 .

[42]  Siti Hafizah Ab Hamid,et al.  Cost and Effectiveness of Search-Based Techniques for Model-Based Testing: An Empirical Analysis , 2017, Int. J. Softw. Eng. Knowl. Eng..

[43]  Dongdong Li,et al.  Collaborative and geometric multi-kernel learning for multi-class classification , 2020, Pattern Recognit..

[44]  Han Zou,et al.  WiFi Fingerprinting Indoor Localization Using Local Feature-Based Deep LSTM , 2020, IEEE Systems Journal.

[45]  Ismail Ari,et al.  Democratization of runtime verification for internet of things , 2018, Comput. Electr. Eng..

[46]  Patrick D. McDaniel,et al.  Program Analysis of Commodity IoT Applications for Security and Privacy , 2018, ACM Comput. Surv..

[47]  Lele Zhang,et al.  INTELLIGENT REGRESSION TESTING FOR INTERNET OF THINGS WIRELESS DEVICE USING MIXED MACHINE LEARNING METHODS , 2019 .

[48]  Sherin M. Moussa,et al.  Testing Techniques in IoT-based Systems , 2019, 2019 Ninth International Conference on Intelligent Computing and Information Systems (ICICIS).

[49]  David Lie,et al.  Ex-vivo dynamic analysis framework for Android device drivers , 2020, 2020 IEEE Symposium on Security and Privacy (SP).

[50]  Hisao Ishibuchi,et al.  Performance evaluation of genetic algorithms for flowshop scheduling problems , 1994, Proceedings of the First IEEE Conference on Evolutionary Computation. IEEE World Congress on Computational Intelligence.

[51]  K. S. Patnaik,et al.  Prioritizing JUnit Test Cases Without Coverage Information: An Optimization Heuristics Based Approach , 2019, IEEE Access.

[52]  Seyed-Hassan Mirian-Hosseinabadi,et al.  Incorporating fault-proneness estimations into coverage-based test case prioritization methods , 2019, Inf. Softw. Technol..

[53]  Ana C. R. Paiva,et al.  A Brief Overview of Existing Tools for Testing the Internet-of-Things , 2018, 2018 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW).

[54]  Gang Liu,et al.  Bidirectional LSTM with attention mechanism and convolutional layer for text classification , 2019, Neurocomputing.

[55]  Mehrdad Sabetzadeh,et al.  Test case prioritization for acceptance testing of cyber physical systems: a multi-objective search-based approach , 2018, ISSTA.

[56]  Shuai Li,et al.  Independently Recurrent Neural Network (IndRNN): Building A Longer and Deeper RNN , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[57]  Inmaculada Medina-Bulo,et al.  IoT-TEG: Test event generator system , 2018, J. Syst. Softw..