Root causing, detecting, and fixing flaky tests: State of the art and future roadmap

[1]  Satish Chandra,et al.  Predictive Test Selection , 2018, 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP).

[2]  Zhiyu Fan,et al.  A Systematic Evaluation of Problematic Tests Generated by EvoSuite , 2019, 2019 IEEE/ACM 41st International Conference on Software Engineering: Companion Proceedings (ICSE-Companion).

[3]  Georgios Gousios,et al.  Developer Testing in the IDE: Patterns, Beliefs, and Behavior , 2019, IEEE Trans. Software Eng..

[4]  Michael D. Ernst,et al.  Empirically revisiting the test independence assumption , 2014, ISSTA 2014.

[5]  Amin Milani Fard,et al.  An empirical study of bugs in test code , 2015, 2015 IEEE International Conference on Software Maintenance and Evolution (ICSME).

[6]  Vahid Garousi,et al.  Smells in software test code: A survey of knowledge in industry and academia , 2018, J. Syst. Softw..

[7]  Anders Sjöbom.  Studying Test Flakiness in Python Projects : Original Findings for Machine Learning , 2019 .

[8]  Jürgen Großmann,et al.  A Taxonomy to Assess and Tailor Risk-Based Testing in Recent Testing Standards , 2019, IEEE Software.

[9]  Tao Xie,et al.  iFixFlakies: a framework for automatically fixing order-dependent flaky tests , 2019, ESEC/SIGSOFT FSE.

[10]  Andrew Curry,et al.  ßΣΤΛ testing. , 2021, Diabetes forecast.

[11]  Béla Vancsics,et al.  Simulating the Effect of Test Flakiness on Fault Localization Effectiveness , 2020, 2020 IEEE Workshop on Validation, Analysis and Evolution of Software Tests (VST).

[12]  Armin B. Cremers,et al.  Formal Modeling of Virtual Machines , 1978, IEEE Transactions on Software Engineering.

[13]  Santosh Biswas,et al.  Formal Modeling of Network-on-Chip Using CFSM and its Application in Detecting Deadlock , 2020, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[14]  Zebao Gao,et al.  Quantifying flakiness and Minimizing its effects on Software Testing , 2017 .

[15]  Darko Marinov,et al.  DeFlaker: Automatically Detecting Flaky Tests , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[16]  Andreas Zeller,et al.  Practical Test Dependency Detection , 2018, 2018 IEEE 11th International Conference on Software Testing, Verification and Validation (ICST).

[17]  Sigfrido Waidelich,et al.  Fire propagation visualization in real time , 2018, CloudCom 2018.

[18]  Christoph Treude,et al.  What is the Vocabulary of Flaky Tests? , 2020, 2020 IEEE/ACM 17th International Conference on Mining Software Repositories (MSR).

[19]  Darko Marinov,et al.  Detecting Assumptions on Deterministic Implementations of Non-deterministic Specifications , 2016, 2016 IEEE International Conference on Software Testing, Verification and Validation (ICST).

[20]  Samad Paydar,et al.  An Experimental Study on Flakiness and Fragility of Randoop Regression Test Suites , 2019, FSEN.

[21]  Rafael Serapilha Durelli,et al.  Machine Learning Applied to Software Testing: A Systematic Mapping Study , 2019, IEEE Transactions on Reliability.

[22]  Darko Marinov,et al.  Understanding and Improving Regression Test Selection in Continuous Integration , 2019, 2019 IEEE 30th International Symposium on Software Reliability Engineering (ISSRE).

[23]  Raymond A. DeCarlo,et al.  A Formal Model of the Software Test Process , 2002, IEEE Trans. Software Eng..

[24]  Sarah Smith Heckman,et al.  Wait, Wait. No, Tell Me. Analyzing Selenium Configuration Effects on Test Flakiness , 2019, 2019 IEEE/ACM 14th International Workshop on Automation of Software Test (AST).

[25]  Darko Marinov,et al.  An empirical analysis of flaky tests , 2014, SIGSOFT FSE.

[26]  Emanuel Irrazábal,et al.  Identifying Key Success Factors in Stopping Flaky Tests in Automated REST Service Testing , 2018 .

[27]  Na Meng,et al.  An Empirical Study of Flaky Tests in Android Apps , 2018, 2018 IEEE International Conference on Software Maintenance and Evolution (ICSME).

[28]  Andy Zaidman,et al.  RETRACTED ARTICLE: The smell of fear: on the relation between test smells and flaky tests , 2019, Empirical Software Engineering.

[29]  Thomas Bach,et al.  Coverage-Based Reduction of Test Execution Time: Lessons from a Very Large Industrial Project , 2017, 2017 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW).

[30]  Lori A. Clarke,et al.  A Formal Model of Program Dependences and Its Implications for Software Testing, Debugging, and Maintenance , 1990, IEEE Trans. Software Eng..

[31]  Reid Holmes,et al.  Measuring the cost of regression testing in practice: a study of Java projects using continuous integration , 2017, ESEC/SIGSOFT FSE.

[32]  Kai Petersen,et al.  Guidelines for conducting systematic mapping studies in software engineering: An update , 2015, Inf. Softw. Technol..

[33]  Suman Nath,et al.  Root causing flaky tests in a large-scale industrial setting , 2019, ISSTA.

[34]  Wing Lam,et al.  iDFlakies: A Framework for Detecting and Partially Classifying Flaky Tests , 2019, 2019 12th IEEE Conference on Software Testing, Validation and Verification (ICST).

[35]  Tariq M. King,et al.  Towards a Bayesian Network Model for Predicting Flaky Automated Tests , 2018, 2018 IEEE International Conference on Software Quality, Reliability and Security Companion (QRS-C).

[36]  Qiang Li,et al.  Software Testing Data Analysis Based on Data Mining , 2017, 2017 4th International Conference on Information Science and Control Engineering (ICISCE).

[37]  John Micco,et al.  The State of Continuous Integration Testing @Google , 2017 .

[38]  Andy Zaidman,et al.  Does Refactoring of Test Smells Induce Fixing Flaky Tests? , 2017, 2017 IEEE International Conference on Software Maintenance and Evolution (ICSME).

[39]  Sasa Misailovic,et al.  Detecting flaky tests in probabilistic and machine learning applications , 2020, International Symposium on Software Testing and Analysis.

[40]  Md Tajmilur Rahman,et al.  The impact of failing, flaky, and high failure tests on the number of crash reports associated with Firefox builds , 2018, ESEC/SIGSOFT FSE.

[41]  Valeriy Vyatkin,et al.  Formal Modeling of Testing Software for Cyber-Physical Automation Systems , 2015, 2015 IEEE Trustcom/BigDataSE/ISPA.

[42]  Earl T. Barr,et al.  Automated Fix Detection Given Flaky Tests , 2018, ArXiv.

[43]  Muhammad Ali Gulzar,et al.  Perception and Practices of Differential Testing , 2019, 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP).

[44]  Kai Petersen,et al.  Systematic Mapping Studies in Software Engineering , 2008, EASE.

[45]  Arif Ali Khan,et al.  Analytic Hierarchy Process Based Prioritisation and Taxonomy of Success Factors for Scaling Agile Methods in Global Software Development , 2020, IET Softw..

[46]  Yves Le Traon,et al.  Assessing Transition-Based Test Selection Algorithms at Google , 2019, 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP).

[47]  Fabio Palomba,et al.  Understanding flaky tests: the developer’s perspective , 2019, ESEC/SIGSOFT FSE.

[48]  Darko Marinov,et al.  Mitigating the effects of flaky tests on mutation testing , 2019, ISSTA.