Continuous Experimentation: Challenges, Implementation Techniques, and Current Research

Continuous experimentation is an up-and-coming technique for requirements engineering and testing, particularly for web-based systems. On the basis of a practitioner survey, this article gives an overview of challenges, implementation techniques, and current research in the field. This article is part of a theme issue on release engineering.

[1]  Ron Kohavi,et al.  Seven rules of thumb for web site experimenters , 2014, KDD.

[2]  Jan Bosch,et al.  The Evolution of Continuous Experimentation in Software Product Development: From Data to a Data-Driven Organization at Scale , 2017, 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE).

[3]  Harald C. Gall,et al.  We're doing it live: A multi-method empirical study on continuous experimentation , 2018, Inf. Softw. Technol..

[4]  Laurie A. Williams,et al.  Characterizing Experimentation in Continuous Deployment: A Case Study on Bing , 2017, 2017 IEEE/ACM 39th International Conference on Software Engineering: Software Engineering in Practice Track (ICSE-SEIP).

[5]  Harald C. Gall,et al.  Bifrost: Supporting Continuous Deployment with Automated Enactment of Multi-Phase Live Testing Strategies , 2016, Middleware.

[6]  Robert Karl,et al.  Holistic configuration management at Facebook , 2015, SOSP.

[7]  Bram Adams,et al.  Feature Toggles: Practitioner Practices and a Case Study , 2016, 2016 IEEE/ACM 13th Working Conference on Mining Software Repositories (MSR).

[8]  Jürgen Münch,et al.  Raising the odds of success: the current state of experimentation in product development , 2016, Inf. Softw. Technol..