More for Less: Automated Experimentation in Software-Intensive Systems

Companies developing autonomous and software-intensive systems show an increasing need to adopt experimentation and data-driven strategies in their development process. With the growing complexity of the systems, companies are increasing their data analytic and experimentation teams to support data-driven development. However, organizations cannot increase in size at the same pace as the system complexity grows. Experimentation teams could run a larger number of experiments by letting the system itself to coordinate its own experiments, instead of the humans. This process is called automated experimentation. However, currently, no tools or frameworks address the challenge of running automated experiments.

[1]  Kerstin Dautenhahn,et al.  An Autonomous Proxemic System for a Mobile Companion Robot , 2010 .

[2]  Michael S. Bernstein,et al.  Designing and deploying online field experiments , 2014, WWW.

[3]  Ron Kohavi,et al.  Responsible editor: R. Bayardo. , 2022 .

[4]  Jan Bosch,et al.  Data-Driven Continuous Evolution of Smart Systems , 2016, 2016 IEEE/ACM 11th International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS).

[5]  Ashish Agarwal,et al.  Overlapping experiment infrastructure: more, better, faster experimentation , 2010, KDD.

[6]  Alan R. Hevner,et al.  Design Science in Information Systems Research , 2004, MIS Q..

[7]  Jan Bosch,et al.  Software Architecture as a Set of Architectural Design Decisions , 2005, 5th Working IEEE/IFIP Conference on Software Architecture (WICSA'05).

[8]  Wei Chu,et al.  A contextual-bandit approach to personalized news article recommendation , 2010, WWW '10.

[9]  Jan Bosch,et al.  The Evolution of Continuous Experimentation in Software Product Development: From Data to a Data-Driven Organization at Scale , 2017, 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE).

[10]  Jan Bosch,et al.  From Opinions to Data-Driven Software R&D: A Multi-case Study on How to Close the 'Open Loop' Problem , 2014, 2014 40th EUROMICRO Conference on Software Engineering and Advanced Applications.

[11]  Ron Kohavi,et al.  Online controlled experiments at large scale , 2013, KDD.

[12]  Jan Bosch,et al.  Time to Say 'Good Bye': Feature Lifecycle , 2016, 2016 42th Euromicro Conference on Software Engineering and Advanced Applications (SEAA).

[13]  Jan Bosch,et al.  Your System Gets Better Every Day You Use It: Towards Automated Continuous Experimentation , 2017, 2017 43rd Euromicro Conference on Software Engineering and Advanced Applications (SEAA).