Evaluating Bug-Fixing in Software Product Lines: an Industrial Case Study

[Background] Bug-fixing could be complex in industrial practice since thousands of products share features in their configuration. Despite the importance and complexity of bug-fixing, there is still a lack of empirical data about the difficulties found in industrial Software Product Lines (SPLs). [Aims] This paper aims to evaluate engineers' performance fixing errors and propagating the fixes to other configured products in the context of an industrial SPL. [Method] We designed and conducted an empirical study to collect data with regard to bug-fixing tasks within the context of a Induction Hob SPL in the BSH group, the largest manufacturer of home appliances in Europe. [Results] We found that effectiveness, efficiency and satisfaction got reached good values. Through interviews we also found difficulties related to unused features, cloning features unintentionally, detecting modified features, and propagating the fix when the source of the bug is the interaction between features. [Conclusions] The identified difficulties are relevant to know how to better apply SPLs in industry in the future.

[1]  Kyo Chul Kang,et al.  Feature-Oriented Domain Analysis (FODA) Feasibility Study , 1990 .

[2]  Per Runeson,et al.  Guidelines for conducting and reporting case study research in software engineering , 2009, Empirical Software Engineering.

[3]  Sarfraz Khurshid,et al.  Incremental Test Generation for Software Product Lines , 2010, IEEE Transactions on Software Engineering.

[4]  Robyn R. Lutz,et al.  Empirical evaluation of reliability improvement in an evolving software product line , 2011, MSR '11.

[5]  Claes Wohlin,et al.  Experimentation in software engineering: an introduction , 2000 .

[6]  Robert A. Virzi,et al.  Refining the Test Phase of Usability Evaluation: How Many Subjects Is Enough? , 1992 .

[7]  Hiroyuki Sato,et al.  Efficient testing of software product lines via centralization (short paper) , 2015 .

[8]  Natalia Juristo Juzgado,et al.  How to integrate usability into the software development process , 2006, ICSE.

[9]  Paul Clements,et al.  Better, faster, cheaper: Pick any three , 2012 .

[10]  Tanja E. J. Vos,et al.  An empirical approach for evaluating the usability of model-driven tools , 2013, Sci. Comput. Program..

[11]  Ken-ichi Matsumoto,et al.  The impact of bug management patterns on bug fixing: A case study of Eclipse projects , 2012, 2012 28th IEEE International Conference on Software Maintenance (ICSM).

[12]  Malte Lochau,et al.  Industrial evaluation of pairwise SPL testing with MoSo-PoLiTe , 2012, VaMoS.

[13]  Thomas S. Tullis,et al.  A Comparison of Questionnaires for Assessing Website Usability , 2004 .

[14]  Philip T. Kortum,et al.  Determining what individual SUS scores mean: adding an adjective rating scale , 2009 .

[15]  Hiroyuki Sato,et al.  Efficient testing of software product lines via centralization (short paper) , 2014, GPCE.