FEVER: An approach to analyze feature-oriented changes and artefact co-evolution in highly configurable systems

The evolution of highly configurable systems is known to be a challenging task. Thorough understanding of configuration options their relationships, and their implementation in various types of artefacts (variability model, mapping, and implementation) is required to avoid compilation errors, invalid products, or dead code. Recent studies focusing on co-evolution of artefacts detailed feature-oriented change scenarios, describing how related artefacts might change over time. However, relying on manual analysis of commits, such work do not provide the means to obtain quantitative information on the frequency of described scenarios nor information on the exhaustiveness of the presented scenarios for the evolution of a large scale system. In this work, we propose FEVER and its instantiation for the Linux kernel. FEVER extracts detailed information on changes in variability models (KConfig files), assets (preprocessor based C code), and mappings (Makefiles). We apply this methodology to the Linux kernel and build a dataset comprised of 15 releases of the kernel history. We performed an evaluation of the FEVER approach by manually inspecting the data and compared it with commits in the system’s history. The evaluation shows that FEVER accurately captures feature related changes for more than 85% of the 810 manually inspected commits. We use the collected data to reflect on occurrences of co-evolution in practice. Our analysis shows that complex co-evolution scenarios occur in every studied release but are not among the most frequent change scenarios, as they only occur for 8 to 13% of the evolving features. Moreover, only a minority of developers working on a given release will make changes to all artefacts related to a feature (between 10% and 13% of authors). While our conclusions are derived from observations on the evolution of the Linux kernel, we believe that they may have implications for tool developers as well as guide further research in the field of co-evolution of artefacts.

[1]  Oscar Nierstrasz,et al.  Assigning bug reports using a vocabulary-based expertise model of developers , 2009, 2009 6th IEEE International Working Conference on Mining Software Repositories.

[2]  Thomas Leich,et al.  TypeChef: toward type checking #ifdef variability in C , 2010, FOSD '10.

[3]  Krzysztof Czarnecki,et al.  A dataset of feature additions and feature removals from the Linux kernel , 2014, MSR 2014.

[4]  Diomidis Spinellis,et al.  Does Your Configuration Code Smell? , 2016, 2016 IEEE/ACM 13th Working Conference on Mining Software Repositories (MSR).

[5]  Gail C. Murphy,et al.  Hipikat: recommending pertinent software development artifacts , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..

[6]  Kyo Chul Kang,et al.  Feature-Oriented Domain Analysis (FODA) Feasibility Study , 1990 .

[7]  Árpád Beszédes,et al.  Supporting software product line testing by optimizing code configuration coverage , 2015, 2015 IEEE Eighth International Conference on Software Testing, Verification and Validation Workshops (ICSTW).

[8]  Krzysztof Czarnecki,et al.  Variability modeling in the real: a perspective from the operating systems domain , 2010, ASE '10.

[9]  Jafar M. Al-Kofahi,et al.  Extracting Configuration Knowledge from Build Files with Symbolic Analysis , 2015, 2015 IEEE/ACM 3rd International Workshop on Release Engineering.

[10]  Harald C. Gall,et al.  Comparing fine-grained source code changes and code churn for bug prediction , 2011, MSR '11.

[11]  Sven Apel,et al.  Scalable analysis of variable software , 2013, ESEC/FSE 2013.

[12]  Sven Apel,et al.  Coevolution of variability models and related software artifacts , 2016, Empirical Software Engineering.

[13]  Gunter Saake,et al.  A Classification and Survey of Analysis Strategies for Software Product Lines , 2014, ACM Comput. Surv..

[14]  Wolfgang Schröder-Preikschat,et al.  A robust approach for variability extraction from the Linux build system , 2012, SPLC '12.

[15]  Andrew Begel,et al.  Codebook: discovering and exploiting relationships in software repositories , 2010, 2010 ACM/IEEE 32nd International Conference on Software Engineering.

[16]  Sven Apel,et al.  Preprocessor-based variability in open-source and industrial software systems: An empirical study , 2016, Empirical Software Engineering.

[17]  Richard C. Holt,et al.  Mining Kbuild to Detect Variability Anomalies in Linux , 2012, 2012 16th European Conference on Software Maintenance and Reengineering.

[18]  Sven Apel,et al.  Integrating Compositional and Annotative Approaches for Product Line Engineering , 2008 .

[19]  Sven Apel,et al.  An analysis of the variability in forty preprocessor-based software product lines , 2010, 2010 ACM/IEEE 32nd International Conference on Software Engineering.

[20]  Serge Demeyer,et al.  Change-based test selection: an empirical evaluation , 2015, Empirical Software Engineering.

[21]  Jan Bosch,et al.  On the notion of variability in software product lines , 2001, Proceedings Working IEEE/IFIP Conference on Software Architecture.

[22]  Daniel M. Germán,et al.  Continuously mining distributed version control systems: an empirical study of how Linux uses Git , 2014, Empirical Software Engineering.

[23]  Franz Wotawa,et al.  Automatic Software Bug Triage System (BTS) Based on Latent Semantic Indexing and Support Vector Machine , 2009, 2009 Fourth International Conference on Software Engineering Advances.

[24]  Paulo Borba,et al.  Partially safe evolution of software product lines , 2016, SPLC.

[25]  Paul Grünbacher,et al.  A Case Study on the Evolution of a Component-based Product Line , 2012, 2012 Joint Working IEEE/IFIP Conference on Software Architecture and European Conference on Software Architecture.

[26]  Alexander Serebrenik,et al.  Who's who in Gnome: Using LSA to merge software repository identities , 2012, 2012 28th IEEE International Conference on Software Maintenance (ICSM).

[27]  Uirá Kulesza,et al.  Safe evolution templates for software product lines , 2015, J. Syst. Softw..

[28]  Wolfgang Schröder-Preikschat,et al.  Dead or Alive: finding zombie features in the Linux kernel , 2009, FOSD '09.

[29]  Arie van Deursen,et al.  Evaluating Feature Change Impact on Multi-product Line Configurations Using Partial Information , 2014, ICSR.

[30]  Iago Abal,et al.  42 variability bugs in the linux kernel: a qualitative analysis , 2014, ASE.

[31]  Krzysztof Czarnecki,et al.  Mining configuration constraints: static analyses and empirical results , 2014, ICSE.

[32]  Krzysztof Czarnecki,et al.  Evolution of the Linux Kernel Variability Model , 2010, SPLC.

[33]  Henry Spencer,et al.  #ifdef Considered Harmful, or Portability Experience with C News , 1992, USENIX Summer.

[34]  Tom Mens,et al.  Challenges in software evolution , 2005, Eighth International Workshop on Principles of Software Evolution (IWPSE'05).

[35]  Arie van Deursen,et al.  Extracting feature model changes from the Linux kernel using FMDiff , 2014, VaMoS '14.

[36]  Arie van Deursen,et al.  FEVER: extracting feature-oriented changes from commits , 2016, MSR.

[37]  Juha Savolainen,et al.  Coevolution of variability models and code: an industrial case study , 2014, SPLC.

[38]  Krzysztof Czarnecki,et al.  A survey of variability modeling in industrial practice , 2013, VaMoS.

[39]  Daniel Lohmann,et al.  Understanding linux feature distribution , 2012, MISS '12.

[40]  Premkumar T. Devanbu,et al.  Latent social structure in open source projects , 2008, SIGSOFT '08/FSE-16.

[41]  Uirá Kulesza,et al.  Investigating the safe evolution of software product lines , 2011, GPCE '11.

[42]  Wolfgang De Meuter,et al.  The Evolution of the Linux Build System , 2007, Electron. Commun. Eur. Assoc. Softw. Sci. Technol..

[43]  Don S. Batory,et al.  GENESIS: An Extensible Database Management System , 1988, IEEE Trans. Software Eng..

[44]  David Lo,et al.  Identifying Linux bug fixing patches , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[45]  Arie van Deursen,et al.  Analysing the Linux kernel feature model changes using FMDiff , 2015, Software & Systems Modeling.

[46]  Thomas Thüm,et al.  Reasoning about edits to feature models , 2009, 2009 IEEE 31st International Conference on Software Engineering.

[47]  Daniel Lohmann,et al.  Feature Models in Linux: From Symbols to Semantics , 2016, VaMoS.

[48]  Paul Clements,et al.  Software product lines - practices and patterns , 2001, SEI series in software engineering.

[49]  Jun Han,et al.  Sharing with a Difference: Realizing Service-Based SaaS Applications with Runtime Sharing and Variation in Dynamic Software Product Lines , 2013, 2013 IEEE International Conference on Services Computing.

[50]  Wolfgang Schröder-Preikschat,et al.  Feature consistency in compile-time-configurable system software: facing the linux 10,000 feature problem , 2011, EuroSys '11.