A formal framework for measuring technical lag in component repositories — and its application to npm

Reusable Open Source Software (OSS) components for major programming languages are available in package repositories. Developers rely on package management tools to automate deployments, specifying which package releases satisfy the needs of their applications. However, these specifications may lead to deploying package releases that are outdated, or otherwise undesirable, because they do not include bug fixes, security fixes, or new functionality. In contrast, automatically updating to a more recent release may introduce incompatibility issues. To capture this delicate balance, we formalise a generic model of technical lag, a concept that quantifies to which extent a deployed collection of components is outdated, with respect to the ideal deployment. We operationalise this model for the npm package manager. We empirically analyze the history of package update practices and technical lag for more than 500K packages with about 4M package releases over a seven‐year period. We consider both development and runtime dependencies, and study both direct and transitive dependencies. We also analyze the technical lag of external GitHub applications depending on npm packages. We report our findings, suggesting the need for more awareness of, and integrated tool support for, controlling technical lag in software libraries.

[1]  Xiaoyin Wang,et al.  Experience paper: a study on behavioral backward incompatibilities of Java software libraries , 2017, ISSTA.

[2]  Christian Kästner,et al.  Adding Sparkle to Social Coding: An Empirical Study of Repository Badges in the npm Ecosystem , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering: Companion (ICSE-Companion).

[3]  Jesús M. González-Barahona,et al.  Technical Lag in Software Compilations: Measuring How Outdated a Software Deployment Is , 2017, OSS.

[4]  Roberto Di Cosmo,et al.  Learning from the future of component repositories , 2012, CBSE '12.

[5]  Katsuro Inoue,et al.  Trusting a library: A study of the latency to adopt the latest Maven release , 2015, 2015 IEEE 22nd International Conference on Software Analysis, Evolution, and Reengineering (SANER).

[6]  Neil A. M. Maiden,et al.  Acquiring COTS Software Selection Requirements , 1998, IEEE Softw..

[7]  Shane McIntosh,et al.  Automatically repairing dependency-related build breakage , 2018, 2018 IEEE 25th International Conference on Software Analysis, Evolution and Reengineering (SANER).

[8]  Eleni Constantinou,et al.  An empirical comparison of developer retention in the RubyGems and npm software ecosystems , 2017, Innovations in Systems and Software Engineering.

[9]  James D. Herbsleb,et al.  When It Breaks, It Breaks: How Ecosystem Developers Reason about the Stability of Dependencies , 2015, 2015 30th IEEE/ACM International Conference on Automated Software Engineering Workshop (ASEW).

[10]  Andrew Nesbitt,et al.  Libraries.io Open Source Repository and Dependency Metadata , 2017 .

[11]  Jordi Cabot,et al.  A Systematic Mapping Study of Software Development With GitHub , 2017, IEEE Access.

[12]  William B. Frakes,et al.  Software reuse research: status and future , 2005, IEEE Transactions on Software Engineering.

[13]  Eleni Constantinou,et al.  On the Impact of Security Vulnerabilities in the npm Package Dependency Network , 2018, 2018 IEEE/ACM 15th International Conference on Mining Software Repositories (MSR).

[14]  Eleni Constantinou,et al.  An Empirical Analysis of Technical Lag in npm Package Dependencies , 2018, ICSR.

[15]  Tom Mens,et al.  An empirical comparison of dependency network evolution in seven software packaging ecosystems , 2017, Empirical Software Engineering.

[16]  Tobias Lauinger,et al.  Thou Shalt Not Depend on Me: Analysing the Use of Outdated JavaScript Libraries on the Web , 2018, NDSS.

[17]  Charles W. Krueger,et al.  Software reuse , 1992, CSUR.

[18]  Shawn A. Bohner,et al.  Extending software change impact analysis into COTS components , 2002, 27th Annual NASA Goddard/IEEE Software Engineering Workshop, 2002. Proceedings..

[19]  Eleni Constantinou,et al.  On the Evolution of Technical Lag in the npm Package Dependency Network , 2018, 2018 IEEE International Conference on Software Maintenance and Evolution (ICSME).

[20]  Daniel M. Germán,et al.  Macro-level software evolution: a case study of a large software compilation , 2009, Empirical Software Engineering.

[21]  Arie van Deursen,et al.  Semantic Versioning versus Breaking Changes: A Study of the Maven Repository , 2014, 2014 IEEE 14th International Working Conference on Source Code Analysis and Manipulation.

[22]  Rabe Abdalkareem,et al.  Why do developers use trivial packages? an empirical case study on npm , 2017, ESEC/SIGSOFT FSE.

[23]  Tom Mens,et al.  An empirical comparison of dependency issues in OSS packaging ecosystems , 2017, 2017 IEEE 24th International Conference on Software Analysis, Evolution and Reengineering (SANER).

[24]  Roberto Di Cosmo,et al.  Learning from the future of component repositories , 2014, Sci. Comput. Program..

[25]  Roberto Di Cosmo,et al.  Strong dependencies between software components , 2009, 2009 3rd International Symposium on Empirical Software Engineering and Measurement.

[26]  Roberto Di Cosmo,et al.  On software component co-installability , 2013, TSEM.

[27]  Roberto Di Cosmo,et al.  Dependency solving: A separate concern in component evolution management , 2012, J. Syst. Softw..

[28]  James D. Herbsleb,et al.  How to break an API: cost negotiation and community values in three software ecosystems , 2016, SIGSOFT FSE.

[29]  Roberto Di Cosmo,et al.  On software component co-installability , 2011, ESEC/FSE '11.

[30]  Roland Mittermeir,et al.  A survey of software reuse libraries , 1998, Ann. Softw. Eng..

[31]  Katsuro Inoue,et al.  On the Impact of Micro-Packages: An Empirical Study of the npm JavaScript Ecosystem , 2017, ArXiv.

[32]  Anders Møller,et al.  Type Regression Testing to Detect Breaking Changes in Node.js Libraries , 2018, ECOOP.

[33]  Marko C. J. D. van Eekelen,et al.  Measuring Dependency Freshness in Software Systems , 2015, 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering.

[34]  Martin Burger,et al.  Mining trends of library usage , 2009, IWPSE-Evol '09.

[35]  Katsuro Inoue,et al.  Do developers update their library dependencies? , 2017, Empirical Software Engineering.