An empirical investigation of perceived reliability of open source Java programs

Background: Open Source Software (OSS) is used by a continuously growing number of people, both end-users and developers. The quality of OSS is thus an issue of increasing interest. Specifically, OSS stakeholders need to trust OSS with respect to a number of qualities. Objective: This paper focuses on the level of trust that OSS stakeholders have in OSS reliability, one of the most important software qualities. The goal of the work reported here is to investigate to what extent the perception of reliability by users depends on objectively measurable characteristics of software. Method: We collected subjective user evaluations of the reliability of 22 Java OSS products, and then we measured their code characteristics that are generally believed to affect the quality of software. Finally, we carried out a correlational study to predict the perceived level of reliability of OSS based on the measured characteristics of the software code. Result: We obtained a set of statistically significant quantitative models, collectively called MOSST\REL, which account for the dependence of the perceived reliability of OSS on objectively observable qualities of Java code. Conclusions: The models we obtained can be used by: 1) endusers and developers that would like to reuse existing OSS products and components, to evaluate the perceived level of reliability of these OSS products that can be expected based on the characteristics of code; 2) the developers of OSS products, who can set code quality targets based on the level of perceived reliability they want to achieve.