Automated measurement of API usability: The API Concepts Framework

ContextUsability is an important software quality attribute for APIs. Unfortunately, measuring it is not an easy task since many things like experienced evaluators, suitable test users, and a functional product are needed. This makes existing usability measurement methods difficult to use, especially for non-professionals. ObjectiveTo make API usability measurement easier, an automated and objective measurement method would be needed. This article proposes such a method. Since it would be impossible to find and integrate all possible factors that influence API usability in one step, the main goal is to prove the feasibility of the introduced approach, and to define an extensible framework so that additional factors can easily be defined and added later. MethodA literature review is conducted to find potential factors influencing API usability. From these factors, a selected few are investigated more closely with usability studies. The statistically evaluated results from these studies are used to define specific elements of the introduced framework. Further, the influence of the user as a critical factor for the framework's feasibility is evaluated. ResultsThe API Concepts Framework is defined, with an extensible structure based on concepts that represent the user's actions, measurable properties that define what influences the usability of these concepts, and learning effects that represent the influence of the user's experience. A comparison of values calculated by the framework with user studies shows promising results. ConclusionIt is concluded that the introduced approach is feasible and provides useful results for evaluating API usability. The extensible framework easily allows to add new concepts and measurable properties in the future.

[1]  Ken Arnold.  Programmers Are People, too , 2005, ACM Queue.

[2]  Brad A. Myers,et al.  Six Learning Barriers in End-User Programming Systems , 2004, 2004 IEEE Symposium on Visual Languages - Human Centric Computing.

[3]  Cleidson R. B. de Souza,et al.  Automatic evaluation of API usability using complexity metrics and visualizations , 2009, 2009 31st International Conference on Software Engineering - Companion Volume.

[4]  Brad A. Myers,et al.  The implications of method placement on API learnability , 2008, SIGSOFT '08/FSE-16.

[5]  Martin P. Robillard,et al.  Using Structure-Based Recommendations to Facilitate Discoverability in APIs , 2011, ECOOP.

[6]  Mark Lorenz,et al.  Object-oriented software metrics - a practical guide , 1994 .

[7]  John E. Gaffney,et al.  Software Function, Source Lines of Code, and Development Effort Prediction: A Software Science Validation , 1983, IEEE Transactions on Software Engineering.

[8]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[9]  Brian Ellis,et al.  The Factory Pattern in API Design: A Usability Evaluation , 2007, 29th International Conference on Software Engineering (ICSE'07).

[10]  Miriam A. M. Capretz,et al.  Metrics suite for class complexity , 2005, International Conference on Information Technology: Coding and Computing (ITCC'05) - Volume II.

[11]  Martin Schrepp,et al.  Construction and Evaluation of a User Experience Questionnaire , 2008, USAB.

[12]  P. Oman,et al.  Metrics for assessing a software system's maintainability , 1992, Proceedings Conference on Software Maintenance 1992.

[13]  Morten Hertzum,et al.  The Evaluator Effect: A Chilling Fact About Usability Evaluation Methods , 2001, Int. J. Hum. Comput. Interact..

[14]  Chanchal Kumar Roy,et al.  Useful, But Usable? Factors Affecting the Usability of APIs , 2011, 2011 18th Working Conference on Reverse Engineering.

[15]  Jaroslav Tulach.  Practical API Design: Confessions of a Java Framework Architect , 2008 .

[16]  Ralph Johnson,et al.  design patterns elements of reusable object oriented software , 2019 .

[17]  Vlad Gorelik,et al.  One Step Ahead , 2007, ACM Queue.

[18]  eva Kühn,et al.  Usability Evaluation of Configuration-Based API Design Concepts , 2013, SouthCHI.

[19]  Nasib S. Gill,et al.  Few important considerations for deriving interface complexity metric for component-based systems , 2004, SOEN.

[20]  Chris F. Kemerer,et al.  A Metrics Suite for Object Oriented Design , 2015, IEEE Trans. Software Eng..

[21]  Brad A. Myers,et al.  Improving API documentation using API usage information , 2009, 2009 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC).

[22]  Jeffrey Stylos,et al.  Usability Implications of Requiring Parameters in Objects' Constructors , 2007, 29th International Conference on Software Engineering (ICSE'07).

[23]  Michi Henning.  API: Design Matters , 2007, ACM Queue.

[24]  Brad Abrams,et al.  Framework Design Guidelines: Conventions, Idioms, and Patterns for Reusable .NET Libraries , 2005 .

[25]  Joshua J. Bloch.  How to design a good API and why it matters , 2006, OOPSLA '06.

[26]  Ebba Þóra Hvannberg,et al.  Heuristic evaluation: Comparing ways of finding and reporting usability problems , 2007, Interact. Comput..

[27]  Anas N. Al-Rabadi,et al.  A comparison of modified reconstructability analysis and Ashenhurst‐Curtis decomposition of Boolean functions , 2004 .

[28]  Bertrand Meyer,et al.  An Empirical Study of API Usability , 2013, 2013 ACM / IEEE International Symposium on Empirical Software Engineering and Measurement.

[29]  Antonio Vallecillo,et al.  Measuring the usability of software components , 2006, J. Syst. Softw..

[30]  eva Kühn,et al.  Influencing Factors on the Usability of API Classes and Methods , 2012, 2012 IEEE 19th International Conference and Workshops on Engineering of Computer-Based Systems.

[31]  Avinash C. Kak,et al.  Some structural measures of API usability , 2015, Softw. Pract. Exp..