An Empirical Study of API Usability

Modern software development extensively involves reusing library components accessed through their Application Programming Interfaces (APIs). Usability is therefore a fundamental goal of API design, but rigorous empirical studies of API usability are still relatively uncommon. In this paper, we present the design of an API usability study which combines interview questions based on the cognitive dimensions framework, with systematic observations of programmer behavior while solving programming tasks based on ``tokens''. We also discuss the implementation of the study to assess the usability of a persistence library API (offering functionalities such as storing objects into relational databases). The study involved 25 programmers (including students, researchers, and professionals), and provided additional evidence to some critical features evidenced by related studies, such as the difficulty of finding good names for API features and of discovering relations between API types. It also discovered new issues relevant to API design, such as the impact of flexibility, and confirmed the crucial importance of accurate documentation for usability.

[1]  Brad A. Myers,et al.  The implications of method placement on API learnability , 2008, SIGSOFT '08/FSE-16.

[2]  Brad A. Myers,et al.  Mapping the Space of API Design Decisions , 2007, IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC 2007).

[3]  Martin P. Robillard,et al.  What Makes APIs Hard to Learn? Answers from Developers , 2009, IEEE Software.

[4]  K. A. Ericsson,et al.  Protocol Analysis: Verbal Reports as Data , 1984 .

[5]  Ken Arnold.  Programmers Are People, too , 2005, ACM Queue.

[6]  Brian Ellis,et al.  The Factory Pattern in API Design: A Usability Evaluation , 2007, 29th International Conference on Software Engineering (ICSE'07).

[7]  Michi Henning.  API: Design Matters , 2007, ACM Queue.

[8]  Margaret M. Burnett,et al.  Using cognitive dimensions: Advice from the trenches , 2006, J. Vis. Lang. Comput..

[9]  Forrest Shull,et al.  Investigating Reading Techniques for Object-Oriented Framework Learning , 2000, IEEE Trans. Software Eng..

[10]  Westley Weimer,et al.  Synthesizing API usage examples , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[11]  Ewan D. Tempero,et al.  Visualizing the word structure of Java class names , 2008, OOPSLA Companion.

[12]  David Lorge Parnas,et al.  Precise Documentation: The Key to Better Software , 2010, The Future of Software Engineering.

[13]  F. J. Fowler,et al.  Standardized Survey Interviewing: Minimizing Interviewer-Related Error. , 1989 .

[14]  Clay Spinuzzi,et al.  Building More Usable APIs , 1998, IEEE Softw..

[15]  Maria Kutar,et al.  Cognitive Dimensions of Notations: Design Tools for Cognitive Technology , 2001, Cognitive Technology.

[16]  Bertrand Meyer,et al.  Object-oriented software construction (2nd ed.) , 1997 .

[17]  Bo Leuf,et al.  The Wiki Way: Quick Collaboration on the Web , 2001 .

[18]  Martin P. Robillard,et al.  Asking and answering questions about unfamiliar APIs: An exploratory study , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[19]  Jeffrey Stylos,et al.  Informing API Design through Usability Studies of API Design Choices: A Research Abstract , 2006, Visual Languages and Human-Centric Computing (VL/HCC'06).

[20]  Bjarne Stroustrup,et al.  Programming: Principles and Practice Using C++ , 2008 .

[21]  Robert B. Watson.  Improving software API usability through text analysis: A case study , 2009, 2009 IEEE International Professional Communication Conference.

[22]  Guy L. Steele,et al.  The Java Language Specification , 1996 .

[23]  Jaroslav Tulach.  Practical API Design: Confessions of a Java Framework Architect , 2008 .

[24]  James D. Herbsleb,et al.  Improving API documentation usability with knowledge pushing , 2009, 2009 IEEE 31st International Conference on Software Engineering.

[25]  Brad A. Myers,et al.  Six Learning Barriers in End-User Programming Systems , 2004, 2004 IEEE Symposium on Visual Languages - Human Centric Computing.

[26]  Ralph Johnson,et al.  design patterns elements of reusable object oriented software , 2019 .

[27]  Guy L. Steele,et al.  The Java Language Specification, Java SE 8 Edition , 2013 .

[28]  Marco Piccioni,et al.  A Seamless Framework for Object-Oriented Persistence in Presence of Class Schema Evolution , 2012 .

[29]  S. M. Robertson.  Postsecondary Education & Autism: Developing an Online Community , 2007 .

[30]  Martin Reddy,et al.  API Design for C , 2011 .

[31]  Robert DeLine,et al.  Information Needs in Collocated Software Development Teams , 2007, 29th International Conference on Software Engineering (ICSE'07).

[32]  Steven Clarke.  Evaluating a new programming language , 2001, PPIG.

[33]  Brad Abrams,et al.  Framework Design Guidelines: Conventions, Idioms, and Patterns for Reusable .NET Libraries , 2005 .

[34]  Jeffrey Stylos,et al.  Usability Implications of Requiring Parameters in Objects' Constructors , 2007, 29th International Conference on Software Engineering (ICSE'07).

[35]  簡聰富,et al.  物件導向軟體之架構(Object-Oriented Software Construction)探討 , 1989 .

[36]  Ted Boren,et al.  Thinking aloud: reconciling theory and practice , 2000 .