A graphical user interface for presenting integrated development environment command recommendations: Design, evaluation, and implementation

Abstract Context A set of algorithms exist to generate integrated development environment (IDE) command recommendations. The recommendations are aimed at improving software developer’s interaction with an IDE. Even though the interface is a critical element of every recommender system, we are not aware of any existing graphical user interface to present such recommendations. Objective This paper describes and evaluates a novel design of a graphical user interface to recommend commands within an IDE. The interface contains a description of the suggested command, an explanation of why the command is recommended, and a command usage example. Method The proposed design is based on the analysis of guidelines identified in the literature. Its acceptance and usability were evaluated through a user study with 36 software developers and semi-structured interviews with 11 software developers. Results The results indicate that the suggested interface is well accepted, but it can be further improved. Through the interviews and the implementation of the interface, we identified a series of requirements important for the development of future IDE command recommender systems. Conclusions This paper shows that a convenient graphical user interface is critical to achieve high acceptance of IDE command recommendations. Our work also illustrates steps useful for undertaking user studies related to IDE command recommendations in a practical setting without human intervention. A future step is to evaluate the interface within the business environment, where recommendations are generated and presented in an IDE used by practicing software developers as part of their normal workday.

[1]  S. Dreyfus,et al.  A Five-Stage Model of the Mental Activities Involved in Directed Skill Acquisition , 1980 .

[2]  Judith Masthoff,et al.  Explaining Recommendations: Design and Evaluation , 2015, Recommender Systems Handbook.

[3]  Katharina Reinecke,et al.  Choice Architecture for Human-Computer Interaction , 2014, Found. Trends Hum. Comput. Interact..

[4]  Walt Scacchi.  Understanding Software Productivity: towards a Knowledge-Based Approach , 1991, Int. J. Softw. Eng. Knowl. Eng..

[5]  Tovi Grossman,et al.  Design and evaluation of a command recommendation system for software applications , 2011, TCHI.

[6]  Wolfgang Menz,et al.  Das Experteninterview: Theorie, Methode, Anwendung , 2002 .

[7]  Jakob Nielsen,et al.  Improving a human-computer dialogue , 1990, CACM.

[8]  Sharan B. Merriam,et al.  Qualitative Research: A Guide to Design and Implementation , 2009 .

[9]  Andrea Bunt,et al.  Exploring Personalized Command Recommendations based on Information Found in Web Documentation , 2015, IUI.

[10]  Martin Bichler,et al.  Design science in information systems research , 2006, Wirtschaftsinf..

[11]  Gabriele Bavota,et al.  Too Long; Didn't Watch! Extracting Relevant Fragments from Software Development Video Tutorials , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE).

[12]  Gail C. Murphy,et al.  What to Learn Next: Recommending Commands in a Feature-Rich Environment , 2015, 2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA).

[13]  J. B. Brooke,et al.  SUS: A 'Quick and Dirty' Usability Scale , 1996 .

[14]  Emerson R. Murphy-Hill.  Continuous social screencasting to facilitate software tool discovery , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[15]  Emerson Murphy-Hill,et al.  Improving software developers' fluency by recommending development environment commands , 2012, SIGSOFT FSE.

[16]  Claes Wohlin,et al.  Experimentation in Software Engineering , 2012, Springer Berlin Heidelberg.

[17]  Bob Walraet.  A Discipline of Software Engineering , 1992 .

[18]  Salvatore T. March,et al.  Design and natural science research on information technology , 1995, Decis. Support Syst..

[19]  Francesco Ricci,et al.  Development Tools Usage Inside Out , 2016, XP.

[20]  Jennifer Anderson-Meger.  Why Do I Need Research and Theory?: A Guide for Social Workers , 2016 .

[21]  Rashmi R. Sinha,et al.  The role of transparency in recommender systems , 2002, CHI Extended Abstracts.

[22]  Victor R. Basili,et al.  A Methodology for Collecting Valid Software Engineering Data , 1984, IEEE Transactions on Software Engineering.

[23]  Hinrich Schütze,et al.  Introduction to information retrieval , 2008 .

[24]  Dieter Rombach,et al.  Aligning Organizations Through Measurement: The Gqm+strategies Approach , 2014 .

[25]  Gail C. Murphy,et al.  Improving program navigation with an active help system , 2010, CASCON.

[26]  Anselm L. Strauss,et al.  Basics of qualitative research : techniques and procedures for developing grounded theory , 1998 .

[27]  Paul Resnick,et al.  Recommender systems , 1997, CACM.

[28]  Gordon B. Davis,et al.  User Acceptance of Information Technology: Toward a Unified View , 2003, MIS Q..

[29]  Andrea Bunt,et al.  Exploring User Attitudes Towards Different Approaches to Command Recommendation in Feature-Rich Software , 2016, IUI.

[30]  Tovi Grossman,et al.  A survey of software learnability: metrics, methodologies and guidelines , 2009, CHI.

[31]  Gerhard Fischer,et al.  User Modeling in Human–Computer Interaction , 2001, User Modeling and User-Adapted Interaction.

[32]  Mouzhi Ge,et al.  How should I explain? A comparison of different explanation types for recommender systems , 2014, Int. J. Hum. Comput. Stud..

[33]  Tural Gurbanov,et al.  Context-aware integrated development environment command recommender systems , 2017, 2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE).

[34]  R. Yin.  Case Study Research: Design and Methods , 1984 .

[35]  Manfred Tscheligi,et al.  Paper prototyping - what is it good for?: a comparison of paper- and computer-based low-fidelity prototyping , 2003, CHI Extended Abstracts.

[36]  Francesco Ricci,et al.  Recommender Systems , 2007, 2007 40th Annual Hawaii International Conference on System Sciences (HICSS'07).

[37]  Francesco Ricci,et al.  GUI Design for IDE Command Recommendations , 2017, IUI.

[38]  Marten Schönherr,et al.  Artifact Types in Information Systems Design Science - A Literature Review , 2010, DESRIST.

[39]  Alain Abran,et al.  The Guide to the Software Engineering Body of Knowledge , 1999, IEEE Softw..

[40]  Tovi Grossman,et al.  CommunityCommands: command recommendations for software applications , 2009, UIST '09.

[41]  Lior Rokach,et al.  Recommender Systems: Introduction and Challenges , 2015, Recommender Systems Handbook.

[42]  Jakob Nielsen,et al.  Paper versus computer implementations as mockup scenarios for heuristic evaluation , 1990, INTERACT.

[43]  M. Zanker,et al.  An empirical study on the persuasiveness of fact-based explanations for recommender systems , 2014 .

[44]  Herbert A. Simon,et al.  The Sciences of the Artificial , 1970 .

[45]  Francesco Ricci,et al.  A context model for IDE-based recommendation systems , 2017, J. Syst. Softw..

[46]  Frank Linton,et al.  Recommender Systems for Learning: Building User and Expert Models through Long-Term Observation of Application Use , 2000, User Modeling and User-Adapted Interaction.

[47]  John Riedl,et al.  Explaining collaborative filtering recommendations , 2000, CSCW '00.

[48]  U. Flick.  An Introduction to Qualitative Research , 1998 .

[49]  Dustin Campbell,et al.  Designing refactoring tools for developers , 2008, WRT '08.