Predicting Future Developer Behavior in the IDE Using Topic Models

While early software command recommender systems drew negative user reaction, recent studies show that users of unusually complex applications will accept and utilize command recommendations. Given this new interest, more than a decade after first attempts, both the recommendation generation (backend) and the user experience (frontend) should be revisited. In this work, we focus on recommendation generation. One shortcoming of existing command recommenders is that algorithms focus primarily on mirroring the short-term past,—i.e., assuming that a developer who is currently debugging will continue to debug endlessly. We propose an approach to improve on the state of the art by modeling future task context to make better recommendations to developers. That is, the approach can predict that a developer who is currently debugging may continue to debug OR may edit their program. To predict future development commands, we applied Temporal Latent Dirichlet Allocation, a topic model used primarily for natural language, to software development interaction data (i.e., command streams). We evaluated this approach on two large interaction datasets for two different IDEs, Microsoft Visual Studio and ABB Robot Studio. Our evaluation shows that this is a promising approach for both predicting future IDE commands and producing empirically-interpretable observations.

[1]  Tovi Grossman,et al.  CommunityCommands: command recommendations for software applications , 2009, UIST '09.

[2]  Johannes Schneider,et al.  Mining Sequences of Developer Interactions in Visual Studio for Usage Smells , 2017, IEEE Transactions on Software Engineering.

[3]  Foutse Khomh,et al.  Noises in Interaction Traces Data and Their Impact on Previous Research Studies , 2015, 2015 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM).

[4]  Brad A. Myers,et al.  Capturing and analyzing low-level events from the code editor , 2011, PLATEAU '11.

[5]  Jaideep Srivastava,et al.  Web usage mining: discovery and applications of usage patterns from Web data , 2000, SKDD.

[6]  Michael Kölling,et al.  The BlueJ System and its Pedagogy , 2003, Comput. Sci. Educ..

[7]  Camille Salinesi,et al.  Unsupervised discovery of intentional process models from event logs , 2014, MSR 2014.

[8]  P. Donnelly,et al.  Inference of population structure using multilocus genotype data. , 2000, Genetics.

[9]  Andrew P. Black,et al.  How We Refactor, and How We Know It , 2012, IEEE Trans. Software Eng..

[10]  Andrew McCallum,et al.  Rethinking LDA: Why Priors Matter , 2009, NIPS.

[11]  Tovi Grossman,et al.  Deploying CommunityCommands: A Software Command Recommender System Case Study , 2014, AI Mag..

[12]  Emerson Murphy-Hill,et al.  Improving software developers' fluency by recommending development environment commands , 2012, SIGSOFT FSE.

[13]  Mik Kersten,et al.  Mylar: a degree-of-interest model for IDEs , 2005, AOSD '05.

[14]  Brian P. Bailey,et al.  Understanding and developing models for detecting and differentiating breakpoints during interactive tasks , 2007, CHI.

[15]  Andrei Chis,et al.  Towards moldable development tools , 2015, PLATEAU@SPLASH.

[16]  Sarah Nadi,et al.  FeedBaG: An interaction tracker for Visual Studio , 2016, 2016 IEEE 24th International Conference on Program Comprehension (ICPC).

[17]  Zhendong Su,et al.  On the naturalness of software , 2012, ICSE 2012.

[18]  Mik Kersten,et al.  How are Java software developers using the Elipse IDE? , 2006, IEEE Software.

[19]  Gilbert Hamann,et al.  Abstracting Execution Logs to Execution Events for Enterprise Applications (Short Paper) , 2008, 2008 The Eighth International Conference on Quality Software.

[20]  Michele Lanza,et al.  I know what you did last summer: an investigation of how developers spend their time , 2015, ICPC '15.

[21]  Ralph E. Johnson,et al.  Alternate refactoring paths reveal usability problems , 2014, ICSE.

[22]  Stas Negara,et al.  Mining fine-grained code changes to detect unknown change patterns , 2014, ICSE.

[23]  Lori L. Pollock,et al.  A field study of how developers locate features in source code , 2016, Empirical Software Engineering.

[24]  Lori L. Pollock,et al.  Interactive Exploration of Developer Interaction Traces using a Hidden Markov Model , 2016, 2016 IEEE/ACM 13th Working Conference on Mining Software Repositories (MSR).

[25]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[26]  Wil M. P. van der Aalst,et al.  Process Mining - Discovery, Conformance and Enhancement of Business Processes , 2011 .

[27]  Sarah Smith Heckman,et al.  Bespoke tools: adapted to the concepts developers know , 2015, ESEC/SIGSOFT FSE.

[28]  Thomas Fritz,et al.  Collecting and Processing Interaction Data for Recommendation Systems , 2014, Recommendation Systems in Software Engineering.

[29]  Clifford Nass,et al.  The media equation - how people treat computers, television, and new media like real people and places , 1996 .

[30]  Tovi Grossman,et al.  Deploying community commands: a software command recommender system case study , 2014, AAAI 2014.

[31]  Martin P. Robillard,et al.  The Influence of the Task on Programmer Behaviour , 2011, 2011 IEEE 19th International Conference on Program Comprehension.

[32]  J. Lafferty,et al.  Mixed-membership models of scientific publications , 2004, Proceedings of the National Academy of Sciences of the United States of America.