A Practical Guide to Analyzing IDE Usage Data

Integrated development environments such as Eclipse and Visual Studio provide tools and capabilities to perform tasks such as navigating among classes and methods, continuous compilation, code refactoring, automated testing, and integrated debugging, all designed to increase productivity. Instrumenting the integrated development environment to collect usage data provides a more fine-grained understanding of developers' work than was previously possible. Usage data supports analysis of how developers spend their time, what activities might benefit from greater tool support, where developers have difficulty comprehending code, and whether they are following specific practices such as test-driven development. With usage data, we expect to uncover more nuggets of how developers create mental models, how they investigate code, how they perform mini trial-and-error experiments, and what might drive productivity improvements for everyone.

[1]  Stas Negara,et al.  Is It Dangerous to Use Version Control Histories to Study Source Code Evolution? , 2012, ECOOP.

[2]  L. Shattuck,et al.  The Critical Incident Technique: 40 Years Later , 1994 .

[3]  Qiong Yang,et al.  Bmc Medical Genetics Genome-wide Association and Linkage Analyses of Hemostatic Factors and Hematological Phenotypes in the Framingham Heart Study , 2022 .

[4]  Andrew P. Black,et al.  Restructuring software with gestures , 2011, 2011 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC).

[5]  Andrew P. Black,et al.  How we refactor, and how we know it , 2009, 2009 IEEE 31st International Conference on Software Engineering.

[6]  Meiyappan Nagappan,et al.  Creating operational profiles of software systems by transforming their log files to directed cyclic graphs , 2011, TEFSE '11.

[7]  Martin P. Robillard,et al.  How effective developers investigate source code: an exploratory study , 2004, IEEE Transactions on Software Engineering.

[8]  Emerson R. Murphy-Hill,et al.  A degree-of-knowledge model to capture source code familiarity , 2010, 2010 ACM/IEEE 32nd International Conference on Software Engineering.

[9]  Prasun Dewan,et al.  Are you having difficulty? , 2010, CSCW '10.

[10]  Andrew P. Black,et al.  Breaking the barriers to successful refactoring: observations and tools for extract method , 2008, ICSE.

[11]  Mik Kersten,et al.  Mylar: a degree-of-interest model for IDEs , 2005, AOSD '05.

[12]  Stas Negara,et al.  A Compositional Paradigm of Automating Refactorings , 2013, ECOOP.

[13]  Michael W. Godfrey,et al.  An industrial case study of Coman's automated task detection algorithm: What Worked, What Didn't, and Why , 2012, 2012 28th IEEE International Conference on Software Maintenance (ICSM).

[14]  Emerson Murphy-Hill,et al.  Improving software developers' fluency by recommending development environment commands , 2012, SIGSOFT FSE.

[15]  Stas Negara,et al.  The need for richer refactoring usage data , 2011, PLATEAU '11.

[16]  Brad A. Myers,et al.  An exploratory study of backtracking strategies used by developers , 2012, 2012 5th International Workshop on Co-operative and Human Aspects of Software Engineering (CHASE).

[17]  Alberto Sillitti,et al.  Automated Identification of Tasks in Development Sessions , 2008, 2008 16th IEEE International Conference on Program Comprehension.

[18]  Spencer Rugaber,et al.  Resumption strategies for interrupted programming tasks , 2009, ICPC.

[19]  Mary Czerwinski,et al.  FASTDash: a visual dashboard for fostering awareness in software teams , 2007, CHI.

[20]  Martin P. Robillard,et al.  The Influence of the Task on Programmer Behaviour , 2011, 2011 IEEE 19th International Conference on Program Comprehension.

[21]  Brad A. Myers,et al.  Capturing and analyzing low-level events from the code editor , 2011, PLATEAU '11.

[22]  Philip M. Johnson,et al.  Beyond the Personal Software Process: Metrics collection and analysis for the differently disciplined , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..

[23]  Robert C. Williges,et al.  An Evaluation of Critical Incidents for Software Documentation Design , 1986 .

[24]  Thomas Fritz,et al.  Collecting and Processing Interaction Data for Recommendation Systems , 2014, Recommendation Systems in Software Engineering.

[25]  J. C. Flanagan.  Psychological Bulletin THE CRITICAL INCIDENT TECHNIQUE , 2022 .

[26]  Zhendong Niu,et al.  An Initial Study on Refactoring Tactics , 2012, 2012 IEEE 36th Annual Computer Software and Applications Conference.

[27]  Brad A. Myers,et al.  Visualization of fine-grained code change history , 2013, 2013 IEEE Symposium on Visual Languages and Human Centric Computing.

[28]  Victor R. Basili,et al.  The TAME Project: Towards Improvement-Oriented Software Environments , 1988, IEEE Trans. Software Eng..

[29]  Emerson R. Murphy-Hill,et al.  Reconciling manual and automatic refactoring , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[30]  Hakan Erdogmus,et al.  Operational definition and automated inference of test-driven development with Zorro , 2010, Automated Software Engineering.

[31]  Ralph E. Johnson,et al.  Alternate refactoring paths reveal usability problems , 2014, ICSE.

[32]  Gail C. Murphy,et al.  Using activity traces to characterize programming behaviour beyond the lab , 2009, 2009 IEEE 17th International Conference on Program Comprehension.

[33]  Emerson R. Murphy-Hill,et al.  Experiences gamifying developer adoption of practices and tools , 2014, ICSE Companion.

[34]  William G. Griswold,et al.  WitchDoctor: IDE support for real-time auto-completion of refactorings , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[35]  Mik Kersten,et al.  How are Java software developers using the Elipse IDE? , 2006, IEEE Software.

[36]  Ralph E. Johnson,et al.  Drag-and-drop refactoring: Intuitive and efficient program transformation , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[37]  Stas Negara,et al.  Use, disuse, and misuse of automated refactorings , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[38]  Stas Negara,et al.  A Comparative Study of Manual and Automated Refactorings , 2013, ECOOP.

[39]  Mik Kersten,et al.  Using task context to improve programmer productivity , 2006, SIGSOFT '06/FSE-14.

[40]  Mik Kersten.  Focusing knowledge work with task context , 2007 .