Do Programmers do Change Impact Analysis in Debugging?

Abstract“Change Impact Analysis” is the process of determining the consequences of a modification to software. In theory, change impact analysis should be done during software maintenance, to make sure changes do not introduce new bugs. Many approaches and techniques are proposed to help programmers do change impact analysis automatically. However, it is still an open question whether and how programmers do change impact analysis. In this paper, we conducted two studies, one in-depth study and one breadth study. For the in-depth study, we recorded videos of nine professional programmers repairing two bugs for two hours. For the breadth study, we surveyed 35 professional programmers using an online system. We found that the programmers in our studies did static change impact analysis before they made changes by using IDE navigational functionalities, and they did dynamic change impact analysis after they made changes by running the programs. We also found that they did not use any change impact analysis tools.

[1]  Alessandro Orso,et al.  Leveraging field data for impact analysis and regression testing , 2003, ESEC/FSE-11.

[2]  Mohammed Aziz Parande,et al.  A longitudinal analysis of the dependency concentration in smaller modules for open-source software products , 2010, 2010 IEEE International Conference on Software Maintenance.

[3]  Andreas Zeller,et al.  Mining version histories to guide software changes , 2005, Proceedings. 26th International Conference on Software Engineering.

[4]  Gail C. Murphy,et al.  Asking and Answering Questions during a Programming Change Task , 2008, IEEE Transactions on Software Engineering.

[5]  Bogdan Dit,et al.  Integrated impact analysis for managing software changes , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[6]  Ben Shneiderman,et al.  Software psychology: Human factors in computer and information systems (Winthrop computer systems series) , 1980 .

[7]  Shawn A. Bohner,et al.  A graph traceability approach for software change impact analysis , 1996 .

[8]  Suzette Person,et al.  A change impact analysis to characterize evolving program behaviors , 2012, 2012 28th IEEE International Conference on Software Maintenance (ICSM).

[9]  Shane McIntosh,et al.  Mining Co-change Information to Understand When Build Changes Are Necessary , 2014, 2014 IEEE International Conference on Software Maintenance and Evolution.

[10]  Rudolf Ramler,et al.  Improving Manual Change Impact Analysis with Tool Support: A Study in an Industrial Project , 2015, SWQD.

[11]  Collin McMillan,et al.  An Empirical Study of the Effects of Expert Knowledge on Bug Reports , 2014, 2014 IEEE International Conference on Software Maintenance and Evolution.

[12]  Brad A. Myers,et al.  An Exploratory Study of How Developers Seek, Relate, and Collect Relevant Information during Software Maintenance Tasks , 2006, IEEE Transactions on Software Engineering.

[13]  Lionel C. Briand,et al.  Using coupling measurement for impact analysis in object-oriented systems , 1999, Proceedings IEEE International Conference on Software Maintenance - 1999 (ICSM'99). 'Software Maintenance for Business Change' (Cat. No.99CB36360).

[14]  Aditya K. Ghose,et al.  Automated change impact analysis for agent systems , 2011, 2011 27th IEEE International Conference on Software Maintenance (ICSM).

[15]  Thomas D. LaToza,et al.  Developers ask reachability questions , 2010, 2010 ACM/IEEE 32nd International Conference on Software Engineering.

[16]  James D. Herbsleb,et al.  Program comprehension as fact finding , 2007, ESEC-FSE '07.

[17]  Arun Lakhotia,et al.  Understanding someone else's code: Analysis of experiences , 1993, J. Syst. Softw..

[18]  Roland H. C. Yap,et al.  Comprehending module dependencies and sharing , 2010, 2010 ACM/IEEE 32nd International Conference on Software Engineering.

[19]  Tibor Gyimóthy,et al.  Using information retrieval based coupling measures for impact analysis , 2009, Empirical Software Engineering.

[20]  Hareton K. N. Leung,et al.  A survey of code‐based change impact analysis techniques , 2013, Softw. Test. Verification Reliab..

[21]  Gerardo Canfora,et al.  Fine grained indexing of software repositories to support impact analysis , 2006, MSR '06.

[22]  Hausi A. Müller,et al.  How do program understanding tools affect how programmers understand programs? , 2000, Sci. Comput. Program..

[23]  Lefteris Angelis,et al.  An Empirical Study on Views of Importance of Change Impact Analysis Issues , 2008, IEEE Transactions on Software Engineering.

[24]  Dongmei Zhang,et al.  How do software engineers understand code changes?: an exploratory study in industry , 2012, SIGSOFT FSE.

[25]  Mark Guzdial,et al.  Learning on the job: characterizing the programming knowledge and learning strategies of web designers , 2010, CHI.

[26]  Scott R. Klemmer,et al.  Hacking, Mashing, Gluing: Understanding Opportunistic Design , 2008, IEEE Pervasive Computing.

[27]  C MurphyGail,et al.  Asking and Answering Questions during a Programming Change Task , 2008 .

[28]  Mithun Acharya,et al.  Practical change impact analysis based on static program slicing for industrial software systems , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[29]  Alfred V. Aho,et al.  Do Crosscutting Concerns Cause Defects? , 2008, IEEE Transactions on Software Engineering.

[30]  Margaret-Anne D. Storey,et al.  Theories, tools and research methods in program comprehension: past, present and future , 2006, Software Quality Journal.

[31]  Tibor Gyimóthy,et al.  The Dynamic Function Coupling Metric and Its Use in Software Evolution , 2007, 11th European Conference on Software Maintenance and Reengineering (CSMR'07).

[32]  Andreas Zeller,et al.  Why Programs Fail: A Guide to Systematic Debugging , 2005 .

[33]  Martin P. Robillard,et al.  How effective developers investigate source code: an exploratory study , 2004, IEEE Transactions on Software Engineering.

[34]  Jerod W. Wilkerson.  A software change impact analysis taxonomy , 2012, 2012 28th IEEE International Conference on Software Maintenance (ICSM).

[35]  Rachel K. E. Bellamy,et al.  How Programmers Debug, Revisited: An Information Foraging Theory Perspective , 2013, IEEE Transactions on Software Engineering.

[36]  Hausi A. Müller,et al.  Requirements of Software Visualization Tools: A Literature Survey , 2007, 2007 4th IEEE International Workshop on Visualizing Software for Understanding and Analysis.

[37]  Dalton Serey Guerrero,et al.  On the Precision and Accuracy of Impact Analysis Techniques , 2008, Seventh IEEE/ACIS International Conference on Computer and Information Science (icis 2008).

[38]  Václav Rajlich,et al.  Variable granularity for improving precision of impact analysis , 2009, 2009 IEEE 17th International Conference on Program Comprehension.

[39]  Rainer Koschke,et al.  How do professional developers comprehend software? , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[40]  J. Altmann,et al.  Observational study of behavior: sampling methods. , 1974, Behaviour.

[41]  Rudolf K. Keller,et al.  Software visualization tools: survey and analysis , 2001, Proceedings 9th International Workshop on Program Comprehension. IWPC 2001.

[42]  Ruven E. Brooks,et al.  Towards a Theory of the Comprehension of Computer Programs , 1983, Int. J. Man Mach. Stud..

[43]  Steffen Lehnert,et al.  A taxonomy for software change impact analysis , 2011, IWPSE-EVOL '11.

[44]  Frank Tip,et al.  Chianti: a tool for change impact analysis of java programs , 2004, OOPSLA.

[45]  Thomas D. LaToza,et al.  Maintaining mental models: a study of developer work habits , 2006, ICSE.

[46]  Denys Poshyvanyk,et al.  Integrating conceptual and logical couplings for change impact analysis in software , 2013, Empirical Software Engineering.

[47]  Andreas Zeller.  Chapter 6 – Scientific Debugging , 2006 .

[48]  Razvan C. Bunescu,et al.  Learning to rank relevant files for bug reports using domain knowledge , 2014, SIGSOFT FSE.

[49]  Per Runeson,et al.  Guidelines for conducting and reporting case study research in software engineering , 2009, Empirical Software Engineering.

[50]  Philip J. Guo,et al.  Opportunistic programming: how rapid ideation and prototyping occur in practice , 2008, WEUSE@ICSE.

[51]  David F. Redmiles,et al.  An empirical study of software developers' management of dependencies and changes , 2008, 2008 ACM/IEEE 30th International Conference on Software Engineering.

[52]  Collin McMillan,et al.  Portfolio: finding relevant functions and their usage , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[53]  Andreas Zeller,et al.  Mining Version Histories to Guide Software Changes , 2004 .

[54]  Lori L. Pollock,et al.  Integrating Influence Mechanisms into Impact Analysis for Increased Precision , 2006, 2006 22nd IEEE International Conference on Software Maintenance.

[55]  Gregg Rothermel,et al.  Whole program path-based dynamic impact analysis , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..

[56]  Philip J. Guo,et al.  Two studies of opportunistic programming: interleaving web foraging, learning, and writing code , 2009, CHI.

[57]  A. Orso,et al.  Efficient and precise dynamic impact analysis using execute-after sequences , 2005, Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005..