Asking and Answering Questions during a Programming Change Task

Little is known about the specific kinds of questions programmers ask when evolving a code base and how well existing tools support those questions. To better support the activity of programming, answers are needed to three broad research questions: 1) What does a programmer need to know about a code base when evolving a software system? 2) How does a programmer go about finding that information? 3) How well do existing tools support programmers in answering those questions? We undertook two qualitative studies of programmers performing change tasks to provide answers to these questions. In this paper, we report on an analysis of the data from these two user studies. This paper makes three key contributions. The first contribution is a catalog of 44 types of questions programmers ask during software evolution tasks. The second contribution is a description of the observed behavior around answering those questions. The third contribution is a description of how existing deployed and proposed tools do, and do not, support answering programmers' questions.

[1]  Amy J. Ko,et al.  Eliciting design requirements for maintenance-oriented IDEs: a detailed study of corrective and perfective maintenance tasks , 2005, Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005..

[2]  Xiangyu Zhang,et al.  Cost effective dynamic program slicing , 2004, PLDI '04.

[3]  Marc H. Brown,et al.  Zeus: a system for algorithm animation and multi-view editing , 1991, Proceedings 1991 IEEE Workshop on Visual Languages.

[4]  Harry M. Sneed,et al.  Partial comprehension of complex programs (enough to perform maintenance) , 1998, Proceedings. 6th International Workshop on Program Comprehension. IWPC'98 (Cat. No.98TB100242).

[5]  J. E. Sammet,et al.  Software psychology: human factors in computer and information systems , 1983, SGCH.

[6]  Ruven E. Brooks,et al.  Towards a Theory of the Comprehension of Computer Programs , 1983, Int. J. Man Mach. Stud..

[7]  Jim Q. Ning,et al.  Knowledge-based program analysis , 1990, IEEE Software.

[8]  Cornelia Boldyreff,et al.  Using application understanding to support impact analysis , 1998, J. Softw. Maintenance Res. Pract..

[9]  Andrew Walenstein,et al.  Theory-based analysis of cognitive support in software comprehension tools , 2002, Proceedings 10th International Workshop on Program Comprehension.

[10]  James D. Herbsleb,et al.  Representing knowledge in requirements engineering: an empirical study of what software engineers need to know , 1993, [1993] Proceedings of the IEEE International Symposium on Requirements Engineering.

[11]  Anneliese Amschler Andrews,et al.  Program Comprehension During Software Maintenance and Evolution , 1995, Computer.

[12]  Aziz Deraman,et al.  Case study: reconnaissance techniques to support feature location using RECON2 , 2003, Tenth Asia-Pacific Software Engineering Conference, 2003..

[13]  Kris De Volder,et al.  Navigating and querying code without getting lost , 2003, AOSD '03.

[14]  Daniel Jackson.  ASPECT: an economical bug-detector , 1991, [1991 Proceedings] 13th International Conference on Software Engineering.

[15]  Scott Henningen,et al.  Retrieving software objects in an example-based programming environment , 1991, SIGIR '91.

[16]  Janice Singer,et al.  An examination of software engineering work practices , 1997, CASCON.

[17]  Shinji Kusumoto,et al.  CCFinder: A Multilinguistic Token-Based Code Clone Detection System for Large Scale Source Code , 2002, IEEE Trans. Software Eng..

[18]  Elliot Soloway,et al.  Designing documentation to compensate for delocalized plans , 1988, CACM.

[19]  David R. Karger,et al.  Relo: Helping Users Manage Context during Interactive Exploratory Visualization of Large Codebases , 2005, Visual Languages and Human-Centric Computing (VL/HCC'06).

[20]  N. Hoffart.  Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory , 2000 .

[21]  Wojtek Kozaczynski,et al.  A Knowledge-based Approach To Software System Understanding , 1991, Proceedings., 6th Annual Knowledge-Based Software Engineering Conference.

[22]  R. Stallman.  EMACS the extensible, customizable self-documenting display editor , 1981, SIGPLAN SIGOA Symposium on Text Manipulation.

[23]  Ruven E. Brooks.  Towards a theory of the cognitive processes in computer programming , 1999, Int. J. Hum. Comput. Stud..

[24]  Scott P. Robertson,et al.  Program comprehension beyond the line , 1990, IFIP TC13 International Conference on Human-Computer Interaction.

[25]  Jacob L. Cybulski,et al.  A hypertext based software-engineering environment , 1992, IEEE Software.

[26]  Namir C. Shammas.  Windows batch file programming , 1995 .

[27]  Brian W. Kernighan,et al.  The C Programming Language , 1978 .

[28]  Marian Petre,et al.  Usability Analysis of Visual Programming Environments: A 'Cognitive Dimensions' Framework , 1996, J. Vis. Lang. Comput..

[29]  W. Lewis Johnson,et al.  Interactive Explanation of Software Systems , 2004, Automated Software Engineering.

[30]  Hausi A. Müller,et al.  A cognitive framework for describing and evaluating software exploration tools , 1998 .

[31]  Andrés Moreno.  Algorithm Animation , 2006, Human-Centered Visualization Environments.

[32]  Maarten van Someren,et al.  The Think Aloud Method: A Practical Guide to Modelling Cognitive Processes , 1994 .

[33]  Robert R. Henry,et al.  The University of Washington illustrating compiler , 1990, PLDI '90.

[34]  Alexander E. Quilici,et al.  Constraint-based design recovery for software reengineering: theory and experiments , 1997 .

[35]  Stanley Letovsky,et al.  Cognitive processes in program comprehension , 1986, J. Syst. Softw..

[36]  Ron Petrusha,et al.  ASP in a nutshell , 1999 .

[37]  N. Pennington.  Stimulus structures and mental representations in expert comprehension of computer programs , 1987, Cognitive Psychology.

[38]  Larry Masinter,et al.  The Interlisp Programming Environment , 1981, Computer.

[39]  Mary Czerwinski,et al.  Towards understanding programs through wear-based filtering , 2005, SoftVis '05.

[40]  Ben Shneiderman,et al.  Syntactic/semantic interactions in programmer behavior: A model and experimental results , 1979, International Journal of Computer & Information Sciences.

[41]  A. von Mayrhauser,et al.  From code understanding needs to reverse engineering tool capabilities , 1993, Proceedings of 6th International Workshop on Computer-Aided Software Engineering.

[42]  Gail C. Murphy,et al.  Using Visual Momentum to Explain Disorientation in the Eclipse IDE , 2006, Visual Languages and Human-Centric Computing (VL/HCC'06).

[43]  Norman Wilde,et al.  Early field experience with the Software Reconnaissance technique for program comprehension , 1996, 1996 Proceedings of International Conference on Software Maintenance.

[44]  Mark Harman,et al.  Code extraction algorithms which unify slicing and concept assignment , 2002, Ninth Working Conference on Reverse Engineering, 2002. Proceedings..

[45]  Martin P. Robillard,et al.  How effective developers investigate source code: an exploratory study , 2004, IEEE Transactions on Software Engineering.

[46]  Andrew Walenstein,et al.  COGNITIVE SUPPORT IN SOFTWARE ENGINEERING TOOLS: A DISTRIBUTED COGNITION FRAMEWORK , 2002 .

[47]  Václav Rajlich,et al.  Case study of feature location using dependence graph , 2000, Proceedings IWPC 2000. 8th International Workshop on Program Comprehension.

[48]  G. A. Miller.  THE PSYCHOLOGICAL REVIEW THE MAGICAL NUMBER SEVEN, PLUS OR MINUS TWO: SOME LIMITS ON OUR CAPACITY FOR PROCESSING INFORMATION 1 , 1956 .

[49]  Emden R. Gansner,et al.  Using automatic clustering to produce high-level system organizations of source code , 1998, Proceedings. 6th International Workshop on Program Comprehension. IWPC'98 (Cat. No.98TB100242).

[50]  Guy L. Steele,et al.  The Java Language Specification , 1996 .

[51]  Gail C. Murphy,et al.  Questions programmers ask during software evolution tasks , 2006, SIGSOFT '06/FSE-14.

[52]  Daniel C. Halbert,et al.  The Trellis programming environment , 1987, OOPSLA '87.

[53]  Elliotte Rusty Harold,et al.  XML in a Nutshell , 2001 .

[54]  Jack L. Hursch,et al.  SQL, the structured query language , 1991 .

[55]  Hausi A. Müller,et al.  Rigi: a system for programming-in-the-large , 1988, Proceedings. [1989] 11th International Conference on Software Engineering.

[56]  Kamran Sartipi,et al.  Pattern-based Software Architecture Recovery , 2003 .

[57]  Margaret-Anne Storey,et al.  Manipulating And Documenting Software Structures , 1995 .

[58]  Victor R. Basili,et al.  Comparing the Effectiveness of Software Testing Strategies , 1987, IEEE Transactions on Software Engineering.

[59]  Jiajie Zhang,et al.  The Nature of External Representations in Problem Solving , 1997, Cogn. Sci..

[60]  Nicolas Edwin Gold,et al.  Hypothesis-based concept assignment to support software maintenance , 2001, Proceedings IEEE International Conference on Software Maintenance. ICSM 2001.

[61]  Bjarne Stroustrup,et al.  The C++ Programming Language, Second Edition , 1991 .

[62]  Norman Wilde,et al.  A comparison of methods for locating features in legacy software , 2003, J. Syst. Softw..

[63]  Jiajie Zhang,et al.  Representations in Distributed Cognitive Tasks , 1994, Cogn. Sci..

[64]  Scott P. Robertson,et al.  Expert problem solving strategies for program comprehension , 1991, CHI.

[65]  J. Bain,et al.  PSYCHOLOGICAL SCIENCE Research Article How Many Variables Can Humans Process? , 2022 .

[66]  Bjarne Stroustrup,et al.  C++ Programming Language , 1986, IEEE Softw..

[67]  R. Holmes,et al.  Using structural context to recommend source code examples , 2005, Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005..

[68]  Nathanael Schärli,et al.  Traits: tools and methodology , 2004, Proceedings. 26th International Conference on Software Engineering.

[69]  Frank Tip,et al.  Chianti: a tool for change impact analysis of java programs , 2004, OOPSLA.

[70]  John Steven,et al.  jRapture: A Capture/Replay tool for observation-based testing , 2000, ISSTA '00.

[71]  Blaine A. Price,et al.  A Principled Taxonomy of Software Visualization , 1993, J. Vis. Lang. Comput..

[72]  Jörg M. Haake,et al.  Hypermedia and cognition: designing for comprehension , 1995, CACM.

[73]  Ted J. Biggerstaff,et al.  Program understanding and the concept assignment problem , 1994, CACM.

[74]  David W. Binkley,et al.  Program slicing , 2008, 2008 Frontiers of Software Maintenance.

[75]  Brad A. Myers,et al.  Designing the whyline: a debugging interface for asking questions about program behavior , 2004, CHI.

[76]  Mik Kersten,et al.  How are Java software developers using the Elipse IDE? , 2006, IEEE Software.

[77]  Kate Ehrlich,et al.  Empirical Studies of Programming Knowledge , 1984, IEEE Transactions on Software Engineering.

[78]  Steven P. Reiss,et al.  Connecting tools using message passing in the Field environment , 1990, IEEE Software.

[79]  Laurie A. Williams,et al.  Strengthening the Case for Pair Programming , 2000, IEEE Softw..

[80]  Brad A. Myers,et al.  An Exploratory Study of How Developers Seek, Relate, and Collect Relevant Information during Software Maintenance Tasks , 2006, IEEE Transactions on Software Engineering.

[81]  Hausi A. Müller,et al.  Cognitive design elements to support the construction of a mental model during software exploration , 1999, J. Syst. Softw..

[82]  Martin P. Robillard,et al.  FEAT a tool for locating, describing, and analyzing concerns in source code , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..

[83]  Elliot Soloway,et al.  Mental models and software maintenance , 1986, J. Syst. Softw..

[84]  Sivakumar Harinath,et al.  Professional SQL Server Analysis Services 2005 with MDX , 2006 .

[85]  Hausi A. Müller,et al.  Cognitive design elements to support the construction of a mental model during software visualization , 1997, Proceedings Fifth International Workshop on Program Comprehension. IWPC'97.

[86]  Paul Dourish,et al.  Unifying artifacts and activities in a visual tool for distributed software development teams , 2004, Proceedings. 26th International Conference on Software Engineering.

[87]  Stacy Marsella,et al.  Task oriented software understanding , 1998, Proceedings 13th IEEE International Conference on Automated Software Engineering (Cat. No.98EX239).

[88]  C MurphyGail,et al.  How Are Java Software Developers Using the Eclipse IDE , 2006 .

[89]  Adele Goldberg,et al.  Smalltalk-80 - the interactive programming environment , 1984 .

[90]  Brad A. Myers,et al.  Automatic data visualization for novice Pascal programmers , 1988, [Proceedings] 1988 IEEE Workshop on Visual Languages.

[91]  Rainer Koschke,et al.  Locating Features in Source Code , 2003, IEEE Trans. Software Eng..

[92]  Sougata Mukherjea,et al.  Toward visual debugging: integrating algorithm animation capabilities within a source-level debugger , 1994, TCHI.

[93]  Robert DeLine,et al.  Information Needs in Collocated Software Development Teams , 2007, 29th International Conference on Software Engineering (ICSE'07).

[94]  David D. Woods,et al.  Visual Momentum: A Concept to Improve the Cognitive Coupling of Person and Computer , 1984, Int. J. Man Mach. Stud..

[95]  Gail C. Murphy,et al.  Conceptual module querying for software reengineering , 1998, Proceedings of the 20th International Conference on Software Engineering.

[96]  Hausi A. Müller,et al.  How do program understanding tools affect how programmers understand programs? , 1997, Proceedings of the Fourth Working Conference on Reverse Engineering.

[97]  James D. Herbsleb,et al.  Preserving knowledge in design projects: what designers need to know , 1993, INTERCHI.

[98]  Naomi Miyake,et al.  Constructive Interaction and the Iterative Process of Understanding , 1986, Cogn. Sci..

[99]  Hausi A. Müller,et al.  A reverse-engineering approach to subsystem structure identification , 1993, J. Softw. Maintenance Res. Pract..

[100]  Brian D. Fisher,et al.  Managing software change tasks: an exploratory study , 2005, 2005 International Symposium on Empirical Software Engineering, 2005..

[101]  Dennis B. Smith,et al.  Towards a framework for program understanding , 1996, WPC '96. 4th Workshop on Program Comprehension.

[102]  Kamran Sartipi,et al.  A graph pattern matching approach to software architecture recovery , 2001, Proceedings IEEE International Conference on Software Maintenance. ICSM 2001.

[103]  Daniel Jackson,et al.  A new model of program dependences for reverse engineering , 1994, SIGSOFT '94.

[104]  Frederick P. Brooks,et al.  No Silver Bullet: Essence and Accidents of Software Engineering , 1987 .

[105]  A. Strauss,et al.  The discovery of grounded theory: strategies for qualitative research aldine de gruyter , 1968 .

[106]  Tim Miller,et al.  A framework and tool support for the systematic testing of model-based specifications , 2003, TSEM.

[107]  Stéphane Ducasse,et al.  Recovering high-level views of object-oriented applications from static and dynamic information , 1999, Proceedings IEEE International Conference on Software Maintenance - 1999 (ICSM'99). 'Software Maintenance for Business Change' (Cat. No.99CB36360).