Framework for measuring program comprehension

Feature-oriented software development is a promising paradigm to implement variable software. One advantage is that crosscutting concerns can be modularized, which in turn has a positive effect on program comprehension. However, benefits for program comprehension are mostly based on plausibility arguments and theoretical discussions. This is not sufficient, since program comprehension is an internal cognitive process that requires empirical evaluation. Up to today, there are only few empirical studies that evaluate the effect of feature-oriented software development on program comprehension. With our work, we aim at filling this gap and providing sound empirical evidence about the effect of featureoriented software development on program comprehension. Keywords-program comprehension, feature-oriented software development, variable software

[1]  Mira Mezini,et al.  Variability management with feature-oriented programming and aspects , 2004, SIGSOFT '04/FSE-12.

[2]  Donovan A. Schneider,et al.  The Gamma Database Machine Project , 1990, IEEE Trans. Knowl. Data Eng..

[3]  Padmal Vitharana,et al.  Computer-Mediated Group Support, Anonymity, and the Software Inspection Process: An Empirical Investigation , 2003, IEEE Trans. Software Eng..

[4]  R. Peterson.  Constructing Effective Questionnaires , 1999 .

[5]  Cecília M. F. Rubira,et al.  Explicit exception handling variability in component-based product line architectures , 2008, WEH '08.

[6]  Marco Torchiano.  Empirical assessment of UML static object diagrams , 2004, Proceedings. 12th IEEE International Workshop on Program Comprehension, 2004..

[7]  Amela Karahasanovic,et al.  A survey of controlled experiments in software engineering , 2005, IEEE Transactions on Software Engineering.

[8]  Stefan Hanenberg,et al.  An experiment about static and dynamic type systems: doubts about the positive impact of static type systems on development time , 2010, OOPSLA.

[9]  Gail C. Murphy,et al.  Design pattern rationale graphs: linking design to source , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..

[10]  Walter F. Tichy,et al.  Hints for Reviewing Empirical Work in Software Engineering , 2000, Empirical Software Engineering.

[11]  Carlos José Pereira de Lucena,et al.  Quantifying the Effects of Aspect-Oriented Programming: A Maintenance Study , 2006, 2006 22nd IEEE International Conference on Software Maintenance.

[12]  Jason W. Osborne,et al.  Best practices in exploratory factor analysis: four recommendations for getting the most from your analysis. , 2005 .

[13]  Peter M. Bentler,et al.  Practical Issues in Structural Modeling , 1987 .

[14]  Thomas Thelin,et al.  Team-based fault content estimation in the software inspection process , 2004, Proceedings. 26th International Conference on Software Engineering.

[15]  W. Mackenzie,et al.  The Management and the Worker , 2008 .

[16]  Maarten van Someren,et al.  The Think Aloud Method: A Practical Guide to Modelling Cognitive Processes , 1994 .

[17]  Walter F. Tichy,et al.  A Controlled Experiment Comparing the Maintainability of Programs Designed with and without Design Patterns—A Replication in a Real Programming Environment , 2004, Empirical Software Engineering.

[18]  Sven Apel,et al.  View infinity: a zoomable interface for feature-oriented software development , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[19]  David A. Fisher,et al.  Introducing Ada , 1980, ACM '80.

[20]  Michal Antkiewicz,et al.  Mapping features to models: a template approach based on superimposed variants , 2005, GPCE'05.

[21]  Spencer Rugaber,et al.  Programmer information needs after memory failure , 2012, 2012 20th IEEE International Conference on Program Comprehension (ICPC).

[22]  Christian Kästner,et al.  Virtual Separation of Concerns: Toward Preprocessors 2.0 , 2010, it Inf. Technol..

[23]  Wolfgang Schröder-Preikschat,et al.  Toolchain-independent variant management with the Leviathan filesystem , 2010, FOSD '10.

[24]  Ahmed E. Hassan,et al.  Studying the Impact of Social Structures on Software Quality , 2010, 2010 IEEE 18th International Conference on Program Comprehension.

[25]  Matthias M. Müller,et al.  Are Reviews an Alternative to Pair Programming? , 2004, Empirical Software Engineering.

[26]  Yvonne Coady,et al.  CViMe: viewing conditionally compiled C/C++ sources through Java , 2006, OOPSLA '06.

[27]  Michel R. V. Chaudron,et al.  Effects of defects in UML models: an experimental investigation , 2006, ICSE.

[28]  Jeffrey C. Carver,et al.  An Empirical Study of the Effects of Gestalt Principles on Diagram Understandability , 2007, ESEM 2007.

[29]  W. Shadish,et al.  Experimental and Quasi-Experimental Designs for Generalized Causal Inference , 2001 .

[30]  Janet Siegmund,et al.  On the Role of Program Comprehension in Embedded Systems , 2011, Softwaretechnik-Trends.

[31]  Ira D. Baxter,et al.  Preprocessor conditional removal by simple partial evaluation , 2001, Proceedings Eighth Working Conference on Reverse Engineering.

[32]  Claes Wohlin,et al.  Evaluation of Usage-Based Reading—Conclusions after Three Experiments , 2004, Empirical Software Engineering.

[33]  Christian Wende,et al.  FeatureMapper: mapping features to models , 2008, ICSE Companion '08.

[34]  Mika V. Mäntylä,et al.  Defect Detection Efficiency: Test Case Based vs. Exploratory Testing , 2007, ESEM 2007.

[35]  Gregor Kiczales,et al.  Aspect-oriented programming , 1996, CSUR.

[36]  Claes Wohlin,et al.  Using Students as Subjects—A Comparative Study of Students and Professionals in Lead-Time Impact Assessment , 2000, Empirical Software Engineering.

[37]  Pearl Brereton,et al.  Systematic literature reviews in software engineering - A systematic literature review , 2009, Inf. Softw. Technol..

[38]  Márcio Ribeiro,et al.  Emergent feature modularization , 2010, SPLASH/OOPSLA Companion.

[39]  Anneliese Amschler Andrews,et al.  Program Comprehension During Software Maintenance and Evolution , 1995, Computer.

[40]  Pierre Dragicevic,et al.  Using text animated transitions to support navigation in document histories , 2010, CHI.

[41]  Amy J. Ko,et al.  Individual differences in program comprehension strategies in unfamiliar programming systems , 2003, 11th IEEE International Workshop on Program Comprehension, 2003..

[42]  Michel Dagenais,et al.  C/C++ conditional compilation analysis using symbolic execution , 2000, Proceedings 2000 International Conference on Software Maintenance.

[43]  Mario Piattini,et al.  Building measure-based prediction models for UML class diagram maintainability , 2007, Empirical Software Engineering.

[44]  Don Batory,et al.  Architecture Design by Transformation , 2010 .

[45]  H. E. Dunsmore,et al.  Software engineering metrics and models , 1986 .

[46]  Steve McConnell.  What Does 10x Mean? Measuring Variations in Programmer Productivity , 2011, Making Software.

[47]  Claes Wohlin,et al.  Experimentation in software engineering: an introduction , 2000 .

[48]  R. Thouless.  Experimental Psychology , 1939, Nature.

[49]  Yann-Gaël Guéhéneuc,et al.  Impact of the visitor pattern on program comprehension and maintenance , 2009, ESEM 2009.

[50]  Charles Rich.  Inspection methods in programming , 1980 .

[51]  Horst Zuse,et al.  Software complexity: Measures and methods , 1990 .

[52]  Henry Spencer,et al.  #ifdef Considered Harmful, or Portability Experience with C News , 1992, USENIX Summer.

[53]  Jochen Quante,et al.  Do Dynamic Object Process Graphs Support Program Understanding? - A Controlled Experiment. , 2008, 2008 16th IEEE International Conference on Program Comprehension.

[54]  J. Yellott.  Correction for fast guessing and the speed-accuracy tradeoff in choice reaction time , 1971 .

[55]  Sven Apel,et al.  Analyzing the discipline of preprocessor annotations in 30 million lines of C code , 2011, AOSD '11.

[56]  Sven Apel,et al.  A Case Study Implementing Features Using AspectJ , 2007, 11th International Software Product Line Conference (SPLC 2007).

[57]  M. Humphrey,et al.  Evaluation of the maintainability of object-oriented software , 1990, IEEE TENCON'90: 1990 IEEE Region 10 Conference on Computer and Communication Systems. Conference Proceedings.

[58]  Pankaj Jalote,et al.  An Experimental Evaluation of the Effectiveness and Efficiency of the Test Driven Development , 2007, ESEM 2007.

[59]  Gul A. Agha,et al.  Concurrent object-oriented programming , 1993, CACM.

[60]  Christian Bunse,et al.  Using patterns for the refinement and translationof UML models: A controlled experiment , 2006, Empirical Software Engineering.

[61]  Bo Zhang,et al.  Extraction and improvement of conditionally compiled product line code , 2012, 2012 20th IEEE International Conference on Program Comprehension (ICPC).

[62]  Ralph E. Johnson,et al.  Analyzing multiple configurations of a C program , 2005, 21st IEEE International Conference on Software Maintenance (ICSM'05).

[63]  Ray Welland,et al.  An empirical study of specification by example in a software engineering tool , 2010, ESEM '10.

[64]  Ioannis Stamelos,et al.  An experimental investigation of personality types impact on pair effectiveness in pair programming , 2009, Empirical Software Engineering.

[65]  Gregg Rothermel,et al.  Supporting Controlled Experimentation with Testing Techniques: An Infrastructure and its Potential Impact , 2005, Empirical Software Engineering.

[66]  E. E. Grant,et al.  Exploratory experimental studies comparing online and offline programming performance , 1968, CACM.

[67]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[68]  Nicolas E. Gold,et al.  Programming style changes in evolving source code , 2004, Proceedings. 12th IEEE International Workshop on Program Comprehension, 2004..

[69]  Martin P. Robillard,et al.  How effective developers investigate source code: an exploratory study , 2004, IEEE Transactions on Software Engineering.

[70]  Paul Clements,et al.  Software product lines - practices and patterns , 2001, SEI series in software engineering.

[71]  N. Nachar.  The Mann ‐ Whitney U: A Test for Assessing Whether Two Independent Samples Come from the Same Distribution , 2007 .

[72]  Andrew Walenstein,et al.  Observing and measuring cognitive support: steps toward systematic tool evaluation and engineering , 2003, 11th IEEE International Workshop on Program Comprehension, 2003..

[73]  Lionel C. Briand,et al.  Assessing, Comparing, and Combining Statechart- based testing and Structural testing: An Experiment , 2007, ESEM 2007.

[74]  Sven Apel,et al.  Superimposition: A Language-Independent Approach to Software Composition , 2008, SC@ETAPS.

[75]  A. Zeller,et al.  Predicting Defects for Eclipse , 2007, Third International Workshop on Predictor Models in Software Engineering (PROMISE'07: ICSE Workshops 2007).

[76]  Sven Apel,et al.  Granularity in software product lines , 2008, 2008 ACM/IEEE 30th International Conference on Software Engineering.

[77]  Tore Dybå,et al.  Evaluating Pair Programming with Respect to System Complexity and Programmer Expertise , 2007, IEEE Transactions on Software Engineering.

[78]  Ruven E. Brooks,et al.  Using a behavioral theory of program comprehension in software engineering , 1978, ICSE '78.

[79]  Andreas Hasselberg,et al.  PROPHET : Tool Infrastructure To Support Program Comprehension Experiments , 2011 .

[80]  S. Shapiro,et al.  An Analysis of Variance Test for Normality (Complete Samples) , 1965 .

[81]  Carlos José Pereira de Lucena,et al.  Modularizing design patterns with aspects: a quantitative study , 2005, AOSD '05.

[82]  Thomas Leich,et al.  Do background colors improve program comprehension in the #ifdef hell? , 2012, Empirical Software Engineering.

[83]  A. Adam.  Whatever happened to information systems ethics? Caught between the devil and the deep blue sea , 2004 .

[84]  Iris Vessey,et al.  Research Report - The Relevance of Application Domain Knowledge: The Case of Computer Program Comprehension , 1995, Inf. Syst. Res..

[85]  Norman Wilde,et al.  The role of concepts in program comprehension , 2002, Proceedings 10th International Workshop on Program Comprehension.

[86]  Barry W. Boehm,et al.  Software Engineering Economics , 1993, IEEE Transactions on Software Engineering.

[87]  Shing-Chi Cheung,et al.  Work experience versus refactoring to design patterns: a controlled experiment , 2006, SIGSOFT '06/FSE-14.

[88]  Jonathan I. Maletic,et al.  An Eye Tracking Study on camelCase and under_score Identifier Styles , 2010, 2010 IEEE 18th International Conference on Program Comprehension.

[89]  Dietmar Pfahl,et al.  Reporting Experiments in Software Engineering , 2008, Guide to Advanced Empirical Software Engineering.

[90]  Barbara Kitchenham,et al.  Procedures for Performing Systematic Reviews , 2004 .

[91]  Cláudio Sant'Anna,et al.  On the Impact of Aspectual Decompositions on Design Stability: An Empirical Study , 2007, ECOOP.

[92]  Gregor Snelting,et al.  On the inference of configuration structures from source code , 1994, Proceedings of 16th International Conference on Software Engineering.

[93]  John P. Boysen,et al.  Factors affecting computer program comprehension. , 1979 .

[94]  Marco Torchiano,et al.  The Role of Experience and Ability in Comprehension Tasks Supported by UML Stereotypes , 2007, 29th International Conference on Software Engineering (ICSE'07).

[95]  Stefan Biffl,et al.  Investigating the Defect Detection Effectiveness and Cost Benefit of Nominal Inspection Teams , 2003, IEEE Trans. Software Eng..

[96]  Thomas Leich,et al.  Aspectual Feature Modules , 2008, IEEE Transactions on Software Engineering.

[97]  D L Fisher,et al.  Visual Displays: The Highlighting Paradox , 1989, Human factors.

[98]  Jean-Marie Favre.  Understanding-in-the-large , 1997, Proceedings Fifth International Workshop on Program Comprehension. IWPC'97.

[99]  T. W. Anderson,et al.  The New Statistical Analysis of Data , 1986 .

[100]  Philip H. Demp,et al.  A mathematical perspective , 2001 .

[101]  Don S. Batory,et al.  Is the derivation of a model easier to understand than the model itself? , 2012, 2012 20th IEEE International Conference on Program Comprehension (ICPC).

[102]  David W. Binkley,et al.  Leveraged Quality Assessment using Information Retrieval Techniques , 2006, 14th IEEE International Conference on Program Comprehension (ICPC'06).

[103]  Jorma Sajaniemi,et al.  An investigation into professional programmers' mental representations of variables , 2005, 13th International Workshop on Program Comprehension (IWPC'05).

[104]  Thomas Leich,et al.  How to compare program comprehension in FOSD empirically: an experience report , 2009, FOSD '09.

[105]  Sven Apel,et al.  An Overview of Feature-Oriented Software Development , 2009, J. Object Technol..

[106]  Eric A. Brewer,et al.  ASTEC: a new approach to refactoring C , 2005, ESEC/FSE-13.

[107]  Eileen Kraemer,et al.  Empirical Evaluation of a UML Sequence Diagram with Adornments to Support Understanding of Thread Interactions , 2007, 15th IEEE International Conference on Program Comprehension (ICPC '07).

[108]  Carlos José Pereira de Lucena,et al.  Composing design patterns: a scalability study of aspect-oriented programming , 2006, AOSD.

[109]  Wolfgang Schröder-Preikschat,et al.  A quantitative analysis of aspects in the eCos kernel , 2006, EuroSys.

[110]  Kate Ehrlich,et al.  Empirical Studies of Programming Knowledge , 1984, IEEE Transactions on Software Engineering.

[111]  Daniela E. Damian,et al.  Selecting Empirical Methods for Software Engineering Research , 2008, Guide to Advanced Empirical Software Engineering.

[112]  Brad A. Myers,et al.  An Exploratory Study of How Developers Seek, Relate, and Collect Relevant Information during Software Maintenance Tasks , 2006, IEEE Transactions on Software Engineering.

[113]  Martin P. Robillard,et al.  FEAT a tool for locating, describing, and analyzing concerns in source code , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..

[114]  Lionel C. Briand,et al.  A Realistic Empirical Evaluation of the Costs and Benefits of UML in Software Maintenance , 2008, IEEE Transactions on Software Engineering.

[115]  Lerina Aversano,et al.  Handling preprocessor-conditioned declarations , 2002, Proceedings. Second IEEE International Workshop on Source Code Analysis and Manipulation.

[116]  M. Lewis-Beck.  Applied Regression: An Introduction , 1980 .

[117]  Brice Morin,et al.  Taming Dynamically Adaptive Systems using models and aspects , 2009, 2009 IEEE 31st International Conference on Software Engineering.

[118]  Thomas Leich,et al.  Toward measuring program comprehension with functional magnetic resonance imaging , 2012, SIGSOFT FSE.

[119]  Robert L. Glass,et al.  Science and substance: a challenge to software engineers , 1994, IEEE Software.

[120]  Jacob Cohen,et al.  Applied multiple regression/correlation analysis for the behavioral sciences , 1979 .

[121]  Stanley M. Sutton,et al.  Hyper/J™: multi-dimensional separation of concerns for Java™ , 2001, ICSE '02.

[122]  Simon,et al.  Mental models, consistency and programming aptitude , 2008, ACE '08.

[123]  Brian Henderson-Sellers,et al.  Object-Oriented Metrics , 1995, TOOLS.

[124]  Thomas Leich,et al.  FeatureIDE: A tool framework for feature-oriented software development , 2009, 2009 IEEE 31st International Conference on Software Engineering.

[125]  Cláudio Sant'Anna,et al.  On the Maintainability of Aspect-Oriented Software: A Concern-Oriented Measurement Framework , 2008, 2008 12th European Conference on Software Maintenance and Reengineering.

[126]  John von Neumann,et al.  First draft of a report on the EDVAC , 1993, IEEE Annals of the History of Computing.

[127]  F. edridge-green.  Tests for Colour-Blindness , 1895, Nature.

[128]  Jeffrey Overbey,et al.  Generating Rewritable Abstract Syntax Trees , 2009, SLE.

[129]  Eric Jensen,et al.  Teaching with the Brain in Mind , 1998 .

[130]  Alexandre Bergel,et al.  Asking and Answering Questions during a Programming Change Task in Pharo Language , 2014, PLATEAU.

[131]  G. A. Miller.  THE PSYCHOLOGICAL REVIEW THE MAGICAL NUMBER SEVEN, PLUS OR MINUS TWO: SOME LIMITS ON OUR CAPACITY FOR PROCESSING INFORMATION 1 , 1956 .

[132]  Jonathan I. Maletic,et al.  An empirical study on the comprehension of stereotyped UML class diagram layouts , 2009, 2009 IEEE 17th International Conference on Program Comprehension.

[133]  Martin P. Robillard,et al.  Representing concerns in source code , 2007, TSEM.

[134]  Ralph E. Johnson,et al.  Designing Reuseable Classes , 1988 .

[135]  Yngve Lindsjørn,et al.  SESE – an Experiment Support Environment for Evaluating Software Engineering Technologies , 2002 .

[136]  Bram Adams,et al.  Aspect mining in the presence of the C preprocessor , 2008, LATE@AOSD.

[137]  J. Favre.  The CPP paradox , 1995 .

[138]  Arie van Deursen,et al.  Connecting Traces: Understanding Client-Server Interactions in Ajax Applications , 2010, 2010 IEEE 18th International Conference on Program Comprehension.

[139]  Mira Mezini,et al.  An Overview of CaesarJ , 2006, LNCS Trans. Aspect Oriented Softw. Dev..

[140]  Thaís Vasconcelos Batista,et al.  On the Quantitative Analysis of Architecture Stability in Aspectual Decompositions , 2008, Seventh Working IEEE/IFIP Conference on Software Architecture (WICSA 2008).

[141]  F. W. Edridge-Green,et al.  Test for Colour Blindness , 1895 .

[142]  Don Batory,et al.  Scaling step-wise refinement , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..

[143]  John W. Daly,et al.  The effect of inheritance on the maintainability of object-oriented software: an empirical study , 1995, Proceedings of International Conference on Software Maintenance.

[144]  I. B. Myers.  Manual: A Guide to the Development and Use of the Myers-Briggs Type Indicator , 1985 .

[145]  Weishan Zhang,et al.  XVCL: XML-based variant configuration language , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..

[146]  Gerard K. Rambally.  The influence of color on program readability and comprehensibility , 1986, SIGCSE.

[147]  R. Fildes.  Journal of the Royal Statistical Society (B): Gary K. Grunwald, Adrian E. Raftery and Peter Guttorp, 1993, “Time series of continuous proportions”, 55, 103–116.☆ , 1993 .

[148]  Krzysztof Czarnecki,et al.  Verifying feature-based model templates against well-formedness OCL constraints , 2006, GPCE '06.

[149]  Adele E. Howe,et al.  Program understanding behaviour during enhancement of large-scale software , 1997, J. Softw. Maintenance Res. Pract..

[150]  David Notkin,et al.  Error reporting with graduated color , 1992, IEEE Software.

[151]  F. J. Roethlisberger,et al.  Management and the Worker , 1941 .

[152]  D. L. Parnas,et al.  On the criteria to be used in decomposing systems into modules , 1972, Software Pioneers.

[153]  Lodewijk Bergmans,et al.  Experimental evaluation of a tool for the verification and transformation of source code in event-driven systems , 2009, Empirical Software Engineering.

[154]  Douglas G. Mook,et al.  Motivation: The Organization of Action , 1987 .

[155]  Pim van den Broek,et al.  A model for variability design rationale in SPL , 2010, ECSA '10.

[156]  R. Cabeza,et al.  Imaging Cognition II: An Empirical Review of 275 PET and fMRI Studies , 2000, Journal of Cognitive Neuroscience.

[157]  Stephan Diehl,et al.  Software Visualization - Visualizing the Structure, Behaviour, and Evolution of Software , 2007 .

[158]  Raimund Dachselt,et al.  Using background colors to support program comprehension in software product lines , 2011, EASE.

[159]  James D. Herbsleb,et al.  Improving API documentation usability with knowledge pushing , 2009, 2009 IEEE 31st International Conference on Software Engineering.

[160]  Pearl Brereton,et al.  Empirical evidence about the UML: a systematic literature review , 2011, Softw. Pract. Exp..

[161]  Martin P. Robillard,et al.  Concern graphs: finding and describing concerns using structural program dependencies , 2002, Proceedings of the 24th International Conference on Software Engineering. ICSE 2002.

[162]  Marco Torchiano,et al.  The effectiveness of source code obfuscation: An experimental assessment , 2009, 2009 IEEE 17th International Conference on Program Comprehension.

[163]  David W. Binkley,et al.  Impact of Limited Memory Resources , 2008, 2008 16th IEEE International Conference on Program Comprehension.

[164]  Herman Rubin,et al.  Statistical Inference in Factor Analysis , 1956 .

[165]  Michael Philippsen,et al.  Two Controlled Experiments Assessing the Usefulness of Design Pattern Documentation in Program Maintenance , 2002, IEEE Trans. Software Eng..

[166]  Sven Apel,et al.  A model of refactoring physically and virtually separated features , 2009, GPCE '09.

[167]  Janet Siegmund,et al.  Supporting comprehension experiments with human subjects , 2012, 2012 20th IEEE International Conference on Program Comprehension (ICPC).

[168]  David W. Binkley,et al.  To camelcase or under_score , 2009, 2009 IEEE 17th International Conference on Program Comprehension.

[169]  Eduardo Figueiredo,et al.  ConcernMorph: metrics-based detection of crosscutting patterns , 2009, ESEC/FSE '09.

[170]  Fatemeh Zahedi,et al.  Knowledge-Based Repository Scheme for Storing and Retrieving Business Components: A Theoretical Design and an Empirical Analysis , 2003, IEEE Trans. Software Eng..

[171]  Tore Dybå,et al.  A systematic review of statistical power in software engineering experiments , 2006, Inf. Softw. Technol..

[172]  Colin Ware,et al.  Information Visualization: Perception for Design , 2000 .

[173]  Gunter Saake,et al.  Program Comprehension in Preprocessor-Based Software , 2012, SAFECOMP Workshops.

[174]  Gunter Saake,et al.  Supporting program comprehension in large preprocessor-based software product lines , 2012, IET Softw..

[175]  Audris Mockus,et al.  Using Version Control Data to Evaluate the Impact of Software Tools: A Case Study of the Version Editor , 2002, IEEE Trans. Software Eng..

[176]  J. Moran,et al.  Sensation and perception , 1980 .

[177]  V. Basili.  Software modeling and measurement: the Goal/Question/Metric paradigm , 1992 .

[178]  Harold Ossher,et al.  Hyper/J/sup TM/: multi-dimensional separation of concerns for Java/sup TM/ , 2000, Proceedings of the 2000 International Conference on Software Engineering. ICSE 2000 the New Millennium.

[179]  Jens Knodel,et al.  An experiment on the role of graphical elements in architecture visualization , 2008, Empirical Software Engineering.

[180]  Percy Williams Bridgman,et al.  The Logic of Modern Physics , 1927 .

[181]  Anneliese Amschler Andrews,et al.  From program comprehension to tool requirements for an industrial environment , 1993, [1993] IEEE Second Workshop on Program Comprehension.

[182]  David William Barron.  Assemblers and Loaders , 1978 .

[183]  Pearl Brereton,et al.  Performing systematic literature reviews in software engineering , 2006, ICSE.

[184]  Gregg Rothermel,et al.  The Effects of Time Constraints on Test Case Prioritization: A Series of Controlled Experiments , 2010, IEEE Transactions on Software Engineering.

[185]  Sven Apel,et al.  An analysis of the variability in forty preprocessor-based software product lines , 2010, 2010 ACM/IEEE 32nd International Conference on Software Engineering.

[186]  Sebastian Kleinschmager,et al.  How to rate programming skills in programming experiments?: a preliminary, exploratory, study based on university marks, pretests, and self-estimation , 2011, PLATEAU '11.

[187]  Genny Tortora,et al.  An experimental comparison of ER and UML class diagrams for data modelling , 2010, Empirical Software Engineering.

[188]  Sebastian Kleinschmager,et al.  Does aspect-oriented programming increase the development speed for crosscutting code? An empirical study , 2009, ESEM 2009.

[189]  Paul W. Oman,et al.  Experiences developing and maintaining software in a multi-platform environment , 1997, 1997 Proceedings International Conference on Software Maintenance.

[190]  Maurice H. Halstead,et al.  Elements of software science , 1977 .

[191]  Yvonne Coady,et al.  C-CLR: a tool for navigating highly configurable system software , 2007, ACP4IS.

[192]  Jim Buckley,et al.  Inference-based and expectation-based processing in program comprehension , 2001, Proceedings 9th International Workshop on Program Comprehension. IWPC 2001.

[193]  Claes Wohlin,et al.  Using students as subjects - an empirical evaluation , 2008, ESEM '08.

[194]  Thomas A. Standish.  An Essay on Software Reuse , 1984, IEEE Transactions on Software Engineering.

[195]  R. Likert.  “Technique for the Measurement of Attitudes, A” , 2022, The SAGE Encyclopedia of Research Design.

[196]  Cláudio Sant'Anna,et al.  Evolving software product lines with aspects , 2008, 2008 ACM/IEEE 30th International Conference on Software Engineering.

[197]  Wolfgang De Meuter,et al.  Can we refactor conditional compilation into aspects? , 2009, AOSD '09.

[198]  Mark Lorenz,et al.  Object-oriented software metrics - a practical guide , 1994 .

[199]  Stefan Biffl,et al.  Evaluating the accuracy of defect estimation models based on inspection data from two inspection cycles , 2001, Proceedings of the 23rd International Conference on Software Engineering. ICSE 2001.

[200]  Heinz-Martin Süß,et al.  Berliner Intelligenzstruktur-Test : BIS-Test , 1997 .

[201]  Massimiliano Di Penta,et al.  An experimental investigation of formality in UML-based development , 2005, IEEE Transactions on Software Engineering.

[202]  Charles W. Krueger,et al.  Easing the Transition to Software Mass Customization , 2001, PFE.

[203]  Sven Apel,et al.  Visual Support for Understanding Product Lines , 2010, 2010 IEEE 18th International Conference on Program Comprehension.

[204]  Danilo Beuche,et al.  Variability management with feature models , 2004, Sci. Comput. Program..

[205]  Ruven E. Brooks,et al.  Towards a Theory of the Comprehension of Computer Programs , 1983, Int. J. Man Mach. Stud..

[206]  Marc Roper,et al.  A Comparative Evaluation of Program Comprehension Measures , 2000 .

[207]  Marc Roper,et al.  Further investigations into the development and evaluation of reading techniques for object-oriented code inspection , 2002, ICSE '02.

[208]  Sven Apel,et al.  Measuring programming experience , 2012, 2012 20th IEEE International Conference on Program Comprehension (ICPC).

[209]  Rebecca Tiarks.  What Programmers Really Do - An Observational Study , 2011, Softwaretechnik-Trends.

[210]  David Salomon,et al.  Assemblers and loaders , 1992, Ellis Horwood series in computers and their applications.

[211]  Walter Schneider,et al.  The neural correlates of strategic reading comprehension: Cognitive control and discourse comprehension , 2011, NeuroImage.

[212]  Brian Ellis,et al.  The Factory Pattern in API Design: A Usability Evaluation , 2007, 29th International Conference on Software Engineering (ICSE'07).

[213]  Rudolf Ferenc,et al.  Columbus schema for C/C++ preprocessing , 2004, Eighth European Conference on Software Maintenance and Reengineering, 2004. CSMR 2004. Proceedings..

[214]  Andrew M. Kuhn,et al.  Code Complete , 2005, Technometrics.

[215]  Wolfgang Schröder-Preikschat,et al.  Feature consistency in compile-time-configurable system software: facing the linux 10,000 feature problem , 2011, EuroSys '11.

[216]  Keith C. C. Chan,et al.  The Effect of Pairs in Program Design Tasks , 2008, IEEE Transactions on Software Engineering.

[217]  Wuu Yang,et al.  How to merge program texts , 1994, J. Syst. Softw..

[218]  Fabrizio Riguzzi.  A Survey of Software Metrics , 2002 .

[219]  Marco Tulio Valente,et al.  Extracting Software Product Lines: A Case Study Using Conditional Compilation , 2011, 2011 15th European Conference on Software Maintenance and Reengineering.

[220]  Liwu Li.  Programming in Java , 1998 .

[221]  R. Rosenthal,et al.  Teachers' Expectancies: Determinants of Pupils' IQ Gains , 1966, Psychological reports.

[222]  A. Baddeley.  Is working memory still working , 2001 .

[223]  Carlos José Pereira de Lucena,et al.  Assessing Aspect-Oriented Artifacts: Towards a Tool-Supported Quantitative Method , 2005 .

[224]  Stephen G. Eick,et al.  Seesoft-A Tool For Visualizing Line Oriented Software Statistics , 1992, IEEE Trans. Software Eng..

[225]  Marco Torchiano,et al.  On the effectiveness of the test-first approach to programming , 2005, IEEE Transactions on Software Engineering.

[226]  Sven Apel,et al.  An Algebra for Features and Feature Composition , 2008, AMAST.

[227]  Roberto Erick Lopez-Herrejon,et al.  Evaluating Support for Features in Advanced Modularization Technologies , 2005, ECOOP.

[228]  Mark Chu-Carroll,et al.  Visual separation of concerns through multidimensional program storage , 2003, AOSD '03.

[229]  Daqing Hou,et al.  Aiding Software Maintenance with Copy-and-Paste Clone-Awareness , 2010, 2010 IEEE 18th International Conference on Program Comprehension.

[230]  Thomas D. LaToza,et al.  Developers ask reachability questions , 2010, 2010 ACM/IEEE 32nd International Conference on Software Engineering.

[231]  José Javier Dolado,et al.  An Initial Experimental Assessment of the Dynamic Modelling in UML , 2004, Empirical Software Engineering.

[232]  Yuanfang Cai,et al.  A Preliminary Study of Quantified , Typed Events , 2010 .

[233]  C. Kastner,et al.  On the Necessity of Empirical Studies in the Assessment of Modularization Mechanisms for Crosscutting Concerns , 2007, First International Workshop on Assessment of Contemporary Modularization Techniques (ACoM '07).

[234]  Gail C. Murphy,et al.  Asking and Answering Questions during a Programming Change Task , 2008, IEEE Transactions on Software Engineering.

[235]  Klaus Pohl,et al.  Software Product Line Engineering - Foundations, Principles, and Techniques , 2005 .

[236]  Sven Apel,et al.  Comparing program comprehension of physically and virtually separated concerns , 2012, FOSD '12.

[237]  J. Bortz.  Statistik für Human- und Sozialwissenschaftler , 2010 .

[238]  J. W. Backus,et al.  The Fortran Automatic Coding System for the IBM 704 EDPM , 1956 .

[239]  David Wechsler.  The Measurement of Adult Intelligence. , 1940 .

[240]  Natalia Juristo Juzgado,et al.  Basics of Software Engineering Experimentation , 2010, Springer US.

[241]  Miroslaw Staron,et al.  An empirical study on using stereotypes to improve understanding of UML models , 2004, Proceedings. 12th IEEE International Workshop on Program Comprehension, 2004..

[242]  Alvy Ray Smith,et al.  Color gamut transform pairs , 1978, SIGGRAPH.

[243]  Dennis G. Kafura.  A survey of software metrics , 1985, ACM '85.

[244]  J. W. Backus,et al.  The FORTRAN automatic coding system , 1899, IRE-AIEE-ACM '57 (Western).

[245]  Yann-Gaël Guéhéneuc,et al.  An empirical study on the efficiency of different design pattern representations in UML class diagrams , 2010, Empirical Software Engineering.

[246]  N. Pennington.  Stimulus structures and mental representations in expert comprehension of computer programs , 1987, Cognitive Psychology.

[247]  David Coppit,et al.  Spotlight: A Prototype Tool for Software Plans , 2007, 29th International Conference on Software Engineering (ICSE'07).

[248]  Michael D. Ernst,et al.  An Empirical Analysis of C Preprocessor Use , 2002, IEEE Trans. Software Eng..

[249]  Anneliese Amschler Andrews,et al.  Building a research infrastructure for program comprehension observations , 1997, Proceedings Fifth International Workshop on Program Comprehension. IWPC'97.

[250]  Ben Shneiderman,et al.  Syntactic/semantic interactions in programmer behavior: A model and experimental results , 1979, International Journal of Computer & Information Sciences.

[251]  B. Rosen,et al.  Functional mapping of the human visual cortex by magnetic resonance imaging. , 1991, Science.

[252]  Björn Regnell,et al.  An experiment on linguistic tool support for consolidation of requirements from multiple sources in market-driven product development , 2006, Empirical Software Engineering.

[253]  Christian Prehofer,et al.  Feature-Oriented Programming: A Fresh Look at Objects , 1997, ECOOP.

[254]  Scott P. Robertson,et al.  Expert problem solving strategies for program comprehension , 1991, CHI.

[255]  Anas N. Al-Rabadi,et al.  A comparison of modified reconstructability analysis and Ashenhurst‐Curtis decomposition of Boolean functions , 2004 .

[256]  Mohamed El-Attar,et al.  A subject-based empirical evaluation of SSUCD’s performance in reducing inconsistencies in use case models , 2009, Empirical Software Engineering.

[257]  Eric Clayberg,et al.  Eclipse: Building Commercial-Quality Plug-ins , 2004 .

[258]  Claes Wohlin,et al.  An Investigation of a Method for Identifying a Software Architecture Candidate with Respect to Quality Attributes , 2005, Empirical Software Engineering.

[259]  Dag I. K. Sjøberg,et al.  Effects of Personality on Pair Programming , 2010, IEEE Transactions on Software Engineering.

[260]  Sven Apel,et al.  Exploring Software Measures to Assess Program Comprehension , 2011, 2011 International Symposium on Empirical Software Engineering and Measurement.

[261]  Tore Dybå,et al.  A systematic review of quasi-experiments in software engineering , 2009, Inf. Softw. Technol..

[262]  N. Macrae.  John Von Neumann , 1992 .

[263]  Martin P. Robillard,et al.  A Comparative Study of Three Program Exploration Tools , 2007, 15th IEEE International Conference on Program Comprehension (ICPC '07).

[264]  簡聰富,et al.  物件導向軟體之架構(Object-Oriented Software Construction)探討 , 1989 .

[265]  André van der Hoek,et al.  Empirical evidence of the benefits of workspace awareness in software configuration management , 2008, SIGSOFT '08/FSE-16.