Theory-oriented software engineering

Abstract There has been a growing interest in the role of theory within Software Engineering (SE) research. For several decades, researchers within the SE research community have argued that, to become a ‘real’ engineering science, SE needs to develop stronger theoretical foundations. However, so far, the role of theory is neither fully appreciated nor well understood in SE research. Without a good common understanding of what theory is, what it constitutes in SE research, and the various roles it can play in SE research, it is difficult to appreciate how theory building can help to strengthen SE research. In this paper we discuss the importance of theory and conceptualization, and review the key components that comprise a theory. We then present the Research Path Schema (RPS), which is an adaptation of an analytical framework from the social sciences. The RPS defines a research study as consisting of three components: some phenomenon, system or substance that a researcher is interested in; some technique or method to study that substance; and some form of conceptualization or theory that provides an explanation for, or abstraction of the observations made in a study. Different research studies have a different archetypical ‘architecture,’ depending on the selection of these three components. Consequently, the role of the conceptualization or theory will be different for each archetypical study design, or selected research path. We conclude this paper by outlining a number of implications for future SE research, and argue for a Theory-Oriented Software Engineering research perspective, which can complement the recent focus on Evidence Based Software Engineering.

[1]  Helen M. Edwards,et al.  The repertory grid technique: Its place in empirical software engineering research , 2009, Inf. Softw. Technol..

[2]  Muhammad Ali Babar,et al.  Design and Evaluation of a Process for Identifying Architecture Patterns in Open Source Software , 2011, ECSA.

[3]  Nelly Condori-Fernández,et al.  The Structure of Design Theories, and an Analysis of their Use in Software Engineering Experiments , 2011, 2011 International Symposium on Empirical Software Engineering and Measurement.

[4]  M. Sandelowski.  Focus on Research Methods Whatever Happened to Qualitative Description? , 2022 .

[5]  Michelle Cartwright,et al.  An Empirical Investigation of an Object-Oriented Software System , 2000, IEEE Trans. Software Eng..

[6]  Brian Fitzgerald,et al.  Towards dissolution of the is research debate: from polarization to polarity , 1998, J. Inf. Technol..

[7]  Michael W. Godfrey,et al.  On the evolution of Lehman's Laws , 2014, J. Softw. Evol. Process..

[8]  E. Burton Swanson,et al.  The dimensions of maintenance , 1976, ICSE '76.

[9]  Helen Sharp,et al.  A systematic review of theory use in studies investigating the motivations of software engineers , 2009, TSEM.

[10]  Meir M. Lehman,et al.  Approach to a theory of software evolution , 2001, Eighth International Workshop on Principles of Software Evolution (IWPSE'05).

[11]  J. McGrath,et al.  External Validity and the Research Process: A Comment on the Calder/Lynch Dialogue , 1983 .

[12]  Peter J. Denning,et al.  The science in computer science , 2013, CACM.

[13]  Shari Lawrence Pfleeger,et al.  Albert Einstein and Empirical Software Engineering , 1999, Computer.

[14]  Shirley Gregor,et al.  The Anatomy of a Design Theory , 2007, J. Assoc. Inf. Syst..

[15]  Ivar Jacobson,et al.  Where's the Theory for Software Engineering? , 2012, IEEE Software.

[16]  Tore Dybå,et al.  Evidence-based software engineering , 2016, Perspectives on Data Science for Software Engineering.

[17]  Veda C. Storey,et al.  An ontological analysis of the relationship construct in conceptual modeling , 1999, TODS.

[18]  Dag I. K. Sjøberg.  Documenting Theories Working Group Results , 2006, Empirical Software Engineering Issues.

[19]  Kevin Crowston,et al.  Effective work practices for software engineering: free/libre open source software development , 2004, WISER '04.

[20]  Hubert M. Blalock,et al.  Theory Construction: From Verbal to Mathematical Formulations. , 1970 .

[21]  Lorraine Morgan,et al.  Exploring value networks: theorising the creation and capture of value with open source software , 2013, Eur. J. Inf. Syst..

[22]  Neal E. Chalofsky,et al.  Part three: Should the knowledge base come from theory & from practice?. Professionalization comes from theory and research: The why instead of the how to , 1996 .

[23]  J. Mackie,et al.  The Conduct of Inquiry: Methodology for Behavioural Science , 1965 .

[24]  Paul Clements,et al.  Software architecture in practice , 1999, SEI series in software engineering.

[25]  P. Reynolds.  A Primer in Theory Construction , 1971 .

[26]  Helen Sharp,et al.  Motivation in Software Engineering: A systematic literature review , 2008, Inf. Softw. Technol..

[27]  Paul Ralph,et al.  Comparing Two Software Design Process Theories , 2010, DESRIST.

[28]  Samuel B. Bacharach,et al.  Organizational Theories: Some Criteria for Evaluation , 1989 .

[29]  Meir M. Lehman,et al.  Program evolution: processes of software change , 1985 .

[30]  John R. Venable,et al.  The role of theory and theorising in Design Science research , 2006 .

[31]  P. Strevens.  Iii , 1985 .

[32]  E PerryDewayne,et al.  Report on the Second SEMAT Workshop on General Theory of Software Engineering (GTSE 2013) , 2013 .

[33]  Kurt Lewin,et al.  The Research Center for Group Dynamics at Massachusetts Institute of Technology. , 1945 .

[34]  K. Weick.  What Theory Is Not, Theorizing Is , 1995 .

[35]  R. Nisbett.  The geography of thought : how Asians and Westerners think differently--and why , 2003 .

[36]  T. Kuhn,et al.  The Structure of Scientific Revolutions. , 1964 .

[37]  Kevin G. Corley,et al.  Building theory about theory building: What constitutes a theoretical contribution? Academy of Management Review, , . , 2011 .

[38]  S. Leshem,et al.  Overlooking the conceptual framework , 2007 .

[39]  Jan Bosch,et al.  Software Architecture as a Set of Architectural Design Decisions , 2005, 5th Working IEEE/IFIP Conference on Software Architecture (WICSA'05).

[40]  Barry M. Staw,et al.  What Theory is Not , 1995 .

[41]  Peter J. Denning,et al.  Computing is a natural science , 2007, CACM.

[42]  Jay F. Nunamaker,et al.  Systems Development in Information Systems Research , 1990, J. Manag. Inf. Syst..

[43]  L. J. Bourgeois,et al.  Toward A Method Of Middle-Range Theorizing , 1979 .

[44]  Ronald Lippitt,et al.  The research center for group dynamics , 1947 .

[45]  Richard H. Carver,et al.  An Evaluation of the MOOD Set of Object-Oriented Software Metrics , 1998, IEEE Trans. Software Eng..

[46]  Richard N. Taylor,et al.  A Classification and Comparison Framework for Software Architecture Description Languages , 2000, IEEE Trans. Software Eng..

[47]  J. Herbsleb,et al.  Two case studies of open source software development: Apache and Mozilla , 2002, TSEM.

[48]  Rafael Capilla,et al.  A web-based tool for managing architectural design decisions , 2006, SOEN.

[49]  K. B. Zerangue.  On developing a general theory of software engineering , 1993, Proceedings of 1993 IEEE 17th International Computer Software and Applications Conference COMPSAC '93.

[50]  Robert L. Glass,et al.  The relationship between theory and practice in software engineering , 1996, CACM.

[51]  Audris Mockus,et al.  A case study of open source software development: the Apache server , 2000, Proceedings of the 2000 International Conference on Software Engineering. ICSE 2000 the New Millennium.

[52]  Chris F. Kemerer,et al.  A Metrics Suite for Object Oriented Design , 2015, IEEE Trans. Software Eng..

[53]  Yuanfang Cai,et al.  Analyzing the Evolution of Large-Scale Software Systems Using Design Structure Matrices and Design Rule Theory: Two Exploratory Cases , 2008, Seventh Working IEEE/IFIP Conference on Software Architecture (WICSA 2008).

[54]  Shari Lawrence Pfleeger,et al.  Principles of survey research: part 1: turning lemons into lemonade , 2001, SOEN.

[55]  D. Shepherd,et al.  Theory Building , 2017 .

[56]  Colin Robson,et al.  Real World Research: A Resource for Social Scientists and Practitioner-Researchers , 1993 .

[57]  Jesús M. González-Barahona,et al.  Studying the laws of software evolution in a long-lived FLOSS project , 2013, J. Softw. Evol. Process..

[58]  Mario Bunge,et al.  Scientific Research I , 1967 .

[59]  James R Cook.  Engaged Scholarship: A Guide for Organizational and Social Research , 2014 .

[60]  Elizabeth C. Hirschman,et al.  Multiple Orientations for the Conduct of Marketing Research: An Analysis of the Academic/Practitioner Distinction , 1986 .

[61]  Jesús M. González-Barahona,et al.  The evolution of the laws of software evolution , 2013, ACM Comput. Surv..

[62]  Ita Richardson,et al.  Who Are We Doing Global Software Engineering Research For? , 2013, 2013 IEEE 8th International Conference on Global Software Engineering.

[63]  Wynne W. Chin,et al.  Understanding frameworks and reviews: a commentary to assist us in moving our field forward by analyzing our past , 2007, DATB.

[64]  S. Lynham.  The General Method of Theory-Building Research in Applied Disciplines , 2002 .

[66]  Muhammad Ali Babar,et al.  Key factors for adopting inner source , 2014, ACM Trans. Softw. Eng. Methodol..

[67]  Salvatore T. March,et al.  Design and natural science research on information technology , 1995, Decis. Support Syst..

[68]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[69]  Paul Ralph,et al.  Report on the Second SEMAT Workshop on General Theory of Software Engineering (GTSE 2013) , 2013, SOEN.

[70]  David D. McDonald,et al.  Programs , 1984, CL.

[71]  John G. Lynch,et al.  Validity and the research process , 1986 .

[72]  R. Suddaby.  From the Editors: What Grounded Theory is Not , 2006 .

[73]  Muhammad Ali Babar,et al.  The use of empirical methods in Open Source Software research: Facts, trends and future directions , 2009, 2009 ICSE Workshop on Emerging Trends in Free/Libre/Open Source Software Research and Development.

[74]  Albert Endres,et al.  A handbook of software and systems engineering - empirical observations, laws and theories , 2003, The Fraunhofer IESE series on software engineering.

[75]  Tore Dybå,et al.  Building Theories in Software Engineering , 2008, Guide to Advanced Empirical Software Engineering.

[76]  Meir M. Lehman.  Programs, Cities, Students— Limits to Growth? , 1978 .

[77]  Richard C. Holt,et al.  Linux as a case study: its extracted software architecture , 1999, Proceedings of the 1999 International Conference on Software Engineering (IEEE Cat. No.99CB37002).

[78]  Austen Rainer,et al.  Case Study Research in Software Engineering - Guidelines and Examples , 2012 .

[79]  Tore Dybå,et al.  A Systematic Review of Theory Use in Software Engineering Experiments , 2007, IEEE Transactions on Software Engineering.

[80]  Paul Ralph,et al.  A teleological process theory of software development , 2008 .

[81]  Is the Scientific Paper a Fraud? or: Who's Got the Paper? , 1965, Canadian Medical Association journal.

[82]  Janice Singer,et al.  Guide to Advanced Empirical Software Engineering , 2007 .

[83]  R. Torretti.  Mario Bunge: Scientific research. I. The search for system. II. The search for truth. Berlin Heidelberg. New York. Springer-Verlag. 1967 , 1967 .

[84]  K. Weick.  Theory Construction as Disciplined Imagination , 1989 .

[85]  Meir M. Lehman,et al.  A Model of Large Program Development , 1976, IBM Syst. J..

[86]  G A Cioffi,et al.  Theoretically speaking , 2015, Nature Photonics.

[87]  George Caspar Homans.  Contemporary Theory in Sociology , 2017 .

[88]  D. Whetten.  What Constitutes a Theoretical Contribution , 1989 .

[89]  Derek J. Pike,et al.  Empirical Model‐building and Response Surfaces. , 1988 .

[90]  Peter J. Denning,et al.  Is computer science science? , 2005, CACM.

[91]  Shirley Gregor,et al.  The Nature of Theory in Information Systems , 2006, MIS Q..

[92]  Marvin V. Zelkowitz,et al.  Empirical studies to build a science of computer science , 2007, CACM.

[93]  Helen Sharp,et al.  An Ethnographic Study of XP Practice , 2004, Empirical Software Engineering.

[94]  Paul A. Swatman,et al.  Structured-case: a methodological framework for building theory in information systems research , 2000, ECIS.

[95]  Forrest Shull,et al.  Building Theories from Multiple Evidence Sources , 2008, Guide to Advanced Empirical Software Engineering.

[96]  Brian Randell,et al.  Report on a conference sponsored by the NATO Science Committee , 1968 .

[97]  Ma. de la Natividad Jiménez Salas,et al.  The Conduct of Inquiry , 1967 .

[98]  H. Reichenbach.  Experience And Prediction , 1938 .

[99]  Ron Weber,et al.  Editor's comment: theoretically speaking , 2003 .

[100]  Mary Shaw,et al.  Prospects for an engineering discipline of software , 1990, IEEE Software.

[101]  M. Yadav.  The Decline of Conceptual Articles and Implications for Knowledge Development , 2010 .

[102]  M. J. Lawrence,et al.  An examination of evolution dynamics , 1982, ICSE '82.

[103]  Manfred Broy.  Can Practitioners Neglect Theory and Theoreticians Neglect Practice? , 2011, Computer.

[104]  Paris Avgeriou,et al.  Capturing tacit architectural knowledge using the repertory grid technique (NIER track): (nier track) , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[105]  J. Creswell,et al.  Determining Validity in Qualitative Inquiry , 2000 .

[106]  R. Merton.  Social Theory and Social Structure , 1958 .

[107]  A. Jefferson Offutt.  Putting the Engineering into Software Engineering Education , 2013, IEEE Softw..

[108]  Dorina C. Petriu,et al.  Architecture-Based Performance Analysis Applied to a Telecommunication System , 2000, IEEE Trans. Software Eng..

[109]  Rory V. O'Connor,et al.  Using grounded theory to understand software process improvement: A study of Irish software product companies , 2007, Inf. Softw. Technol..

[110]  M. Bunge,et al.  Scientific Research I: The Search for System , 1967 .

[111]  Karim R. Lakhani,et al.  Perspectives on Free and Open Source Software , 2005 .

[112]  Daniela Cruzes,et al.  Research synthesis in software engineering: A tertiary study , 2011, Inf. Softw. Technol..

[113]  Frank Buschmann.  A Week in the Life of an Architect , 2012, IEEE Software.

[114]  Brian Fitzgerald,et al.  Uncovering theories in software engineering , 2013, 2013 2nd SEMAT Workshop on a General Theory of Software Engineering (GTSE).

[115]  Lionel Briand,et al.  Embracing the Engineering Side of Software Engineering , 2012, IEEE Softw..