Investigating the impact of requirements elicitation and evolution on course performance in a pre-capstone design course

ABSTRACT This paper investigates the correlation between requirements evolution and course grades in a pre-capstone design course. A case study is performed whereby requirements are collected on a common design project from eight separate design teams across the design process that culminates in a physical deliverable. This research was performed using the case study approach as it is commonly utilised to investigate occurrences in real-life context and focus on answering exploratory questions. The requirements are analysed through a three step process where requirements are interpreted with respect to the solution, classified as constraints or criteria, and classified as functional or non-functional. The evolution of requirements are correlated to the team’s course grade through statistical analysis. Evolution considers both the difference in requirements from start to finish and the week-to-week changes in requirements. The findings reveal the number of requirements students generate at the start does not impact course grades success. However, a higher number of defined requirements throughout the project, early functional requirement definition, and the continuous evolution of functional requirements were found to have a positive effect on course grades. Based on the results of the study, a set of guidelines and recommendations are developed for use within design courses.

[1]  Simon Szykman,et al.  A functional basis for engineering design: Reconciling and evolving previous efforts , 2002 .

[2]  Kevin Otto,et al.  Product Design: Techniques in Reverse Engineering and New Product Development , 2000 .

[3]  Gruia-Catalin Roman,et al.  A taxonomy of current issues in requirements engineering , 1985, Computer.

[4]  Ian Alexander,et al.  Learning How To Discover Requirements , 2008, 2008 Requirements Engineering Education and Training.

[5]  Yong Zeng,et al.  Formalisation of product requirements: from natural language descriptions to formal specifications , 2007, Int. J. Manuf. Res..

[6]  Claudia Eckert,et al.  Change Propagation Analysis in Complex Technical Systems , 2009 .

[7]  Lawrence Chung,et al.  Dealing with Non-Functional Requirements: Three Experimental Studies of a Process-Oriented Approach , 1995, 1995 17th International Conference on Software Engineering.

[8]  Chiradeep Sen,et al.  Utilizing Node Interference Method and Complex Network Centrality Metrics to Explore Requirement Change Propagation , 2017 .

[9]  Georges M. Fadel,et al.  Lightweight Engineering of Military Vehicles Through Requirements Analysis and Function Integration , 2009 .

[10]  Shraddha Joshi,et al.  Mapping Problem and Requirements to Final Solution: A Document Analysis of Capstone Design Projects , 2011 .

[11]  P. Clarkson,et al.  Predicting change propagation in complex design , 2004 .

[12]  Shraddha Joshi,et al.  A Coding Scheme for Analyzing Capstone Design Reports: Problem and Solution Descriptions , 2011 .

[13]  J. Dixon,et al.  Engineering Design , 2019, Springer Handbook of Mechanical Engineering.

[14]  Tzvi Raz,et al.  An empirical analysis of the relationship between project planning and project success , 2003 .

[15]  Susan P. Williams,et al.  Requirements volatility & its impact on change effort: Evidence based research n software development projects , 2006 .

[16]  P. John Clarkson,et al.  A Multidomain Engineering Change Propagation Model to Support Uncertainty Reduction and Risk Management in Design , 2012 .

[17]  Stuart Pugh.  Design activity models: worldwide emergence and convergence , 1986 .

[18]  John S. Gero,et al.  The Situated Function - Behaviour - Structure Framework , 2002, AID.

[19]  Brigitte Moench,et al.  Engineering Design A Systematic Approach , 2016 .

[20]  Christos Spitas,et al.  Analysis of systematic engineering design paradigms in industrial practice: A survey , 2011 .

[21]  Sergey Diev.  Requirements development as a modeling activity , 2007, SOEN.

[22]  Mark R. Cutkosky,et al.  Toward a computational framework for concurrent engineering , 1990, [Proceedings] IECON '90: 16th Annual Conference of IEEE Industrial Electronics Society.

[23]  Meir M. Lehman,et al.  Software's future: managing evolution , 1998, IEEE Software.

[24]  Joshua D. Summers,et al.  A Hierarchical Modeling Scheme With Non Functional Requirements , 2010 .

[25]  Patrick Little,et al.  Engineering Design: A Project Based Introduction , 1999 .

[26]  Knut Holt.  Does the engineer forget the user , 1989 .

[27]  Joshua D. Summers,et al.  Evaluation of the functional basis using an information theoretic approach , 2010, Artificial Intelligence for Engineering Design, Analysis and Manufacturing.

[28]  Yong Zeng,et al.  Classification of Product Requirements Based on Product Environment , 2006, Concurr. Eng. Res. Appl..

[29]  Thanwadee Sunetnanta,et al.  Distributing Responsibilities to Engineer Better Requirements: Leveraging Knowledge and Perspectives for Students to Learn a Key Skill , 2009, 2009 Fourth International Workshop on Requirements Engineering Education and Training.

[30]  Julio Cesar Sampaio do Prado Leite,et al.  On Non-Functional Requirements in Software Engineering , 2009, Conceptual Modeling: Foundations and Applications.

[31]  Joshua D. Summers,et al.  Reasons for change propagation: a case study in an automotive OEM , 2012 .

[32]  Shraddha Joshi,et al.  Requirements and Data Content Evaluation of Industry In-House Data Management System , 2010 .

[33]  Stein Andersen,et al.  Need assessment: a way of improving the value of new products , 1983 .

[34]  Joshua D. Summers,et al.  Comparative analysis of requirements change prediction models: manual, linguistic, and neural network , 2014, Research in Engineering Design.

[35]  Joshua D. Summers,et al.  Case Study Method for Design Research: A Justification , 2008 .

[36]  David Callele,et al.  Teaching requirements engineering to an unsuspecting audience , 2006, SIGCSE.

[37]  Stuart Pugh,et al.  Total Design: Integrated Methods for Successful Product Engineering , 1991 .

[38]  Tibor Bercsey,et al.  The Autogenetic Design Theory: An evolutionary view of the design process , 2003 .

[39]  David D. Walden,et al.  Systems engineering handbook : a guide for system life cycle processes and activities , 2015 .

[40]  Andreas S. Andreou,et al.  A Three-Dimensional Requirements Elicitation and Management Decision-Making Scheme for the Development of New Software Components , 2003, ICEIS.

[41]  Lisbeth Svengren.  Case study methods in design management research , 1993 .

[42]  Erik Kaestner,et al.  The Mechanical Design Process , 2016 .

[43]  P. John Clarkson,et al.  Change and customisation in complex engineering domains , 2004 .

[44]  Susan P. Williams,et al.  Requirements Volatility and Its Impact on Change Effort: Evidence-based Research in Software Development Projects Return to Published Papers , 2006 .

[45]  Anders Haug,et al.  Emergence patterns for client design requirements , 2015 .

[46]  Daniel Gross,et al.  From Non-Functional Requirements to Design through Patterns , 2001, Requirements Engineering.

[47]  Hyung Jun Ahn,et al.  Capturing and reusing knowledge in engineering change management: A case of automobile development , 2006, Inf. Syst. Frontiers.

[48]  K. Jacobsen,et al.  Formalized specification of functional requirements , 1991 .

[49]  Kurt M. Marshek,et al.  A Schema for Negotiation between Intelligent Design Agents in Concurrent Engineering , 1991, IntCAD.

[50]  Sergey Diev.  Structuring complex requirements , 2007, SOEN.

[51]  Petra Badke-Schaub,et al.  Analysis of design projects , 1999 .

[52]  Shamkant B. Navathe,et al.  C-FAR, change favorable representation , 2000, Comput. Aided Des..

[53]  Christine Nadel,et al.  Case Study Research Design And Methods , 2016 .

[54]  Beshoy Morkos,et al.  Predicting requirement change propagation through investigation of physical and functional domains , 2018 .

[55]  John S. Gero,et al.  An approach to the analysis of design protocols , 1998 .

[56]  Neil A. Ernst,et al.  Requirements Evolution and What (Research) to Do about It , 2009 .

[57]  Yuh-Min Chen,et al.  Distributed engineering change management for allied concurrent engineering , 2002, Int. J. Comput. Integr. Manuf..

[58]  Joshua D. Summers,et al.  Predicting requirement change propagation, using higher order design structure matrices: an industry case study , 2012 .

[59]  Thomas G. Dietterich,et al.  A model of the mechanical design process based on empirical data , 1988, Artificial Intelligence for Engineering Design, Analysis and Manufacturing.

[60]  W. Lam,et al.  Requirements change: a dissection of management issues , 1999, Proceedings 25th EUROMICRO Conference. Informatics: Theory and Practice for the New Millennium.

[61]  Evastina Björk,et al.  Research on dynamic systems : some considerations , 2004 .

[62]  Joshua D. Summers,et al.  Enriching requirement-activities in design through french-US instruction comparison , 2013 .

[63]  Grades and incentives: assessing competing grade point average measures and postgraduate outcomes , 2016 .

[64]  Joshua D. Summers,et al.  Requirement Modeling Systems for Mechanical Design: A Systematic Method for Evaluating Requirement M , 2010 .

[65]  Julio Cesar Sampaio do Prado Leite,et al.  Integrating non-functional requirements into data modeling , 1999, Proceedings IEEE International Symposium on Requirements Engineering (Cat. No.PR00188).

[66]  Alison McKay,et al.  Advanced product planning: a comprehensive process for systemic definition of new product requirements , 2008, Requirements Engineering.

[67]  Joshua D. Summers,et al.  Implementing Design Tools in Capstone Design Projects: Requirements Elicitation Through Use of Personas , 2010 .

[68]  Beshoy Morkos,et al.  Computational Representation And Reasoning Support For Requirements Change Management In Complex System Design , 2012 .

[69]  Timothy M. Kowalewski,et al.  Assessing quality of unmet user needs: Effects of need statement characteristics , 2016 .

[70]  N. Ikram,et al.  Making Decision in Requirement Change Management , 2005, 2005 International Conference on Information and Communication Technologies.

[71]  Jean-Yves Fourniols,et al.  Teaching requirements skills within the context of a physical engineering project , 2008, 2008 Requirements Engineering Education and Training.

[72]  Meir M. Lehman,et al.  Program evolution: processes of software change , 1985 .

[73]  Armin Eberlein.  Agile Requirements Definition: A View from Requirements Engineering , 2002 .

[74]  Martin Glinz,et al.  On Non-Functional Requirements , 2007, 15th IEEE International Requirements Engineering Conference (RE 2007).

[75]  Stephen Culley,et al.  A model of factors influencing the design requirement , 2004 .

[76]  Mamoru Maekawa,et al.  Need-based requirements change management , 2001, Proceedings. Eighth Annual IEEE International Conference and Workshop On the Engineering of Computer-Based Systems-ECBS 2001.

[77]  Joshua D. Summers,et al.  Elicitation and Development of Requirements Through Integrated Methods , 2009 .

[78]  S.D.P. Harker,et al.  The change and evolution of requirements as a challenge to the practice of software engineering , 1993, [1993] Proceedings of the IEEE International Symposium on Requirements Engineering.