Requirements Framing Affects Design Creativity

Design creativity, the originality and practicality of a solution concept is critical for the success of many software projects. However, little research has investigated the relationship between the way desiderata are presented and design creativity. This study therefore investigates the impact of presenting desiderata as ideas, requirements or prioritized requirements on design creativity. Two between-subjects randomized controlled experiments were conducted with 42 and 34 participants. Participants were asked to create design concepts from a list of desiderata. Participants who received desiderata framed as requirements or prioritized requirements created designs that are, on average, less original but more practical than the designs created by participants who received desiderata framed as ideas. This suggests that more formal, structured presentations of desiderata are less appropriate where a creative solution is desired. The results also show that design performance is highly susceptible to minor changes in the vernacular used to communicate desiderata.

[1]  Joachim Karlsson,et al.  Software requirements prioritizing , 1996, Proceedings of the Second International Conference on Requirements Engineering.

[2]  Herbert A. Simon,et al.  The Sciences of the Artificial , 1970 .

[3]  Klaas Sikkel,et al.  Agile requirements prioritization in large-scale outsourced system projects: An empirical study , 2013, J. Syst. Softw..

[4]  Silvia Mara Abrahão,et al.  Managing requirements uncertainty in engine control systems development , 2011, 2011 IEEE 19th International Requirements Engineering Conference.

[5]  Sai Peck Lee,et al.  Functional and non-functional requirements prioritization: empirical evaluation of IPA, AHP-based, and HAM-based approaches , 2015, Soft Computing.

[6]  M. Runco,et al.  The Standard Definition of Creativity , 2012 .

[7]  Forrest Shull,et al.  Building Knowledge through Families of Experiments , 1999, IEEE Trans. Software Eng..

[8]  J. Hassard,et al.  Actor Network Theory and After , 1999 .

[9]  Detlev Leutner,et al.  The impact of goal specificity and goal type on learning outcome and cognitive load , 2009, Comput. Hum. Behav..

[10]  J. Matthews.  The Mind's Best Work , 1985 .

[11]  Ronald A. Finke,et al.  Imagery, Creativity, and Emergent Structure , 1996, Consciousness and Cognition.

[12]  R. Sternberg,et al.  The Cambridge handbook of creativity. , 2010 .

[13]  Nigel Cross,et al.  Expertise in Design: an overview , 2004 .

[14]  B. Burns,et al.  Goal specificity and learning with a hypermedia program. , 2002, Experimental psychology.

[15]  T. M. Amabile.  The social psychology of creativity: A componential conceptualization. , 1983 .

[16]  Mark A. Runco,et al.  Judgments of originality and appropriateness as predictors of creativity , 1993 .

[17]  M. Benedek,et al.  Are creative ideas novel and useful , 2015 .

[18]  Karlyn Adams,et al.  The Sources of Innovation and Creativity , 2005 .

[19]  S. Shapiro,et al.  An Analysis of Variance Test for Normality (Complete Samples) , 1965 .

[20]  Sara Jones,et al.  Requirements Engineering as Creative Problem Solving: A Research Agenda for Idea Finding , 2010, 2010 18th IEEE International Requirements Engineering Conference.

[21]  Ian J. Deary,et al.  What Intelligence Tests Miss: The Psychology of Rational Thought , 2010 .

[22]  P. Alam.  ‘E’ , 2021, Composites Engineering: An A–Z Guide.

[23]  Neil A. M. Maiden,et al.  Where Do Requirements Come From? , 2001, IEEE Softw..

[24]  Pearl Brereton,et al.  Robust Statistical Methods for Empirical Software Engineering , 2017, Empirical Software Engineering.

[25]  Paul Ralph,et al.  The two paradigms of software development research , 2018, Sci. Comput. Program..

[26]  Chad Saunders,et al.  Cognitive heuristics in software engineering applying and extending anchoring and adjustment to artifact reuse , 2004, IEEE Transactions on Software Engineering.

[27]  K. Kotovsky,et al.  The influence of open goals on the acquisition of problem-relevant information. , 2007, Journal of experimental psychology. Learning, memory, and cognition.

[28]  Beverly Roskos-Ewoldsen,et al.  Age‐related Changes in Creative Thinking , 2008 .

[29]  J. M. Singer,et al.  IEEE Recommended Practice for Software Requirements SpeciÞcations , 1993 .

[30]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[31]  E. Paul Torrance,et al.  Predictive Validity of the Torrance Tests of Creative Thinking , 1972 .

[32]  Ronald E. Lesher.  Assessment of Creativity. , 1973 .

[33]  H. Christiaans.  Creativity as a Design Criterion , 2002 .

[34]  Massimiliano Palmiero,et al.  Domain-Specificity of Creativity: A Study on the Relationship Between Visual Creativity and Visual Mental Imagery , 2015, Front. Psychol..

[35]  Hokyoung Ryu,et al.  A Design Thinking Rationality Framework: Framing and Solving Design Problems in Early Concept Generation , 2014, Hum. Comput. Interact..

[36]  Webb Stacy,et al.  Cognitive bias in software engineering , 1995, CACM.

[37]  Paul Ralph,et al.  Toward a Theory of Debiasing Software Development , 2011, SIGSAND.

[38]  Jeffrey V. Nickerson,et al.  A fix for fixation? Rerepresenting and abstracting as creative processes in the design of information systems , 2010, Artificial Intelligence for Engineering Design, Analysis and Manufacturing.

[39]  Joseph A. Goguen,et al.  Requirements engineering: social and technical issues , 1994 .

[40]  Matti Perttula,et al.  The idea exposure paradigm in design idea generation , 2007 .

[41]  A. T. Bahill,et al.  Discovering system requirements , 1996 .

[42]  Nigel Cross,et al.  Research in design Thinking , 1992 .

[43]  Julie S. Linsey,et al.  The effects of representation on idea generation and design fixation: A study comparing sketches and function trees , 2016 .

[44]  Shaochun Xu,et al.  Dialog-based protocol: an empirical research method for cognitive activities in software engineering , 2005, 2005 International Symposium on Empirical Software Engineering, 2005..

[45]  Broderick Crawford,et al.  Agile software engineering as creative work , 2012, 2012 5th International Workshop on Co-operative and Human Aspects of Software Engineering (CHASE).

[46]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.

[47]  S. K. Saha,et al.  A systematic review on creativity techniques for requirements engineering , 2012, 2012 International Conference on Informatics, Electronics & Vision (ICIEV).

[48]  Scott Dexter,et al.  Free and open source software (FOSS) as a model domain for answering big questions about creativity , 2013, Mind & Society.

[49]  Harold G. Nelson,et al.  Design Way: Intentional Change in an Unpredictable World - Foundations and Fundamentals of Design Competence , 2003 .

[50]  Hans Lind,et al.  A note on the robustness of a classical framing result , 1992 .

[51]  Bruno D. Zumbo,et al.  A New Nonparametric Levene Test for Equal Variances , 2010 .

[52]  R. Mayer.  Handbook of Creativity: Fifty Years of Creativity Research , 1998 .

[53]  Venkataraman Ramesh,et al.  Improving information requirements determination: a cognitive perspective , 2002, Inf. Manag..

[54]  R. Weisberg,et al.  Following the wrong footsteps: fixation effects of pictorial examples in a design problem-solving task. , 2005, Journal of experimental psychology. Learning, memory, and cognition.

[55]  Paul Ralph,et al.  Is Requirements Engineering Inherently Counterproductive? , 2015, 2015 IEEE/ACM 5th International Workshop on the Twin Peaks of Requirements and Architecture.

[56]  Tsun Chow,et al.  A survey study of critical success factors in agile software projects , 2008, J. Syst. Softw..

[57]  Graeme G. Shanks,et al.  A framework for understanding creativity in requirements engineering , 2009, Inf. Softw. Technol..

[58]  Manuel E. Sosa,et al.  A Structured Approach to Re-Organize for Creativity , 2009 .

[59]  Ali Selamat,et al.  A systematic literature review of software requirements prioritization research , 2014, Inf. Softw. Technol..

[60]  Steven M. Smith,et al.  Metrics for measuring ideation effectiveness , 2003 .

[61]  Jonas Eckhardt,et al.  Does Quality of Requirements Specifications Matter? Combined Results of Two Empirical Studies , 2015, 2015 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM).

[62]  Peter Checkland,et al.  Learning For Action: A Short Definitive Account of Soft Systems Methodology, and its use for Practitioners, Teachers and Students , 2007 .

[63]  J. Guilford.  Three faces of intellect. , 1959 .

[64]  Xavier Franch,et al.  Proceedings of the Fifth International Workshop on Twin Peaks of Requirements and Architecture , 2014, ICSE 2014.

[65]  Ken Schwaber,et al.  Agile Project Management with Scrum , 1980 .

[66]  Ramesh K. Agarwal,et al.  A MODEL OF CREATIVITY AND INNOVATION IN ORGANIZATIONS , 2011 .

[67]  Thomas Oberlechner.  Psychology of Judgment and Decision-Making , 2006 .

[68]  John S. Gero,et al.  Design and other types of fixation , 1996 .

[69]  David Radcliffe,et al.  Impact of CAD tools on creative problem solving in engineering design , 2009, Comput. Aided Des..

[70]  Nakornthip Prompoon,et al.  Forecasting software damage rate from cognitive bias in software requirements gathering and specification process , 2013, 2013 IEEE Third International Conference on Information Science and Technology (ICIST).

[71]  Linden J. Ball,et al.  Dimensions of Creative Evaluation: Distinct Design and Reasoning Strategies for Aesthetic, Functional and Originality Judgments , 2016 .

[72]  R. Ledesma,et al.  Cliff's Delta Calculator: A non-parametric effect size program for two groups of observations , 2010 .

[73]  Robert J. Youmans,et al.  Design fixation: Classifications and modern methods of prevention , 2014, Artificial Intelligence for Engineering Design, Analysis and Manufacturing.

[74]  Roni Reiter-Palmon,et al.  Effects of Solution Elicitation Aids and Need for Cognition on the Generation of Solutions to Ill-Structured Problems , 2003 .

[75]  Zhang Lujun.  Design fixation and solution quality under exposure to example solution , 2011, 2011 IEEE 2nd International Conference on Computing, Control and Industrial Engineering.

[76]  Christos Tjortjis,et al.  A metric of confidence in requirements gathered from legacy systems: two industrial case studies , 2006, Conference on Software Maintenance and Reengineering (CSMR'06).

[77]  Raymonde Guindon,et al.  Knowledge Exploited by Experts during Software System Design , 1990, Int. J. Man Mach. Stud..

[78]  Paul Ralph,et al.  Requirements fixation , 2014, ICSE.

[79]  Paul Ralph.  The illusion of requirements in software development , 2012, Requirements Engineering.

[80]  P. Slovic.  The Construction of Preference , 1995 .

[81]  GuindonRaymonde.  Knowledge exploited by experts during software system design , 1990 .

[82]  Magne Jørgensen,et al.  Software Development Estimation Biases: The Role of Interdependence , 2012, IEEE Transactions on Software Engineering.

[83]  James John Gallagher.  Teaching gifted students : a book of readings , 1965 .

[84]  Vesna Nikolic,et al.  A survey study of critical success factors in agile software projects in former Yugoslavia IT companies , 2013, J. Syst. Softw..

[85]  John Baer,et al.  Bridging generality and specificity: The amusement park theoretical (APT) model of creativity , 2005 .

[86]  Ayse Basar Bener,et al.  Empirical analyses of the factors affecting confirmation bias and the effects of confirmation bias on software developer/tester performance , 2010, PROMISE '10.

[87]  Radhika Jain,et al.  Using traceability to mitigate cognitive biases in software development , 2008, CACM.

[88]  Neil A. M. Maiden,et al.  Provoking Creativity: Imagine What Your Requirements Could Be Like , 2004, IEEE Softw..

[89]  Paul Ralph,et al.  Software engineering process theory: A multi-method comparison of Sensemaking-Coevolution-Implementation Theory and Function-Behavior-Structure Theory , 2013, Inf. Softw. Technol..

[90]  Teresa M. Amabile,et al.  Creativity and Innovation in Organizations , 1996 .

[91]  A. Tversky,et al.  The framing of decisions and the psychology of choice. , 1981, Science.

[92]  M. Mumford.  Where Have We Been, Where Are We Going? Taking Stock in Creativity Research , 2003 .