Improving requirements elicitation: an empirical investigation of procedural prompts

Abstract.  Information system success is dependent upon the effectiveness of requirements elicitation. Requirements elicitation is subject to numerous challenges, however, including cognitive limitations of analysts. In the present study, we test prompts designed to overcome the cognitive limitations that cause analysts to gather inadequate and inaccurate requirements. The study’s results indicate that providing analysts with procedural prompts to aid their requirements elicitation can result in the acquisition of additional meaningful requirements. In addition, the procedural prompts designed to mitigate cognitive challenges were significantly more effective than simple interrogatory prompts. Theoretical implications for the elicitation of requirements and practical implications for the training and practice of systems analysts are discussed.

[1]  Mark Lycett,et al.  Communication issues in requirements elicitation: a content analysis of stakeholder experiences , 2003, Inf. Softw. Technol..

[2]  Glenn J. Browne,et al.  Evoking Information in Probability Assessment: Knowledge Maps and Reasoning-Based Directed Questions , 1997 .

[3]  Edward J. Garrity,et al.  A Problem-solving Perspective on Systems Development , 1990, J. Manag. Inf. Syst..

[4]  A. Graesser,et al.  Mechanisms that generate questions , 1992 .

[5]  Richard J. Boland,et al.  The Process and Product of System Design , 1978 .

[6]  Daniel Kahneman,et al.  Availability: A heuristic for judging frequency and probability , 1973 .

[7]  P. Johnson-Laird,et al.  Psychology of Reasoning: Structure and Content , 1972 .

[8]  N. Schwarz,et al.  The availability heuristic revisited: Ease of recall and content of recall as distinct sources of information. , 2002 .

[9]  Karim Jamal,et al.  Effects of framing on auditor decisions , 1990 .

[10]  H. Simon,et al.  Models Of Man : Social And Rational , 1957 .

[11]  B. Fischhoff,et al.  Reasons for confidence. , 1980 .

[12]  Paul H. Cheney,et al.  A Theoretically Grounded Approach to Assist Memory Recall During Information Requirements Determination , 1998, J. Manag. Inf. Syst..

[13]  Carl-Axel S. Staël von Holstein,et al.  Exceptional Paper---Probability Encoding in Decision Analysis , 1975 .

[14]  Paul Beynon-Davies,et al.  ‘It’s lots of bits of paper and ticks and post‐it notes and things . . .’: a case study of a rapid application development project , 2000, Inf. Syst. J..

[15]  Jack B. Soll,et al.  Overconfidence: It Depends on How, What, and Whom You Ask. , 1999, Organizational behavior and human decision processes.

[16]  James C. Wetherbe.  Executive Information Requirements: Getting It Right , 1991, MIS Q..

[17]  Raymond S. Nickerson,et al.  Confidence Ratings and Level of Performance on a Judgmental Task , 1965, Perceptual and motor skills.

[18]  Margaret Tan,et al.  Establishing Mutual Understanding in Systems Design: An Empirical Study , 1994, J. Manag. Inf. Syst..

[19]  C. Kepner,et al.  The New Rational Manager , 1981 .

[20]  Glenn J. Browne,et al.  Arguments in the practical reasoning underlying constructed probability responses , 1995 .

[21]  Jan Pries-Heje,et al.  Short cycle time systems development , 2004, Inf. Syst. J..

[22]  Hugh J. Watson,et al.  Determining Information Requirements for an EIS , 1993, MIS Q..

[23]  Webb Stacy,et al.  Cognitive bias in software engineering , 1995, CACM.

[24]  Joyce J. Elam,et al.  Inside a software design team: knowledge acquisition, sharing, and integration , 1993, CACM.

[25]  Clifford R. Mynatt,et al.  Confirmation Bias in a Simulated Research Environment: An Experimental Study of Scientific Inference , 1977 .

[26]  P. Wason.  Problem solving and reasoning. , 1971, British medical bulletin.

[27]  Robert W. Zmud,et al.  A Synthesis of Research on Requirements Analysis and Knowledge Acquisition Techniques , 1992, MIS Q..

[28]  Ritu Agarwal,et al.  Knowledge Acquisition Using Structured Interviewing: An Empirical Investigation , 1990, J. Manag. Inf. Syst..

[29]  J. Klayman,et al.  Confirmation, Disconfirmation, and Informa-tion in Hypothesis Testing , 1987 .

[30]  Mark Keil,et al.  The user–developer communication process: a critical case study , 2003, Inf. Syst. J..

[31]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[32]  Clifford R. Mynatt,et al.  Consequences of Confirmation and Disconfirmation in a Simulated Research Environment , 1978 .

[33]  Glenn J. Browne,et al.  An Empirical Investigation of User Requirements Elicitation: Comparing the Effectiveness of Prompting Techniques , 2001, J. Manag. Inf. Syst..

[34]  Venkataraman Ramesh,et al.  Improving information requirements determination: a cognitive perspective , 2002, Inf. Manag..

[35]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.

[36]  Robin Jeffries,et al.  The Processes Involved in Designing Software. , 1980 .

[37]  A. Tversky,et al.  The simulation heuristic , 1982 .

[38]  D. Kahneman,et al.  Representativeness revisited: Attribute substitution in intuitive judgment. , 2002 .

[39]  G. A. Miller.  THE PSYCHOLOGICAL REVIEW THE MAGICAL NUMBER SEVEN, PLUS OR MINUS TWO: SOME LIMITS ON OUR CAPACITY FOR PROCESSING INFORMATION 1 , 1956 .

[40]  Starr Roxanne Hiltz,et al.  Effects of Four Modes of Group Communication on the Outcomes of Software Requirements Determination , 1998, J. Manag. Inf. Syst..

[41]  John R. Anderson.  Cognitive Psychology and Its Implications , 1980 .

[42]  Keng Siau,et al.  Agile Modeling, Agile Software Development, and Extreme Programming: The State of Research , 2005, J. Database Manag..

[43]  Iris Vessey,et al.  Learning to Specify Information Requirements: The Relationship between Application and Methodology , 1993, J. Manag. Inf. Syst..

[44]  D. Kuhn.  THE SKILLS OF ARGUMENT , 2008, Education for Thinking.

[45]  Kenji Takahashi,et al.  Inquiry-based requirements analysis , 1994, IEEE Software.

[46]  Baruch Fischhoff,et al.  Eliciting knowledge for analytical representation , 1989, IEEE Trans. Syst. Man Cybern..

[47]  Gordon B. Davis,et al.  Strategies for Information Requirements Determination , 1982, IBM Syst. J..

[48]  A. Baddeley,et al.  Word length and the structure of short-term memory , 1975 .

[49]  Fatemeh “Mariam” Zahedi.  Requirements analysis and maintenance: where the rubber hits the road in the systems-development cycle , 1999 .

[50]  Justus D. Naumann,et al.  An experimental comparison of abstract and concrete representations in systems analysis , 1992, Inf. Manag..

[51]  Paul Slovic,et al.  Comparison of Bayesian and Regression Approaches to the Study of Information Processing in Judgment. , 1971 .

[52]  Robert W. Zmud,et al.  The Use of Mental Imagery to Facilitate Information Identification in Requirements Analysis , 1993, J. Manag. Inf. Syst..

[53]  Richard J. Boland,et al.  Sense-Making of Accounting Data as a Technique of Organizational Diagnosis , 1984 .

[54]  Gary S. C. Pan.  Information systems project abandonment: a stakeholder analysis , 2005, Int. J. Inf. Manag..

[55]  Gary W. Dickson,et al.  Problem solving for effective systems analysis: an experimental exploration , 1983, CACM.

[56]  Herbert A. Simon,et al.  Information-Processing Theory of Human Problem Solving , 1978 .

[57]  Joyce J. Elam,et al.  Semantic Structuring in Analyst Acquisition and Representation of Facts in Requirements Analysis , 1998, Inf. Syst. Res..

[58]  Bernhard Rumpe,et al.  Assumptions Underlying Agile Software-Development Processes , 2005, J. Database Manag..

[59]  Rachel Reichman-Adar,et al.  Extended Person-Machine Interface , 1984, Artif. Intell..

[60]  R. Conrad,et al.  Acoustic confusions in immediate memory. , 1964 .

[61]  M. Scheerer,et al.  Problem Solving , 1967, Nature.

[62]  Alan M. Davis,et al.  A Unified Model of Requirements Elicitation , 2004, J. Manag. Inf. Syst..

[63]  A. Tenbrunsel,et al.  Organizational Behavior and Human Decision Processes , 2013 .

[64]  W DicksonGary,et al.  Problem solving for effective systems analysis , 1983 .

[65]  Curley,et al.  The Effects of Subject-Defined Categories on Judgmental Accuracy in Confidence Assessment Tasks. , 1999, Organizational behavior and human decision processes.

[66]  Nicholas P. Vitalari,et al.  Structuring the requirements analysis process for information systems: a proposition viewpoint , 1992 .

[67]  Sonia Gatzanis.  Making sense of statistics in psychologyA second level course , 1997 .

[68]  Enid Mumford,et al.  A Socio-Technical Approach to Systems Design , 2000, Requirements Engineering.

[69]  Barry Boehm,et al.  Balancing agility and discipline , 2004 .

[70]  Glenn J. Browne,et al.  An empirical evaluation of context-independent prompting tools for requirements determination , 1998 .

[71]  B. Adelson,et al.  The Role of Domain Expenence in Software Design , 1985, IEEE Transactions on Software Engineering.

[72]  N. Epley,et al.  The Anchoring-and-Adjustment Heuristic , 2006, Psychological science.

[73]  Glenn J. Browne,et al.  Stopping Behavior of Systems Analysts During Information Requirements Elicitation , 2004, J. Manag. Inf. Syst..

[74]  Wendy Grace Lehnert,et al.  The Process of Question Answering , 2022 .

[75]  J. D. Couger,et al.  Creativity/Innovation in Information Systems Organizations , 1995, Proceedings of the Thirtieth Hawaii International Conference on System Sciences.

[76]  B. Fischhoff,et al.  Calibration of probabilities: the state of the art to 1980 , 1982 .