Combining Perceptions and Prescriptions in Requirements Engineering Process Assessment: An Industrial Case Study

Requirements engineering (RE) is a key discipline in software development and several methods are available to help assess and improve RE processes. However, these methods rely on prescriptive models of RE; they do not, like other disciplines within software engineering, draw directly on stakeholder perceptions and subjective judgments. Given this backdrop, we present an empirical study in RE process assessment. Our aim was to investigate how stakeholder perceptions and process prescriptions can be combined during assessments to effectively inform RE process improvement. We first describe existing methods for RE process assessment and the role played by stakeholder perceptions and subjective judgments in the software engineering and management literature. We then present a method that combines perceptions and prescriptions in RE assessments together with an industrial case study in which the method was applied and evaluated over a three-year period at TelSoft. The data suggest that the combined method led to a comprehensive and rich assessment and it helped TelSoft consider RE as an important and integral part of the broader engineering context. This, in turn, led to improvements that combined plan-driven and adaptive principles for RE. Overall, the combined method helped TelSoft move from Level 1 to Level 2 in RE maturity, and the employees perceived the resulting engineering practices to be improved. Based on these results, we suggest that software managers and researchers combine stakeholder perceptions and process prescriptions as one way to effectively balance the specificity, comparability, and accuracy of software process assessments.

[1]  Khaled El Emam,et al.  Measuring the success of requirements engineering processes , 1995, Proceedings of 1995 IEEE International Symposium on Requirements Engineering (RE'95).

[2]  Mahmood Niazi An Instrument for Measuring the Maturity of Requirements Engineering Process , 2005, PROFES.

[3]  Bob McFeeley,et al.  IDEAL: A User's Guide for Software Process Improvement. , 1996 .

[4]  Peter Axel Nielsen,et al.  Accessing Software Processes: Low Maturity or Sensible Practice , 2001, Scand. J. Inf. Syst..

[5]  G. Susman,et al.  An Assessment of the Scientific Merits of Action Research. , 1978 .

[6]  Terence Patrick Rout,et al.  Harmonizing ISO-IEC 15504 and CMMI , 2007 .

[7]  A. Pettigrew Longitudinal Field Research on Change: Theory and Practice , 1990 .

[8]  Victor R. Basili,et al.  The TAME Project: Towards Improvement-Oriented Software Environments , 1988, IEEE Trans. Software Eng..

[9]  Ita Richardson,et al.  Research into long-term improvements in small- to medium-sized organisations using SPICE as a framework for standards , 2007, Softw. Process. Improv. Pract..

[10]  Reijo Sulonen,et al.  Implementing requirements engineering processes throughout organizations: success factors and challenges , 2004, Inf. Softw. Technol..

[11]  Fergal McCaffery,et al.  A Low-overhead method for software process appraisal , 2007, Softw. Process. Improv. Pract..

[12]  Jan Pries-Heje,et al.  A Framework for Selecting Assessment Strategy , 2002 .

[13]  Kalle Lyytinen,et al.  Components of Software Development Risk: How to Address Them? A Project Manager Survey , 2000, IEEE Trans. Software Eng..

[14]  Maya Daneva,et al.  Lessons learnt from five years of experience in ERP requirements engineering , 2003, Proceedings. 11th IEEE International Requirements Engineering Conference, 2003..

[15]  Ian Sommerville,et al.  Requirements Engineering: Processes and Techniques , 1998 .

[16]  Watts S. Humphrey,et al.  Managing the software process , 1989, The SEI series in software engineering.

[17]  Carolyn B. Seaman,et al.  Qualitative Methods in Empirical Studies of Software Engineering , 1999, IEEE Trans. Software Eng..

[18]  Tony Gorschek,et al.  A practitioner's guide to light weight software process assessment and improvement planning , 2008, J. Syst. Softw..

[19]  Claes Wohlin,et al.  Prioritizing and Assessing Software Project Success Factors and Project Characteristics using Subjective Data , 2004, Empirical Software Engineering.

[20]  Richard Baskerville,et al.  Diversity in information systems action research methods , 1998 .

[21]  Maya Daneva,et al.  Using maturity assessments to understand the ERP requirements engineering process , 2002, Proceedings IEEE Joint International Conference on Requirements Engineering.

[22]  Bernard Burnes,et al.  Managing Change , 2004 .

[23]  Lars Mathiassen,et al.  Managing change in software process improvement , 2005, IEEE Software.

[24]  Mark J. Tiedeman Post-Mortems - Methodology and Experiences , 1990, IEEE J. Sel. Areas Commun..

[25]  Lars Mathiassen,et al.  Business process innovation based on stakeholder perceptions , 2007, Inf. Knowl. Syst. Manag..

[26]  Peter Checkland,et al.  Soft Systems Methodology: a 30-year retrospective , 1999 .

[27]  Robert T. Hughes,et al.  Expert judgement as an estimating method , 1996, Inf. Softw. Technol..

[28]  Marjo Kauppinen,et al.  Introducing requirements engineering: how to make a cultural change happen in practice , 2002, Proceedings IEEE Joint International Conference on Requirements Engineering.

[29]  Austen Rainer,et al.  Defining a Requirements Process Improvement Model , 2005, Software Quality Journal.

[30]  Marjo Kauppinen,et al.  Lessons Learned from Applying the Requirements Engineering Good Practice Guide for Process Improvement , 2002, ECSQ.

[31]  Kalle Lyytinen,et al.  Attention Shaping and Software Risk - A Categorical Analysis of Four Classical Risk Management Approaches , 1998, Inf. Syst. Res..

[32]  R. Rapoport Three Dilemmas in Action Research , 1970 .

[33]  Lars Mathiassen,et al.  Information-centric assessment of software metrics practices , 2005, IEEE Transactions on Engineering Management.

[34]  Barry W. Boehm,et al.  Get Ready for Agile Methods, with Care , 2002, Computer.

[35]  Claes Wohlin,et al.  A subjective effort estimation experiment , 1997, Inf. Softw. Technol..

[36]  Lars Mathiassen,et al.  The post mortem paradox: a Delphi study of IT specialist perceptions , 2008, Eur. J. Inf. Syst..

[37]  Daniela E. Damian,et al.  An Industrial Case Study of Immediate Benefits of Requirements Engineering Process Improvement at the Australian Center for Unisys Software , 2004, Empirical Software Engineering.

[38]  Eben A. Weitzman,et al.  Computer Programs for Qualitative Data Analysis , 1996 .

[39]  Matthew B. Miles,et al.  Qualitative Data Analysis: An Expanded Sourcebook , 1994 .

[40]  Timo Varkoi,et al.  Standard based software process assessments in small companies , 2006, Softw. Process. Improv. Pract..

[41]  Nannette P. Napier,et al.  Improving practices in a small software firm: an ambidextrous perspective , 2007 .

[42]  Austen Rainer,et al.  Using an expert panel to validate a requirements process improvement model , 2005, J. Syst. Softw..

[43]  Jean-Martin Simon SPICE: Overview for software process improvement , 1996, J. Syst. Archit..

[44]  Shari Lawrence Pfleeger,et al.  Preliminary Guidelines for Empirical Research in Software Engineering , 2002, IEEE Trans. Software Eng..

[45]  Kalle Lyytinen,et al.  Identifying Software Project Risks: An International Delphi Study , 2001, J. Manag. Inf. Syst..

[46]  Khaled El Emam,et al.  Validating the ISO/IEC 15504 Measure of Software Requirements Analysis Process Capability , 2000, IEEE Trans. Software Eng..

[47]  Lars Mathiassen,et al.  Collaborative Practice Research , 2000, Scand. J. Inf. Syst..

[48]  Xiaoqing Frank Liu,et al.  Priority assessment of software process requirements from multiple perspectives , 2006, J. Syst. Softw..

[49]  Ian Sommerville,et al.  An empirical study of industrial requirements engineering process assessment and improvement , 2005, TSEM.

[50]  Mark C. Paulk,et al.  Creating a software process improvement program , 1993, Inf. Softw. Technol..

[51]  Jan Pries-Heje,et al.  Improving Software Organizations: From Principles to Practice , 2001 .