Reconciliation of scrum and the project management process of the ISO/IEC 29110 standard-Entry profile—an experimental evaluation through usability measures

Software process standards and models are used in large- and medium-sized organizations to reach the Iron Triangle. In contrast, small and very small entities either ignore them or cannot apply them because these standards and models are technically and economically not affordable. Consequently, agile software development practices are usually used by small and very small organizations. The ISO/IEC 29110 series of standards and guides are now available for very small organizations, but their utilization with agile practices represents an agility-rigor reconciliation problem. In this research, we report the experimental evaluation of Scrum + EPG (a reconciled agile-rigorous software Project Management process from Scrum, and the Project Management process of the ISO/IEC 29110 series-Entry profile, documented in an Electronic Process Guide). Scrum + EPG was compared to Scrum EPG (a non-modified Scrum process also documented in an Electronic Process Guide). Thirty-two international academicians and practitioners, including experts and novices on agile practices, from Latin America, North America, and Asia–Pacific regions, evaluated six metrics of usability. A within-subjects design and Wilcoxon matched-pairs signed-rank tests were applied for collecting and analyzing the experimental data. The statistical results support the claim that the Scrum + EPG was considered a high-quality conciliated agile-rigorous software Project Management process for the Entry profile. Given the scarcity of similar studies and the need for reconciling agile-rigorous software development practices, this study contributes to a plausible solution for very small organizations. Finally, further empirical research is encouraged to confirm, update, and extend the results reported in this investigation.

[1]  S. Chatterjee,et al.  Design Science Research in Information Systems , 2010 .

[2]  Barry W. Boehm,et al.  Using Risk to Balance Agile and Plan-Driven Methods , 2003, Computer.

[3]  Christiane Gresse von Wangenheim,et al.  Process Reference Guides - Support for Improving Software Processes in Alignment with Reference Models and Standards , 2008, EuroSPI.

[4]  Rory V. O'Connor,et al.  Ignoring "Best Practice": Why Irish Software SMEs are Rejecting CMMI and ISO 9000 , 2009, Australas. J. Inf. Syst..

[5]  Robert Y. Cavana,et al.  Empirical comparison of traditional plan-based and agile methodologies: Critical success factors for outsourced software development projects from vendors' perspective , 2017, J. Enterp. Inf. Manag..

[6]  D. Ross Jeffery,et al.  An exploratory study of why organizations do not adopt CMMI , 2007, J. Syst. Softw..

[7]  Alan R. Hevner,et al.  POSITIONING AND PRESENTING DESIGN SCIENCE RESEARCH FOR MAXIMUM IMPACT 1 , 2013 .

[8]  María José Escalona Cuaresma,et al.  Agile, Web Engineering and Capability Maturity Model Integration: A systematic literature review , 2016, Inf. Softw. Technol..

[9]  Ita Richardson,et al.  Guest Editors' Introduction: Why are Small Software Organizations Different? , 2007, IEEE Software.

[10]  Reidar Conradi,et al.  A Workshop-Oriented Approach for Defining Electronic Process Guides , 2005 .

[11]  Javier Garzás,et al.  A case study of software process improvement with CMMI‐DEV and Scrum in Spanish companies , 2013, J. Softw. Evol. Process..

[12]  Lionel C. Briand,et al.  A Controlled Experiment for Evaluating Quality Guidelines on the Maintainability of Object-Oriented Designs , 2001, IEEE Trans. Software Eng..

[13]  Jörg Leuser,et al.  Empirical Validation of a Requirements Engineering Process Guide , 2009, EASE.

[14]  Rory V. O'Connor,et al.  Software process improvement standards and guides for very small organizations an overview of eight implementations , 2017 .

[15]  Rory V. O'Connor,et al.  A means-ends design of SCRUM+: an agile-disciplined balanced SCRUM enhanced with the ISO/IEC 29110 Standard , 2017 .

[16]  Claes Wohlin,et al.  Experimentation in Software Engineering , 2012, Springer Berlin Heidelberg.

[17]  Pedro Gonçalves,et al.  A Survey of Failures in the Software Development Process , 2017, ECIS.

[18]  Rory O'Connor,et al.  Investigating software process in practice: A grounded theory perspective , 2008, J. Syst. Softw..

[19]  Borja Fernández-Gauna,et al.  A mapping study about the standard ISO/IEC29110 , 2019, Comput. Stand. Interfaces.

[20]  Fred D. Davis,et al.  Explaining Software Developer Acceptance of Methodologies: A Comparison of Five Theoretical Models , 2002, IEEE Trans. Software Eng..

[21]  Mayuram S. Krishnan,et al.  Evaluating the cost of software quality , 1998, CACM.

[22]  Dietmar Pfahl,et al.  Reporting Experiments in Software Engineering , 2008, Guide to Advanced Empirical Software Engineering.

[23]  Seiko Shirasaka,et al.  Report on an assessment experience based on ISO/IEC 29110 , 2014, J. Softw. Evol. Process..

[24]  Wynne W. Chin How to Write Up and Report PLS Analyses , 2010 .

[25]  Mahmood Niazi,et al.  A comparative study of software process improvement implementation success factors , 2015, J. Softw. Evol. Process..

[26]  Ian K. Allison,et al.  Agile Method Tailoring in a CMMI Level 5 Organization , 2013, Journal of International Technology and Information Management.

[27]  Ken Schwaber,et al.  SCRUM Development Process , 1997 .

[28]  Watts S. Humphrey Pathways to Process Maturity: The Personal Software Process and Team Software Process , 1999 .

[29]  John C. Grundy,et al.  The Rise and Evolution of Agile Software Development , 2018, IEEE Software.

[30]  Raman Ramsin,et al.  Towards a method engineering approach for business process reengineering , 2016, IET Softw..

[31]  Abraham Dávila,et al.  Agile Practices Adoption in CMMI Organizations: A Systematic Literature Review , 2016 .

[32]  Mario Piattini,et al.  Using Scrum to guide the execution of software process improvement in small organizations , 2010, J. Syst. Softw..

[33]  Detmar W. Straub,et al.  Information Technology Adoption Across Time: A Cross-Sectional Comparison of Pre-Adoption and Post-Adoption Beliefs , 1999, MIS Q..

[34]  Aurora Vizcaíno,et al.  Process Management Tools , 2011, IEEE Software.

[35]  Alena Buchalcevová,et al.  Using ArchiMate to model ISO/IEC 29110 standard for very small entities , 2019, Comput. Stand. Interfaces.

[36]  Wilhelm Hasselbring,et al.  Effectiveness and efficiency of a domain-specific language for high-performance marine ecosystem simulation: a controlled experiment , 2016, Empirical Software Engineering.

[37]  Alan R. Hevner,et al.  Design Science in Information Systems Research , 2004, MIS Q..

[38]  Pekka Abrahamsson,et al.  Agile Software Development Methods: A Comparative Review1 , 2019, Agile Software Development.

[39]  David J. Sheskin,et al.  Handbook of Parametric and Nonparametric Statistical Procedures , 1997 .

[40]  Gisele Villas Boas,et al.  Small tech firms : Seizing the benefits of software and systems engineering standards , 2013 .

[41]  Rory V. O'Connor,et al.  Impacts of electronic process guides by types of user: An experimental study , 2016, Int. J. Inf. Manag..

[42]  Zahra Shakeri Hossein Abad,et al.  Enhancing Tool Support for Situational Engineering of Agile Methodologies in Eclipse , 2012, SERA.

[43]  Christof Ebert,et al.  The impacts of software product management , 2007, J. Syst. Softw..

[44]  Barry W. Boehm,et al.  Management challenges to implementing agile processes in traditional development organizations , 2005, IEEE Software.

[45]  Pekka Abrahamsson,et al.  Agile Software Development Method, A Comparative Review1 , 2019, ArXiv.

[46]  Claude Y. Laporte,et al.  International systems and software engineering standards for very small entities , 2012 .

[47]  Robert E. Maurer,et al.  Experimental Design with Applications in Management, Engineering and the Sciences , 2001 .

[48]  John R. Anderson Problem solving and learning. , 1993 .

[49]  William E. Riddle,et al.  Process Guides: Effective Guidance for Process Participants , 1998 .

[50]  Christophe Ponsard,et al.  Software development practices in small entities: an ISO29110‐based survey , 2016, J. Softw. Evol. Process..

[51]  Cláudia Maria Lima Werner,et al.  Reconciling software development models: A quasi-systematic review , 2012, J. Syst. Softw..

[52]  Minna Pikkarainen Towards a Better Understanding of CMMI and Agile Integration - Multiple Case Study of Four Companies , 2009, PROFES.

[53]  Gurpreet Dhillon,et al.  User Acceptance of Agile Information Systems: A Model and Empirical Test , 2011, J. Manag. Inf. Syst..

[54]  Tony Gorschek,et al.  Evaluation and Measurement of Software Process Improvement—A Systematic Literature Review , 2012, IEEE Transactions on Software Engineering.

[55]  Barry W. Boehm,et al.  Balancing agility and discipline: evaluating and integrating agile and plan-driven methods , 2004, Proceedings. 26th International Conference on Software Engineering.

[56]  Rory V. O'Connor,et al.  An empirical examination of the extent of software process improvement in software SMEs , 2013, J. Softw. Evol. Process..

[57]  Urvashi Rathod,et al.  Defining ‘success’ for software projects: An exploratory revelation , 2006 .

[58]  Parag C. Pendharkar,et al.  Benchmarking software development productivity of CMMI level 5 projects , 2015, Inf. Technol. Manag..

[59]  Watts S. Humphrey,et al.  Team Software Process (TSP) , 2000 .

[60]  Didar Zowghi,et al.  A Maturity Model for the Implementation of Software Process Improvement: an Empirical Study , 2022 .

[61]  Jeff Sutherland,et al.  Scrum and CMMI Level 5: The Magic Potion for Code Warriors , 2007, Proceedings of the 41st Annual Hawaii International Conference on System Sciences (HICSS 2008).

[62]  Kent L. Beck,et al.  Embracing Change with Extreme Programming , 1999, Computer.

[63]  Carsten Ruseng Jakobsen,et al.  Mature Agile with a Twist of CMMI , 2008, Agile 2008 Conference.

[64]  Kent L. Beck,et al.  Extreme programming explained - embrace change , 1990 .

[65]  Silvio Romero de Lemos Meira,et al.  Using CMMI together with agile software development: A systematic review , 2015, Inf. Softw. Technol..

[66]  Claude Y. Laporte,et al.  Reinforcing Very Small Entities Using Agile Methodologies with the ISO/IEC 29110 , 2018 .

[67]  Venkataraman Ramesh,et al.  An analysis of research in computing disciplines , 2004, CACM.

[68]  Pearl Brereton,et al.  Lessons from applying the systematic literature review process within the software engineering domain , 2007, J. Syst. Softw..

[69]  Victor R. Basili,et al.  How reuse influences productivity in object-oriented systems , 1996, CACM.

[70]  Maureen Tanner,et al.  A Systematic Literature Review of Agile Maturity Model Research , 2017 .

[71]  Rory O'Connor,et al.  The Evolution of the ISO/IEC 29110 Set of Standards and Guides , 2017, Int. J. Inf. Technol. Syst. Approach.

[72]  Tore Dybå,et al.  The use of an electronic process guide in a medium-sized software development company , 2006, Softw. Process. Improv. Pract..

[73]  Sakgasit Ramingwong,et al.  A framework to apply ISO/IEC29110 on SCRUM , 2016, 2016 International Computer Science and Engineering Conference (ICSEC).

[74]  Orit Hazzan,et al.  The Agile Manifesto , 2014 .

[75]  José A. Macías,et al.  Systematic guidance on usability methods in user-centered software development , 2018, Inf. Softw. Technol..

[76]  Allen Newell,et al.  Human problem solving: The state of the theory in 1970. , 1971 .

[77]  Christiane Gresse von Wangenheim,et al.  Empirical evaluation of an educational game on software measurement , 2008, Empirical Software Engineering.

[78]  Tony Gorschek,et al.  Software engineering in start-up companies: An analysis of 88 experience reports , 2018, Empirical Software Engineering.

[79]  Kai Petersen,et al.  Systematic Mapping Studies in Software Engineering , 2008, EASE.

[80]  Claude Y. Laporte,et al.  Building a Guideline to Reinforce Agile Software Development with the Basic Profile of ISO/IEC 29110 in Very Small Entities , 2021 .

[81]  Natalia Juristo Juzgado,et al.  Crossover Designs in Software Engineering Experiments: Benefits and Perils , 2016, IEEE Transactions on Software Engineering.

[82]  Jarmo J. Ahonen,et al.  Software development project success and failure from the supplier's perspective: A systematic literature review , 2012 .

[83]  Ovsei Gelman,et al.  The Case for Conceptual Research in Information Systems , 2008 .

[84]  Rory V. O'Connor,et al.  An Objective Compliance Analysis of Project Management Process in Main Agile Methodologies with the ISO/IEC 29110 Entry Profile , 2017, Int. J. Inf. Technol. Syst. Approach.