Using CMMI together with agile software development: A systematic review

Abstract Background The search for adherence to maturity levels by using lightweight processes that require low levels of effort is regarded as a challenge for software development organizations. Objective This study seeks to evaluate, synthesize, and present results on the use of the Capability Maturity Model Integration (CMMI) in combination with agile software development, and thereafter to give an overview of the topics researched, which includes a discussion of their benefits and limitations, the strength of the findings, and the implications for research and practice. Methods The method applied was a Systematic Literature Review on studies published up to (and including) 2011. Results The search strategy identified 3193 results, of which 81 included studies on the use of CMMI together with agile methodologies. The benefits found were grouped into two main categories: those related to the organization in general and those related to the development process, and were organized into subcategories, according to the area to which they refer. The limitations were also grouped into these categories. Using the criteria defined, the strength of the evidence found was considered low. The implications of the results for research and practice are discussed. Conclusion Agile methodologies can be used by companies to reduce efforts in getting to levels 2 and 3 of CMMI, there even being reports of applying agile practices that led to achieving level 5. However, agile methodologies alone, according to the studies, were not sufficient to obtain a rating at a given level, it being necessary to resort to additional practices to do so.

[1]  Ivan A. Garcia,et al.  Using a web-based tool to define and implement software process improvement initiatives in a small industrial setting , 2010, IET Softw..

[2]  Carsten Ruseng Jakobsen,et al.  Mature Agile with a Twist of CMMI , 2008, Agile 2008 Conference.

[3]  Çigdem Gencel,et al.  Why a CMMI Level 5 Company Fails to Meet the Deadlines? , 2009, PROFES.

[4]  Donald J. Reifer XP and the CMM , 2003, IEEE Softw..

[5]  G. Noblit,et al.  Meta-Ethnography: Synthesizing Qualitative Studies , 1988 .

[6]  Barry Boehm,et al.  A view of 20th and 21st century software engineering , 2006, ICSE.

[7]  Christ Vriens,et al.  7 Years of Agile Management , 2008, Agile 2008 Conference.

[8]  Keng Siau,et al.  Agile Modeling, Agile Software Development, and Extreme Programming: The State of Research , 2005, J. Database Manag..

[9]  Michela Montesi,et al.  Software engineering article types: An analysis of the literature , 2008, J. Syst. Softw..

[10]  Raymund Sison,et al.  Use of Agile Methods and Practices in the Philippines , 2007, 14th Asia-Pacific Software Engineering Conference (APSEC'07).

[11]  David J. Anderson,et al.  Stretching agile to fit CMMI level 3 - the story of creating MSF for CMMI/spl reg/ process improvement at Microsoft corporation , 2005, Agile Development Conference (ADC'05).

[12]  Justin Babuscio,et al.  How the FBI Learned to Catch Bad Guys One Iteration at a Time , 2009, 2009 Agile Conference.

[13]  Steven W. Baker Formalizing agility, part 2: how an agile organization embraced the CMMI , 2006, AGILE 2006 (AGILE'06).

[14]  Jezreel Mejia,et al.  Definition of a Hybrid Measurement Process for the Models ISO/IEC 15504-ISO/IEC 12207:2008 and CMMI Dev 1.3 in SMEs , 2011, 2011 IEEE Electronics, Robotics and Automotive Mechanics Conference.

[15]  Pekka Abrahamsson,et al.  Agile Software Development Methods: Review and Analysis , 2017, ArXiv.

[16]  Marten van Sinderen,et al.  Software reuse in agile development organizations: a conceptual management tool , 2011, SAC '11.

[17]  Richard Turner,et al.  Agile Development: Good Process or Bad Attitude? , 2002, PROFES.

[18]  Fergal McCaffery,et al.  Adept: A Unified Assessment Method for Small Software Companies , 2007, IEEE Software.

[19]  M. A. Qureshi,et al.  Agile methodology in software development (SMEs) of Pakistan software industry for successful software projects (CMM framework) , 2010, 2010 International Conference on Educational and Network Technology.

[20]  Alexandre Marcos Lins de Vasconcelos,et al.  Agile Software Development and CMMI: What We Do Not Know about Dancing with Elephants , 2009, XP.

[21]  Steve R. Palmer,et al.  A Practical Guide to Feature-Driven Development , 2002 .

[22]  Rafael Prikladnicki,et al.  Transitioning from Distributed and Traditional to Distributed and Agile: An Experience Report , 2010, Agility Across Time and Space.

[23]  Tony Gorschek,et al.  Evaluation and Measurement of Software Process Improvement—A Systematic Literature Review , 2012, IEEE Transactions on Software Engineering.

[24]  Pearl Brereton,et al.  Performing systematic literature reviews in software engineering , 2006, ICSE.

[25]  Markus M. Mäkelä,et al.  Measuring and Comparing the Adoption of Software Process Practices in the Software Product Industry , 2008, ICSP.

[26]  Horst Lichter,et al.  SPIALS: A light-weight Software Process Improvement self-assessment tool , 2011, 2011 Malaysian Conference in Software Engineering.

[27]  Mark C. Paulk,et al.  Agile Methodologies and Process Discipline , 2002 .

[28]  Julio Ariel Hurtado Alegria,et al.  Implementing CMMI using a Combination of Agile Methods , 2006, CLEI Electron. J..

[29]  Prasanth Ganesan,et al.  Pair Programming: Addressing Key Process Areas of the People-CMM , 2002, XP/Agile Universe.

[30]  Mark Lycett,et al.  Migrating Agile Methods to Standardized Development Practice , 2003, Computer.

[31]  Karl Lebsanft Process Improvement in Turbulent Times - Is CMM Still an Answer? , 2001, PROFES.

[32]  Narinder Pal Singh,et al.  Agile Software: Ensuring Quality Assurance and Processes , 2011, Grid 2011.

[33]  Mikael Lindvall,et al.  An introduction to agile methods , 2004, Adv. Comput..

[34]  Eisha Hasnain,et al.  An overview of published agile studies: a systematic literature review , 2010, NSEC '10.

[35]  Klaas Sikkel,et al.  Value Creation by Agile Projects: Methodology or Mystery? , 2009, PROFES.

[36]  Beijun Shen,et al.  A Case Study of Software Process Improvement in a Chinese Small Company , 2008, 2008 International Conference on Computer Science and Software Engineering.

[37]  Steven W. Baker Formalizing agility: an agile organization's journey toward CMMI accreditation , 2005, Agile Development Conference (ADC'05).

[38]  Tore Dybå,et al.  Empirical studies of agile software development: A systematic review , 2008, Inf. Softw. Technol..

[39]  Jeff Sutherland,et al.  Manifesto for Agile Software Development , 2013 .

[40]  Apurva Jain,et al.  Agile Meets CMMI: Culture Clash or Common Cause? , 2002, XP/Agile Universe.

[41]  Drew Hamilton,et al.  Agile versus CMMI - process template selection and integration with microsoft team foundation server , 2008, ACM-SE 46.

[42]  Jeff Sutherland,et al.  Scrum and CMMI Level 5: The Magic Potion for Code Warriors , 2007, Proceedings of the 41st Annual Hawaii International Conference on System Sciences (HICSS 2008).

[43]  Sean Cohan,et al.  An Agile Development Team's Quest for CMMI® Maturity Level 5 , 2009, 2009 Agile Conference.

[44]  Narti Kitiyakara,et al.  Growing a Build Management System from Seed , 2007, Agile 2007 (AGILE 2007).

[45]  S. Raman Lean software development: is it feasible? , 1998, 17th DASC. AIAA/IEEE/SAE. Digital Avionics Systems Conference. Proceedings (Cat. No.98CH36267).

[46]  Lazar Rusu,et al.  Management Guidelines for Database Developers' Teams in Software Development Projects , 2009, WSKS.

[47]  G. Guyatt,et al.  Grading quality of evidence and strength of recommendations , 2004, BMJ : British Medical Journal.

[48]  Rajesh Krishna Balan,et al.  The impact of process choice in high maturity environments: An empirical analysis , 2009, 2009 IEEE 31st International Conference on Software Engineering.

[49]  Adam Wojciechowski,et al.  Toward maturity model for extreme programming , 2001, Proceedings 27th EUROMICRO Conference. 2001: A Net Odyssey.

[50]  Juan Garbajosa,et al.  Mapping CMMI Level 2 to Scrum Practices: An Experience Report , 2009, EuroSPI.

[51]  Jeff Sutherland,et al.  Scrum and CMMI Going from Good to Great , 2009, 2009 Agile Conference.

[52]  Furtado Soares,et al.  An Agile Maturity Model for Software Development Organizations , 2013, ICSEA 2013.

[53]  Adam Wojciechowski,et al.  Extreme programming modified: embrace requirements engineering practices , 2002, Proceedings IEEE Joint International Conference on Requirements Engineering.

[54]  Paul E. McMahon,et al.  Are the Right People Measuring the Right Things? A Lean Path to Achieving Business Objectives , 2008 .

[55]  Mark Staples,et al.  Systematic review of organizational motivations for adopting CMM-based SPI , 2008, Inf. Softw. Technol..

[56]  Enrico de Sousa Visconti,et al.  Software Development: Planning x Agility , 2003, Software Engineering Research and Practice.

[57]  Marion Lepmets,et al.  Comparison of Plan-driven and Agile Project Management Approaches: Theoretical Bases for a Case Study in Estonian Software Industry , 2010, DB&IS.

[58]  Lars Mathiassen,et al.  Agility : How to Navigate Improvement Projects Research Section , 2007 .

[59]  Udayan Banerjee,et al.  Experience of executing fixed price off-shored agile project , 2011, ISEC.

[60]  Werasak Kurutach,et al.  Applying Agility Framework in Small and Medium Enterprises , 2009, FGIT-ASEA.

[61]  Michael R. Smith,et al.  Configuring Hybrid Agile-Traditional Software Processes , 2006, XP.

[62]  Tom Poppendieck,et al.  Lean as a Scrum Troubleshooter , 2011, 2011 AGILE Conference.

[63]  Gerald Quirchmayr,et al.  A Prototype for the Support of Integrated Software Process Development and Improvement , 2010, IAIT.

[64]  Mary Poppendieck,et al.  Lean Software Development: An Agile Toolkit , 2003 .

[65]  Jonas Martinsson Maturing XP through the CMM , 2003, XP.

[66]  John Surdu,et al.  Army Simulation Program Balances Agile and Traditional Methods with Success , 2006 .

[67]  Cláudia Maria Lima Werner,et al.  Reconciling software development models: A quasi-systematic review , 2012, J. Syst. Softw..

[68]  Emilia Mendes,et al.  A Systematic Literature Review of Software Process Improvement in Small and Medium Web Companies , 2009, FGIT-ASEA.

[69]  Haeng-Kon Kim,et al.  Enterprise Process Model for Extreme Programming with CMMI Framework , 2008, Computer and Information Science.

[70]  Francisco J. Pino,et al.  Applying Agile and Lean Practices in a Software Development Project into a CMMI Organization , 2011, PROFES.

[71]  Zakaria Maamar,et al.  Incorporating innovative practices in software engineering education , 2011, 2011 IEEE Global Engineering Education Conference (EDUCON).

[72]  Mark C. Paulk,et al.  Extreme Programming from a CMM Perspective , 2001, IEEE Softw..

[73]  Minna Pikkarainen Towards a Better Understanding of CMMI and Agile Integration - Multiple Case Study of Four Companies , 2009, PROFES.

[74]  Carl Magnus Olsson,et al.  The Role of Extreme Programming in a Plan-Driven Organization , 2006, The Transfer and Diffusion of Information Technology for Organizational Resilience.

[75]  Pekka Abrahamsson,et al.  Integrating agile software development and software process improvement: a longitudinal case study , 2005, 2005 International Symposium on Empirical Software Engineering, 2005..

[76]  Kent L. Beck,et al.  Extreme programming explained - embrace change , 1990 .

[77]  Nikolay Todorov,et al.  Optimizing software development process: A case study for integrated Agile-CMMI process model , 2011, 2011 IEEE EUROCON - International Conference on Computer as a Tool.

[78]  Pekka Abrahamsson,et al.  Strengths and barriers behind the successful agile deployment—insights from the three software intensive companies in Finland , 2011, Empirical Software Engineering.

[79]  Elizabeth Furtado,et al.  Blending Scrum practices and CMMI project management process areas , 2008, Innovations in Systems and Software Engineering.

[80]  Gustav Boström,et al.  Security Engineering and eXtreme Programming: An Impossible Marriage? , 2004, XP/Agile Universe.

[81]  Pekka Abrahamsson,et al.  Achieving CMMI Level 2 with Enhanced Extreme Programming Approach , 2017, PROFES.

[82]  Joe Fecarotta MyBoeingFleet and Agile Software Development , 2008, Agile 2008 Conference.

[83]  Silvio Romero de Lemos Meira,et al.  Benefits and Limitations of Using the MPS.BR Model with Agile Methodologies: A Survey Based on a Systematic Literature Review , 2013, ICSEA 2013.

[84]  Jeffrey L. Dutton An Integrated Framework for Performance Excellence , 2010 .

[85]  JaatunMartin Gilje,et al.  Agile Software Development , 2002, Comput. Sci. Educ..

[86]  Christ Vriens Certifying for CMM Level 2 and IS09001 with XP@Scrum , 2003, Proceedings of the Agile Development Conference, 2003. ADC 2003.

[87]  Muthu Ramachandran,et al.  Story Card Maturity Model (SMM): A Process Improvement Framework for Agile Requirements Engineering Practices , 2009, J. Softw..

[88]  Andraž Cej,et al.  Agile software development with Scrum , 2010 .

[89]  Xavier Franch,et al.  Reconciling Agility and Discipline in COTS Selection Processes , 2007, 2007 Sixth International IEEE Conference on Commercial-off-the-Shelf (COTS)-Based Software Systems (ICCBSS'07).

[90]  Ali Selamat,et al.  Information and Software Technology , 2014 .

[91]  Tihana Galinac Grbac Empirical evaluation of selected best practices in implementation of software process improvement , 2009, Inf. Softw. Technol..

[92]  Michele Marchesi,et al.  Extreme Programming and Agile Processes in Software Engineering , 2003, Lecture Notes in Computer Science.

[93]  Dean Knudson,et al.  Updating CS capstone projects to incorporate new agile methodologies used in industry , 2011, 2011 24th IEEE-CS Conference on Software Engineering Education and Training (CSEE&T).

[94]  Sjaak Brinkkemper,et al.  Using Linear Regression Models to Analyse the Effect of Software Process Improvement , 2006, PROFES.

[95]  Hillel Glazer,et al.  Cmmi Isn't for Everyone— It's for Experts Love and Marriage: Cmmi and Agile Need Each Other , 2022 .

[96]  Annukka Mäntyniemi,et al.  An Approach for Using CMMI in Agile Software Development Assessments: Experiences from Three Case Studies , 2006 .

[97]  James A. Highsmith,et al.  Adaptive Software Development: A Collaborative Approach to Managing Complex Systems , 1999 .

[98]  Fergal McCaffery,et al.  Ahaa --agile, hybrid assessment method for automotive, safety critical smes , 2008, 2008 ACM/IEEE 30th International Conference on Software Engineering.

[99]  A. Omran AGILE CMMI from SMEs perspective , 2008, 2008 3rd International Conference on Information and Communication Technologies: From Theory to Applications.