What is DevOps?: A Systematic Mapping Study on Definitions and Practices

Context: DevOps, the combination of Development and Operations, is a new way of thinking in the software engineering domain that recently received much attention. Given that DevOps is a new term and novel concept recently introduced, no common understanding of what it entails has been achieved yet. Consequently, definitions of DevOps often only represent a part that is relevant to the concept. Objective:This study aims to characterize DevOps by exploring central components of DevOps definitions reported in the literature, specifying practices explicitly proposed for DevOps and investigating the similarities and differences between DevOps and other existing methods in software engineering. Method: A systematic mapping study was conducted that used six electronic databases: IEEE, ACM, Inspec, Scopus, Wiley Online Library and Web of Science. Result: 44 studies have been selected that report a definition of DevOps, 15 studies explicitly stating DevOps practices, and 15 studies stating how DevOps is related to other existing methods. Papers in some cases stated a combination of a definition, practices, and relations to other methods, the total number of primary studies was 49. Conclusion: We proposed a definition for DevOps which may overcome inconsistencies over the various existing definitions of individual research studies. In addition, the practices explicitly proposed for DevOps have been presented as well as the relation to other software development methods.

[1]  Geoffrey C. Fox,et al.  HPC-ABDS High Performance Computing Enhanced Apache Big Data Stack , 2015, 2015 15th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing.

[2]  Maya Daneva,et al.  Cooperation between information system development and operations: a literature review , 2014, ESEM '14.

[3]  Tom Bryan,et al.  VIRL: the virtual internet routing lab , 2014, SIGCOMM.

[4]  Liming Zhu,et al.  Eliciting operations requirements for applications , 2013, 2013 1st International Workshop on Release Engineering (RELENG).

[5]  Frank Leymann,et al.  Automated Capturing and Systematic Usage of DevOps Knowledge for Cloud Applications , 2015, 2015 IEEE International Conference on Cloud Engineering.

[6]  David Lorge Parnas,et al.  Stop the numbers game , 2007, CACM.

[7]  Marina Waldén,et al.  DevOps meets formal modelling in high-criticality complex systems , 2015, QUDOS@SIGSOFT FSE.

[8]  Shigeru Hosono.  A DevOps framework to shorten delivery time for cloud applications , 2012, Int. J. Comput. Sci. Eng..

[9]  Danilo Ardagna,et al.  SPACE4Cloud: a DevOps environment for multi-cloud applications , 2015, QUDOS@SIGSOFT FSE.

[10]  JongWon Kim.  Preparing the end-to-end virtualized networking over software-defined infrastrure , 2014, 2014 12th International Conference on Optical Internet 2014 (COIN).

[11]  Frank Leymann,et al.  Compensation-Based vs. Convergent Deployment Automation for Services Operated in the Cloud , 2014, ICSOC.

[12]  Syed W. Hussaini.  Strengthening harmonization of Development (Dev) and Operations (Ops) silos in IT environment through systems approach , 2014, 17th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[13]  Wolfgang Gottesheim.  Challenges, Benefits and Best Practices of Performance Focused DevOps , 2015, LT@ICPE.

[14]  Alan Sill.  Factors in Development and Adoption of New Cloud Software and Standards , 2014, IEEE Cloud Computing.

[15]  Klaas-Jan Stol,et al.  Continuous software engineering: A roadmap and agenda , 2017, J. Syst. Softw..

[16]  Brian Fitzgerald,et al.  Continuous software engineering and beyond: trends and challenges , 2014, RCoSE 2014.

[17]  Brian M. Belgodere,et al.  Composable DevOps: Automated Ontology Based DevOps Maturity Analysis , 2015, 2015 IEEE International Conference on Services Computing.

[18]  Thomas A. Mikalsen,et al.  Continuous Delivery of Composite Solutions: A Case for Collaborative Software Defined PaaS Environments , 2015, BigSystem@HPDC.

[19]  Tommi Mikkonen,et al.  The social developer: now, then, and tomorrow , 2015, SSE@SIGSOFT FSE.

[20]  Marc J. Dupuis,et al.  A grounded theory analysis of modern web applications: knowledge, skills, and abilities for DevOps , 2013, RIIT '13.

[21]  Marco Miglierina.  Application Deployment and Management in the Cloud , 2014, 2014 16th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing.

[22]  Pooyan Jamshidi,et al.  Modelling multi-tier enterprise applications behaviour with design of experiments technique , 2015, QUDOS@SIGSOFT FSE.

[23]  Giuliano Casale,et al.  Filling the gap: a tool to automate parameter estimation for software performance models , 2015, QUDOS@SIGSOFT FSE.

[24]  Chris Gaughan,et al.  U.S. Army Modeling and Simulation Executable Architecture Deployment Cloud Virtualization Strategy , 2012, 2012 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (ccgrid 2012).

[25]  Rebecca Steinert,et al.  Service provider DevOps for large scale modern network services , 2015, 2015 IFIP/IEEE International Symposium on Integrated Network Management (IM).

[26]  Gergely Pongrácz,et al.  Unifying Cloud and Carrier Network: EU FP7 Project UNIFY , 2013, 2013 IEEE/ACM 6th International Conference on Utility and Cloud Computing.

[27]  Kai Petersen,et al.  Agile Software Development Practice Adoption Survey , 2012, XP.

[28]  Antonio Puliafito,et al.  CloudWave: Where adaptive cloud management meets DevOps , 2014, 2014 IEEE Symposium on Computers and Communications (ISCC).

[29]  B. S. Farroha,et al.  A Framework for Managing Mission Needs, Compliance, and Trust in the DevOps Environment , 2014, 2014 IEEE Military Communications Conference.

[30]  Alain Abran,et al.  Guide to the Software Engineering Body of Knowledge : 2004 Version , 2005 .

[31]  Steven A. Wright,et al.  Open source and standards: The role of open source in the dialogue between research and standardization , 2014, 2014 IEEE Globecom Workshops (GC Wkshps).

[32]  K. R. Jayaram.  Towards Explicitly Elastic Programming Frameworks , 2015, 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering.

[33]  Giuliano Casale,et al.  Towards a DevOps Approach for Software Quality Engineering , 2015, WOSP '15.

[34]  JongWon Kim,et al.  Preparing and Inter-Connecting Hyper-Converged SmartX Boxes for IoT-Cloud Testbed , 2015, 2015 IEEE 29th International Conference on Advanced Information Networking and Applications.

[35]  Pearl Brereton,et al.  A systematic review of systematic review process research in software engineering , 2013, Inf. Softw. Technol..

[36]  Maya Daneva,et al.  A Mapping Study on Cooperation between Information System Development and Operations , 2014, PROFES.

[37]  Frank Leymann,et al.  Dyn Tail - Dynamically Tailored Deployment Engines for Cloud Applications , 2015, 2015 IEEE 8th International Conference on Cloud Computing.

[38]  Horst Lichter,et al.  Towards Definitions for Release Engineering and DevOps , 2015, 2015 IEEE/ACM 3rd International Workshop on Release Engineering.

[39]  Pearl Brereton,et al.  Performing systematic literature reviews in software engineering , 2006, ICSE.

[40]  James Roche,et al.  Adopting DevOps practices in quality assurance , 2013, CACM.

[41]  Frank Leymann,et al.  Compensation and Convergence - Comparing and Combining Deployment Automation Approaches , 2015, Int. J. Cooperative Inf. Syst..

[42]  Sunil Joglekar,et al.  Towards behavior driven operations (BDOps) , 2011, ARTCom 2011.

[43]  Ivan Porres,et al.  DevOps: A Definition and Perceived Adoption Impediments , 2015, XP.

[44]  Harald C. Gall,et al.  Cloud WorkBench: Benchmarking IaaS Providers based on Infrastructure-as-Code , 2015, WWW.

[45]  Diomidis Spinellis.  Package Management Systems , 2012, IEEE Software.

[46]  Barbara Kitchenham,et al.  What's up with software metrics? - A preliminary mapping study , 2010, J. Syst. Softw..

[47]  Pallavi Joshi,et al.  SAMC: Semantic-Aware Model Checking for Fast Discovery of Deep Bugs in Cloud Systems , 2014, OSDI.

[48]  Manish Virmani,et al.  Understanding DevOps & bridging the gap from continuous integration to continuous delivery , 2015, Fifth International Conference on the Innovative Computing Technology (INTECH 2015).

[49]  Shigeru Hosono,et al.  Application Lifecycle Kit for Mass Customization on PaaS Platforms , 2012, 2012 IEEE Eighth World Congress on Services.

[50]  Frank Leymann,et al.  Integrating Configuration Management with Model-driven Cloud Management based on TOSCA , 2013, CLOSER.

[51]  Leonardo Guerreiro Azevedo,et al.  ResearchOps: The case for DevOps in scientific applications , 2015, 2015 IFIP/IEEE International Symposium on Integrated Network Management (IM).

[52]  Alfonso Fuggetta,et al.  Software process , 2014, FOSE.

[53]  Claes Wohlin,et al.  An empirically based terminology and taxonomy for global software engineering , 2014, Empirical Software Engineering.

[54]  Kai Petersen,et al.  Guidelines for conducting systematic mapping studies in software engineering: An update , 2015, Inf. Softw. Technol..

[55]  Claes Wohlin,et al.  A systematic literature review on the industrial use of software process simulation , 2014, J. Syst. Softw..

[56]  Kang G. Shin,et al.  On fault resilience of OpenStack , 2013, SoCC.

[57]  Oliver Kopp,et al.  Streamlining DevOps automation for Cloud applications using TOSCA as standardized metamodel , 2016, Future Gener. Comput. Syst..

[58]  José Miguel Montañana,et al.  Assessment and Intervention with Wii Fit in the Elderly , 2015, KES-AMSTA.

[59]  Anne Connell,et al.  Modern DevOps: Optimizing software development through effective system interactions , 2014, 2014 IEEE International Professional Communication Conference (IPCC).

[60]  Kai Petersen,et al.  Mapping software testing practice with software testing research — SERP-test taxonomy , 2015, 2015 IEEE Eighth International Conference on Software Testing, Verification and Validation Workshops (ICSTW).

[61]  Kai Petersen,et al.  Evaluating strategies for study selection in systematic literature studies , 2014, ESEM '14.

[62]  Dror G. Feitelson,et al.  Development and Deployment at Facebook , 2013, IEEE Internet Computing.

[63]  Frank Leymann,et al.  DevOpSlang - Bridging the Gap between Development and Operations , 2014, ESOCC.

[64]  Oliver Kopp,et al.  Unified Invocation of Scripts and Services for Provisioning, Deployment, and Management of Cloud Applications Based on TOSCA , 2014, CLOSER.