DevOps in practice: A multiple case study of five companies

Abstract Context: DevOps is considered important in the ability to frequently and reliably update a system in operational state. DevOps presumes cross-functional collaboration and automation between software development and operations. DevOps adoption and implementation in companies is non-trivial due to required changes in technical, organisational and cultural aspects. Objectives: This exploratory study presents detailed descriptions of how DevOps is implemented in practice. The context of our empirical investigation is web application and service development in small and medium sized companies. Method: A multiple-case study was conducted in five different development contexts with successful DevOps implementations since its benefits, such as quick releases and minimum deployment errors, were achieved. Data was mainly collected through interviews with 26 practitioners and observations made at the companies. Data was analysed by first coding each case individually using a set of predefined themes and thereafter perform a cross-case synthesis. Results: Our analysis yielded some of the following results: (i) software development team attaining ownership and responsibility to deploy software changes in production is crucial in DevOps. (ii) toolchain usage and support in deployment pipeline activities accelerates the delivery of software changes, bug fixes and handling of production incidents. (ii) the delivery speed to production is affected by context factors, such as manual approvals by the product owner (iii) steep learning curve for new skills is experienced by both software developers and operations staff, who also have to cope with working under pressure. Conclusion: Our findings contributes to the overall understanding of DevOps concept, practices and its perceived impacts, particularly in small and medium sized companies. We discuss two practical implications of the results.

[1]  Frank Leymann,et al.  Automated Capturing and Systematic Usage of DevOps Knowledge for Cloud Applications , 2015, 2015 IEEE International Conference on Cloud Engineering.

[2]  Florian Rosenberg,et al.  Testing Idempotence for Infrastructure as Code , 2013, Middleware.

[3]  Paul Ralph,et al.  Grounded Theory in Software Engineering Research: A Critical Review and Guidelines , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE).

[4]  Mika Mäntylä,et al.  The highways and country roads to continuous deployment , 2015, IEEE Software.

[5]  Pooyan Jamshidi,et al.  Microservices Architecture Enables DevOps: Migration to a Cloud-Native Architecture , 2016, IEEE Software.

[6]  Rory V. O'Connor,et al.  The situational factors that affect the software development process: Towards a comprehensive reference framework , 2012, Inf. Softw. Technol..

[7]  Ivan Porres,et al.  DevOps: A Definition and Perceived Adoption Impediments , 2015, XP.

[8]  Daniel Cukier DevOps patterns to scale web applications using cloud services , 2013, SPLASH '13.

[9]  Wei Liu,et al.  Integrated Solution for Timely Delivery of Customer Change Requests: A Case Study of Using DevOps Approach , 2014 .

[10]  Shane McIntosh,et al.  Modern Release Engineering in a Nutshell -- Why Researchers Should Care , 2016, 2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER).

[11]  Lianping Chen,et al.  The Journal of Systems and Software , 2022 .

[12]  R. Yin Case Study Research: Design and Methods , 1984 .

[13]  Liming Zhu,et al.  DevOps - A Software Architect's Perspective , 2015, SEI series in software engineering.

[14]  Foutse Khomh,et al.  On rapid releases and software testing: a case study and a semi-systematic literature review , 2015, Empirical Software Engineering.

[15]  Pasi Kuvaja,et al.  An Exploratory Study of DevOps: Extending the Dimensions of DevOps with Practices , 2016, ICSEA 2016.

[16]  Stephen Jones,et al.  Management challenges for DevOps adoption within UK SMEs , 2016, QUDOS@ISSTA.

[17]  Per Runeson,et al.  Guidelines for conducting and reporting case study research in software engineering , 2009, Empirical Software Engineering.

[18]  Ivan Porres,et al.  On the Impact of Mixing Responsibilities Between Devs and Ops , 2016, XP.

[19]  Harald C. Gall,et al.  The making of cloud applications: an empirical study on software development for the cloud , 2014, ESEC/SIGSOFT FSE.

[20]  Olly Gotel,et al.  Agile Software Development Meets Corporate Deployment Procedures: Stretching the Agile Envelope , 2007, XP.

[21]  Bjørnar Tessem,et al.  Problems in the interplay of development and IT operations in system development projects: A Delphi study of Norwegian IT experts , 2011, Inf. Softw. Technol..

[22]  Theo Schlossnagle Monitoring in a DevOps world , 2018, CACM.

[23]  Bram Adams,et al.  Who Needs Release and DevOps Engineers, and Why? , 2016, 2016 IEEE/ACM International Workshop on Continuous Software Evolution and Delivery (CSED).

[24]  Eric S. K. Yu,et al.  Modeling DevOps Deployment Choices Using Process Architecture Design Dimensions , 2015, PoEM.

[25]  Claes Wohlin,et al.  Experimentation in Software Engineering , 2012, Springer Berlin Heidelberg.

[26]  Beverley G. Hope,et al.  Author's Personal Copy the Journal of Systems and Software Coordination in Co-located Agile Software Development Projects , 2022 .

[27]  Mike Loukides,et al.  What is DevOps , 2012 .

[28]  Bjørnar Tessem,et al.  Cooperation between developers and operations in software engineering projects , 2008, CHASE '08.

[29]  Tobias Schneider Achieving Cloud Scalability with Microservices and DevOps in the Connected Car Domain , 2016, Software Engineering.

[30]  Matt Callanan,et al.  DevOps: Making It Easy to Do the Right Thing , 2016, IEEE Software.

[31]  James R. Hamilton,et al.  On Designing and Deploying Internet-Scale Services , 2007, LISA.

[32]  Masud Fazal-Baqaie,et al.  Towards DevOps in Multi-provider Projects , 2017, CSE@SE.

[33]  Jan Bosch,et al.  Towards DevOps in the Embedded Systems Domain: Why is It So Hard? , 2016, 2016 49th Hawaii International Conference on System Sciences (HICSS).

[34]  Maya Daneva,et al.  A qualitative study of DevOps usage in practice , 2017, J. Softw. Evol. Process..

[35]  Liming Zhu,et al.  Achieving Reliable High-Frequency Releases in Cloud Environments , 2015, IEEE Software.

[36]  Liming Zhu,et al.  Eliciting operations requirements for applications , 2013, 2013 1st International Workshop on Release Engineering (RELENG).

[37]  Patrick Debois,et al.  Agile Infrastructure and Operations: How Infra-gile are You? , 2008, Agile 2008 Conference.

[38]  Pasi Kuvaja,et al.  Continuous deployment of software intensive products and services: A systematic mapping study , 2017, J. Syst. Softw..

[39]  Jez Humble,et al.  Continuous Delivery: Reliable Software Releases Through Build, Test, and Deployment Automation , 2010 .

[40]  Jan Bosch,et al.  Continuous practices and devops: beyond the buzz, what does it all mean? , 2017, 2017 43rd Euromicro Conference on Software Engineering and Advanced Applications (SEAA).

[41]  Robert Karl,et al.  Holistic configuration management at Facebook , 2015, SOSP.

[42]  Jennifer Davis,et al.  Effective DevOps: Building a Culture of Collaboration, Affinity, and Tooling at Scale , 2016 .

[43]  Laurie Williams,et al.  Software Security in DevOps: Synthesizing Practitioners’ Perceptions and Practices , 2016, 2016 IEEE/ACM International Workshop on Continuous Software Evolution and Delivery (CSED).

[44]  Hans van Vliet,et al.  When agile meets the enterprise , 2013, Inf. Softw. Technol..

[45]  Frank Elberzhager,et al.  From Agile Development to DevOps: Going Towards Faster Releases at High Quality - Experiences from an Industrial Context , 2017, SWQD.

[46]  Maya Daneva,et al.  A Mapping Study on Cooperation between Information System Development and Operations , 2014, PROFES.

[47]  Liming Zhu,et al.  The Intersection of Continuous Deployment and Architecting Process: Practitioners' Perspectives , 2016, ESEM.

[48]  Ramtin Jabbari,et al.  What is DevOps?: A Systematic Mapping Study on Definitions and Practices , 2016, XP Workshops.

[49]  Pekka Abrahamsson,et al.  New directions on agile methods: a comparative analysis , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..

[50]  Guilherme Horta Travassos,et al.  Characterizing DevOps by Hearing Multiple Voices , 2016, SBES '16.

[51]  Liming Zhu,et al.  Securing a Deployment Pipeline , 2015, 2015 IEEE/ACM 3rd International Workshop on Release Engineering.

[52]  Richard Berntsson-Svensson,et al.  On the journey to continuous deployment: Technical and social challenges along the way , 2015, Inf. Softw. Technol..

[53]  V. Braun,et al.  Using thematic analysis in psychology , 2006 .

[54]  Andy Zaidman,et al.  Continuous Delivery Practices in a Large Financial Organization , 2016, ICSME.

[55]  Terhi Kilamo,et al.  Improving the delivery cycle: A multiple-case study of the toolchains in Finnish software intensive enterprises , 2016, Inf. Softw. Technol..