Modeling and measuring attributes influencing DevOps implementation in an enterprise using structural equation modeling

Abstract Context DevOps refer to set of principles that advocate a tight integration between development and operation to achieve higher quality with faster turnaround. It is paramount to assess and measure the DevOps attributes in an enterprise. The literature provides references to these attributes but the detail assessment of these attributes and determination of the maturity of DevOps implementation is still a challenge. Objective This paper provides important insights for practitioners to assess and measure the DevOps attributes using statistical analysis and Two-way assessment. The proposed framework facilitates the detailed assessment of eighteen attributes to identify key independent attributes and measure them to determine the maturity of DevOps implementation in an enterprise. Method The relationship between eighteen attributes was examined; a structural model was established using Exploratory and Confirmatory Factor Analysis, the model was validated using Structural Equation Modelling. Key independent attributes were identified which influences other attributes and overall DevOps implementation. Using Two-way assessment, key independent attributes were measured and the maturity of the DevOps implementation was determined in an enterprise. Results Using Exploratory and Confirmatory Factor Analysis, 18 attributes were categorized under 4 latent variables namely Automation, Source Control, Cohesive Teams and Continuous Delivery. Using Structural Equation Modelling, 10 key independent attributes were determined, that influenced other attributes and overall DevOps implementation. Two-way assessment was applied to measure the key independent attributes and it was found that 4 of these attributes were performing below threshold level. Corrective actions were taken by the management team, and the revised measurement of these attributes demonstrated 40% improvement in the maturity level of DevOps implementation. Conclusion The proposed framework contributes significantly to the field of DevOps by enabling practitioners to conduct the detailed assessment and measurement of DevOps attributes to determine the maturity of DevOps implementation to achieve higher quality.

[1]  Mike Loukides,et al.  What is DevOps , 2012 .

[2]  F. Floyd,et al.  Factor analysis in the development and refinement of clinical assessment instruments. , 1995 .

[3]  Pasi Kuvaja,et al.  Dimensions of DevOps , 2015, XP.

[4]  Tao Zhao,et al.  Analysis of interactions among the barriers to energy saving in China , 2008 .

[5]  Liming Zhu,et al.  Tradeoff and Sensitivity Analysis in Software Architecture Evaluation Using Analytic Hierarchy Process , 2005, Software Quality Journal.

[6]  Florian Rosenberg,et al.  Testing Idempotence for Infrastructure as Code , 2013, Middleware.

[7]  Chin-Yun Hsieh,et al.  Patterns for Continuous Integration Builds in Cross-Platform Agile Software Development , 2015, J. Inf. Sci. Eng..

[8]  Amol Patwardhan,et al.  Embracing Agile methodology during DevOps Developer Internship Program , 2016, ArXiv.

[9]  Victor R. Basili,et al.  Iterative and incremental developments. a brief history , 2003, Computer.

[10]  Mark Stillwell,et al.  A DevOps approach to integration of software components in an EU research project , 2015, QUDOS@SIGSOFT FSE.

[11]  Elisabetta Di Nitto,et al.  Model-driven continuous deployment for quality DevOps , 2016, QUDOS@ISSTA.

[12]  Antonio Puliafito,et al.  CloudWave: Where adaptive cloud management meets DevOps , 2014, 2014 IEEE Symposium on Computers and Communications (ISCC).

[13]  Shigeru Hosono.  A DevOps framework to shorten delivery time for cloud applications , 2012, Int. J. Comput. Sci. Eng..

[14]  H. Adèr,et al.  Structural equation modelling , 1999 .

[15]  Evangelos Triantaphyllou,et al.  USING THE ANALYTIC HIERARCHY PROCESS FOR DECISION MAKING IN ENGINEERING APPLICATIONS: SOME CHALLENGES , 1995 .

[16]  B. Thompson.  Exploratory and Confirmatory Factor Analysis: Understanding Concepts and Applications , 2004 .

[17]  Barbara M. Byrne,et al.  Structural equation modeling with EQS : basic concepts, applications, and programming , 2000 .

[18]  Deepak Kumar,et al.  Assessment of quality factors in enterprise application integration , 2015, 2015 4th International Conference on Reliability, Infocom Technologies and Optimization (ICRITO) (Trends and Future Directions).

[19]  Daniel Cukier.  DevOps patterns to scale web applications using cloud services , 2013, SPLASH '13.

[20]  P. K. Kapur,et al.  Critical success factor utility based tool for ERP health assessment: a general framework , 2014, Int. J. Syst. Assur. Eng. Manag..

[21]  P. Bentler,et al.  Cutoff criteria for fit indexes in covariance structure analysis : Conventional criteria versus new alternatives , 1999 .

[22]  Matthew Sacks,et al.  DevOps Principles for Successful Web Sites , 2012 .

[23]  Marc Reichenbach,et al.  Continuous Integration and Automation for Devops , 2013 .

[24]  Marc J. Dupuis,et al.  A grounded theory analysis of modern web applications: knowledge, skills, and abilities for DevOps , 2013, RIIT '13.

[25]  N. Schmitt.  Uses and abuses of coefficient alpha. , 1996 .

[26]  Angappa Gunasekaran,et al.  Sustainability development in high-tech manufacturing firms in Hong Kong: Motivators and readiness , 2012 .

[27]  Wilhelm Hasselbring,et al.  Including Performance Benchmarks into Continuous Integration to Enable DevOps , 2015, SOEN.

[28]  Dhrubes Biswas,et al.  An Approach to Identify Failure Factors of Enterprise Application Implementation in Indian Micro Enterprises , 2013 .

[29]  Joseph Sarkis,et al.  Evaluating Environmentally Conscious Manufacturing Barriers With Interpretive Structural Modeling , 2006 .

[30]  Christian Bird,et al.  The effect of branching strategies on software quality , 2012, Proceedings of the 2012 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement.

[31]  Chin Wen Cheong,et al.  Design and Development of Decision Making System Using Fuzzy Analytic Hierarchy Process , 2008 .

[32]  Kuldip Singh Sangwan,et al.  Development of a structural model of environmentally conscious manufacturing drivers , 2014 .

[33]  William E. Perry,et al.  Effective methods for software testing , 1995 .

[34]  Giuliano Casale,et al.  Towards a DevOps Approach for Software Quality Engineering , 2015, WOSP '15.

[35]  Thomas L. Saaty,et al.  DECISION MAKING WITH THE ANALYTIC HIERARCHY PROCESS , 2008 .

[36]  Bo K. Wong,et al.  Software selection: a case study of the application of the analytical hierarchical process to the selection of a multimedia authoring system , 1999, Inf. Manag..

[37]  Surendra Naidu Mullaguru.  Changing Scenario of Testing Paradigms using DevOps – A Comparative Study with Classical Models , 2015 .

[38]  Horst Lichter,et al.  Towards Definitions for Release Engineering and DevOps , 2015, 2015 IEEE/ACM 3rd International Workshop on Release Engineering.

[39]  Deepak Kumar,et al.  Modelling and measuring code smells in enterprise applications using TISM and two-way assessment , 2016, Int. J. Syst. Assur. Eng. Manag..

[40]  T. Brown,et al.  Exploratory Factor Analysis: A Five-Step Guide for Novices , 2010 .

[41]  Anne Connell,et al.  Modern DevOps: Optimizing software development through effective system interactions , 2014, 2014 IEEE International Professional Communication Conference (IPCC).

[42]  Kuldip Singh Sangwan,et al.  Development of a model of barriers to environmentally conscious manufacturing implementation , 2014 .

[43]  Weiyi Shang.  Bridging the divide between software developers and operators using logs , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[44]  J. Hair.  Multivariate data analysis , 1972 .

[45]  Maya Daneva,et al.  A Mapping Study on Cooperation between Information System Development and Operations , 2014, PROFES.

[46]  Roger G. Schroeder,et al.  A FRAMEWORK FOR QUALITY MANAGEMENT RESEARCH AND AN ASSOCIATED MEASUREMENT INSTRUMENT , 1994 .