Continuous Integration and Continuous Delivery Pipeline Automation for Agile Software Project Management

Agile practices with Continuous Integration and Continuous Delivery (CICD) pipeline approach has increased the efficiency of projects. In agile, new features are introduced to the system in each sprint delivery, and although it may be well developed, the delivery failures are possible due to performance issues. By considering delivery timeline, moving for system scaling is common solution in such situations. But, how much system should be scaled? System scale requires current system benchmark status and expected system status. Benchmarking the production is a critical task, as it interrupts the live system. The new version should go through a load test to measure expected system status. The traditional load test methods are unable to identify production performance behavior due to simulated traffic patterns are highly deviated from production. To overcome those issues, this approach has extended CICD pipeline to have three automation phases named benchmark, load test and scaling. It minimizes the system interruption by using test bench approach when system benchmarking and it uses the production traffic for load testing which gives more accurate results. Once benchmark and load test phases are completed, system scaling can be evaluated. Initially, the pipeline was developed using Jenkins CI server, Git repository and Nexus repository with Ansible automation. Then GoReplay is used for traffic duplication from production to test bench environment. Nagios monitoring is used to analyze the system behavior in each phase and the result of test bench has proven that scaling is capable to handle the same load while changing the application software, but it doesn't optimize response time of application at significant level and it helps to reduce the risk of application deployment by integrating this three phase approach as CICD automation extended feature. Thereby the research provides effective way to manage Agile based CICD projects.

[1]  Tony Gorschek,et al.  Social networks in software process improvement , 2010 .

[2]  Dror G. Feitelson,et al.  Development and Deployment at Facebook , 2013, IEEE Internet Computing.

[3]  Liam O'Brien,et al.  On a Catalogue of Metrics for Evaluating Commercial Cloud Services , 2012, 2012 ACM/IEEE 13th International Conference on Grid Computing.

[4]  Timo Lehtonen,et al.  Defining metrics for continuous delivery and deployment pipeline , 2015, SPLST.

[5]  Jez Humble,et al.  Continuous Delivery: Reliable Software Releases Through Build, Test, and Deployment Automation , 2010 .

[6]  Sushant G. Gaikwad,et al.  Pipeline Orchestration for Test Automation using Extended Buildbot Architecture , 2015 .

[7]  Liming Zhu,et al.  Continuous Integration, Delivery and Deployment: A Systematic Review on Approaches, Tools, Challenges and Practices , 2017, IEEE Access.

[8]  Vincent Hardion,et al.  Configuration Management of the Control System , 2013 .

[9]  Laurie A. Williams,et al.  Continuous Deployment at Facebook and OANDA , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering Companion (ICSE-C).

[10]  Rajiv Ranjan,et al.  An overview of the commercial cloud monitoring tools: research dimensions, design issues, and state-of-the-art , 2013, Computing.

[11]  Schahram Dustdar,et al.  Microservices: Migration of a Mission Critical System , 2017, IEEE Transactions on Services Computing.

[12]  Eric Helms,et al.  Synthesizing Continuous Deployment Practices Used in Software Development , 2015, 2015 Agile Conference.

[13]  Stephan Krusche,et al.  Introduction of continuous delivery in multi-customer project courses , 2014, ICSE Companion.

[14]  Duen Horng Chau,et al.  Building a research data science platform from industrial machines , 2016, 2016 IEEE International Conference on Big Data (Big Data).

[15]  Jan Bosch,et al.  Climbing the "Stairway to Heaven" -- A Mulitiple-Case Study Exploring Barriers in the Transition from Agile Development towards Continuous Deployment of Software , 2012, 2012 38th Euromicro Conference on Software Engineering and Advanced Applications.

[16]  Lianping Chen,et al.  Continuous Delivery: Huge Benefits, but Challenges Too , 2015, IEEE Software.