Development and evaluation of a lightweight root cause analysis method (ARCA method) - Field studies at four software companies

Abstract in Undetermined Context The key for effective problem prevention is detecting the causes of a problem that has occurred. Root cause analysis (RCA) is a structured investigation of the problem to identify which underlying causes need to be fixed. The RCA method consists of three steps: target problem detection, root cause detection, and corrective action innovation. Its results can help with process improvement. Objective This paper presents a lightweight RCA method, named the ARCA method, and its empirical evaluation. In the ARCA method, the target problem detection is based on a focus group meeting. This is in contrast to prior RCA methods, where the target problem detection is based on problem sampling, requiring heavy startup investments. Method The ARCA method was created with the framework of design science. We evaluated it through field studies at four medium-sized software companies using interviews and query forms to collect feedback from the case attendees. A total of five key representatives of the companies were interviewed, and 30 case participants answered the query forms. The output of the ARCA method was also evaluated by the case attendees, i.e., a total 757 target problem causes and 124 related corrective actions. Results The case attendees considered the ARCA method useful and easy to use, which indicates that it is beneficial for process improvement and problem prevention. In each case, 24–77 target problem root causes were processed and 13–40 corrective actions were developed. The effort of applying the method was 89 man-hours, on average. Conclusion The ARCA method required an acceptable level of effort and resulted in numerous high-quality corrective actions. In contrast to the current company practices, the method is an efficient method to detect new process improvement opportunities and develop new process improvement ideas. Additionally, it is easy to use.

[1]  B. Williams,et al.  Operations management. , 2001, Optometry.

[2]  James J. Rooney,et al.  Collecting Data for Root Cause Analysis , 2003 .

[3]  Stefan Wagner,et al.  Defect classification and defect types revisited , 2008, DEFECTS '08.

[4]  Mika Mäntylä,et al.  Who tested my software? Testing as an organizationally cross-cutting activity , 2011, Software Quality Journal.

[5]  Robert L. Glass,et al.  Project retrospectives, and why they never happen , 2002, IEEE Software.

[6]  Inderpal S. Bhandari,et al.  A Case Study of Software Process Improvement During Development , 1993, IEEE Trans. Software Eng..

[7]  Dewayne E. Perry,et al.  A case study in root cause defect analysis , 2000, Proceedings of the 2000 International Conference on Software Engineering. ICSE 2000 the New Millennium.

[8]  P. Jalote,et al.  Using defect analysis feedback for improving quality and productivity in iterative software development , 2005, 2005 International Conference on Information and Communication Technology.

[9]  Guilherme Horta Travassos,et al.  Towards a Defect Prevention Based Process Improvement Approach , 2008, 2008 34th Euromicro Conference Software Engineering and Advanced Applications.

[10]  D.N. Card.  Defect-causal analysis drives down error rates , 1993, IEEE Software.

[11]  Per Runeson,et al.  Guidelines for conducting and reporting case study research in software engineering , 2009, Empirical Software Engineering.

[12]  Robert J. Latino,et al.  Root Cause Analysis: Improving Performance for Bottom Line Results , 1999 .

[13]  James J. Rooney,et al.  Root cause analysis for beginners , 2004 .

[14]  Reidar Conradi,et al.  A case study comparing defect profiles of a reused framework and of applications reusing it , 2009, Empirical Software Engineering.

[15]  Alan R. Hevner,et al.  Design Science in Information Systems Research , 2004, MIS Q..

[16]  Geoffrey Ho,et al.  Using Root Cause Data Analysis for Requirements and Knowledge Elicitation , 2007, HCI.

[17]  Jeffrey C. Carver,et al.  A systematic literature review to identify and classify software requirement errors , 2009, Inf. Softw. Technol..

[18]  Max Ammerman.  The Root Cause Analysis Handbook: A Simplified Approach to Identifying, Correcting, and Reporting Workplace Errors , 1998 .

[19]  Hongli Zhang,et al.  Intrusion detection alarms reduction using root cause analysis and clustering , 2009, Comput. Commun..

[20]  Erez Zadok,et al.  DARC: dynamic analysis of root causes of latency distributions , 2008, SIGMETRICS '08.

[21]  Matti Siekkinen,et al.  A root cause analysis toolkit for TCP , 2008, Comput. Networks.

[22]  Salvatore T. March,et al.  Design and natural science research on information technology , 1995, Decis. Support Syst..

[23]  W. Foddy.  Constructing Questions for Interviews and Questionnaires: Reducing question threat , 1993 .

[24]  Svenja C. Sommer,et al.  The Effects of Problem Structure and Team Diversity on Brainstorming Effectiveness , 2009, Manag. Sci..

[25]  Ilene Burnstein,et al.  Practical Software Testing , 2003, Springer Professional Computing.

[26]  Robert B. Grady,et al.  Software Failure Analysis for High-Return Process Improvement Decisions , 1996 .

[27]  Tor Stålhane.  Root Cause Analysis and Gap Analysis - A Tale of Two Methods , 2004, EuroSPI.

[28]  Inderpal S. Bhandari,et al.  Orthogonal Defect Classification - A Concept for In-Process Measurements , 1992, IEEE Trans. Software Eng..

[29]  Janice Singer,et al.  Studying Software Engineers: Data Collection Techniques for Software Field Studies , 2005, Empirical Software Engineering.

[30]  Bjørn Andersen,et al.  Root Cause Analysis: Simplified Tools and Techniques , 1999 .

[31]  David N. Card,et al.  Learning from Our Mistakes with Defect Causal Analysis , 1999, IEEE Softw..

[32]  Robert G. Mays.  Applications of Defect Prevention in Software Development , 1990, IEEE J. Sel. Areas Commun..

[33]  Alf Inge Wang,et al.  Improving the effectiveness of root cause analysis in post mortem analysis: A controlled experiment , 2009, Inf. Softw. Technol..