Reproducing concurrency failures from crash stacks

Reproducing field failures is the first essential step for understanding, localizing and removing faults. Reproducing concurrency field failures is hard due to the need of synthesizing a test code jointly with a thread interleaving that induce the failure in the presence of limited information from the field. Current techniques for reproducing concurrency failures focus on identifying failure-inducing interleavings, leaving largely open the problem of synthesizing the test code that manifests such interleavings. In this paper, we present ConCrash, a technique to automatically generate test codes that reproduce concurrency failures that violate thread-safety from crash stacks, which commonly summarize the conditions of field failures. ConCrash efficiently explores the huge space of possible test codes to identify a failure-inducing one by using a suitable set of search pruning strategies. Combined with existing techniques for exploring interleavings, ConCrash automatically reproduces a given concurrency failure that violates the thread-safety of a class by identifying both a failure-inducing test code and corresponding interleaving. In the paper, we define the ConCrash approach, present a prototype implementation of ConCrash, and discuss the experimental results that we obtained on a known set of ten field failures that witness the effectiveness of the approach.

[1]  Thomas R. Gross,et al.  FULLY AUTOMATIC AND PRECISE DETECTION OF THREAD SAFETY VIOLATIONS PLDI 2012 , 2013 .

[2]  Jian Lu,et al.  CARE: cache guided deterministic replay for concurrent Java programs , 2014, ICSE.

[3]  Jürgen Dingel,et al.  Automating comprehensive safety analysis of concurrent programs using verisoft and TXL , 2004, SIGSOFT '04/FSE-12.

[4]  Thomas Zimmermann,et al.  What Makes a Good Bug Report? , 2008, IEEE Transactions on Software Engineering.

[5]  Gordon Fraser,et al.  Generating Unit Tests for Concurrent Classes , 2013, 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation.

[6]  Jeff Huang,et al.  CLAP: recording local executions to reproduce concurrency failures , 2013, PLDI.

[7]  Ion Stoica,et al.  ODR: output-deterministic replay for multicore debugging , 2009, SOSP '09.

[8]  Charles Zhang,et al.  Stride: Search-based deterministic replay in polynomial time via bounded linkage , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[9]  Rongxin Wu Diagnose crashing faults on production software , 2014, FSE 2014.

[10]  Francesco Sorrentino,et al.  Predicting null-pointer dereferences in concurrent programs , 2012, SIGSOFT FSE.

[11]  Stephen N. Freund,et al.  Atomizer: a dynamic atomicity checker for multithreaded programs , 2004, 18th International Parallel and Distributed Processing Symposium, 2004. Proceedings..

[12]  Yuanyuan Zhou,et al.  Bug characteristics in open source software , 2013, Empirical Software Engineering.

[13]  Chao Wang,et al.  Coverage guided systematic concurrency testing , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[14]  Michael D. Ernst,et al.  Feedback-Directed Random Test Generation , 2007, 29th International Conference on Software Engineering (ICSE'07).

[15]  Sarfraz Khurshid,et al.  Test input generation with java PathFinder , 2004, ISSTA '04.

[16]  W. K. Chan,et al.  MagicFuzzer: Scalable deadlock detection for large-scale applications , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[17]  Andreas Zeller,et al.  Reconstructing Core Dumps , 2013, 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation.

[18]  Grigore Rosu,et al.  GPredict: Generic Predictive Concurrency Analysis , 2015, 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering.

[19]  Gail E. Kaiser,et al.  Chronicler: Lightweight recording to reproduce field failures , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[20]  Shing-Chi Cheung,et al.  RECONTEST: Effective Regression Testing of Concurrent Programs , 2015, 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering.

[21]  Rongxin Wu,et al.  Casper: an efficient approach to call trace collection , 2016, POPL.

[22]  Stephen N. Freund,et al.  Velodrome: a sound and complete dynamic atomicity checker for multithreaded programs , 2008, PLDI '08.

[23]  Manu Sridharan,et al.  PSE: explaining program failures via postmortem static analysis , 2004, SIGSOFT '04/FSE-12.

[24]  Yuanyuan Zhou,et al.  Learning from mistakes: a comprehensive study on real world concurrency bug characteristics , 2008, ASPLOS.

[25]  Brandon Lucia,et al.  Production-guided concurrency debugging , 2016, PPoPP.

[26]  Jeff Huang,et al.  LEAP: lightweight deterministic multi-processor replay of concurrent java programs , 2010, SIGSOFT FSE.

[27]  Suresh Jagannathan,et al.  Synthesizing racy tests , 2015, PLDI.

[28]  Shing-Chi Cheung,et al.  Coverage-Driven Test Code Generation for Concurrent Classes , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE).

[29]  Shan Lu,et al.  ConMem: detecting severe concurrency bugs through an effect-oriented approach , 2010, ASPLOS XV.

[30]  Maurice Herlihy,et al.  Linearizability: a correctness condition for concurrent objects , 1990, TOPL.

[31]  Mauro Pezzè,et al.  A Survey of Recent Trends in Testing Concurrent Software Systems , 2018, IEEE Transactions on Software Engineering.

[32]  David Holmes,et al.  Java Concurrency in Practice , 2006 .

[33]  Satish Narayanasamy,et al.  Maple: a coverage-driven testing tool for multithreaded programs , 2012, OOPSLA '12.

[34]  Shan Lu,et al.  A study of interleaving coverage criteria , 2007, ESEC-FSE companion '07.

[35]  Manu Sridharan,et al.  Snugglebug: a powerful approach to weakest preconditions , 2009, PLDI '09.

[36]  Murali Krishna Ramanathan,et al.  Directed synthesis of failing concurrent executions , 2016, OOPSLA.

[37]  George Candea,et al.  Execution synthesis: a technique for automated software debugging , 2010, EuroSys '10.

[38]  Alessandro Orso,et al.  BugRedux: Reproducing field failures for in-house debugging , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[39]  Shan Lu,et al.  ConSeq: detecting concurrency bugs through sequential errors , 2011, ASPLOS XVI.

[40]  Ning Chen,et al.  STAR: Stack Trace Based Automatic Crash Reproduction via Symbolic Execution , 2015, IEEE Transactions on Software Engineering.

[41]  Michael Burrows,et al.  Eraser: a dynamic data race detector for multithreaded programs , 1997, TOCS.

[42]  Jens Palsberg,et al.  Sherlock: scalable deadlock detection for concurrent programs , 2014, SIGSOFT FSE.

[43]  Thomas R. Gross,et al.  Ballerina: Automatic generation and clustering of efficient random unit tests for multithreaded code , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[44]  Satish Narayanasamy,et al.  LiteRace: effective sampling for lightweight data-race detection , 2009, PLDI '09.

[45]  Angelo Gargantini,et al.  Experiments on the test case length in specification based test case generation , 2009, 2009 ICSE Workshop on Automation of Software Test.

[46]  Rongxin Wu,et al.  CrashLocator: locating crashing faults based on crash stacks , 2014, ISSTA 2014.

[47]  Yuanyuan Zhou,et al.  CTrigger: exposing atomicity violation bugs from their hiding places , 2009, ASPLOS.

[48]  Jong-Deok Choi,et al.  Hybrid dynamic data race detection , 2003, PPoPP '03.

[49]  Ben Liblit,et al.  Lightweight control-flow instrumentation and postmortem analysis in support of debugging , 2013, 2013 28th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[50]  Murali Krishna Ramanathan,et al.  Multithreaded test synthesis for deadlock detection , 2014, OOPSLA.

[51]  Michael D. Ernst,et al.  ReCrash: Making Software Failures Reproducible by Preserving Object States , 2008, ECOOP.

[52]  W. K. Chan,et al.  ConLock: a constraint-based approach to dynamic checking on deadlocks in multithreaded programs , 2014, ICSE.

[53]  Nikolaj Bjørner,et al.  Z3: An Efficient SMT Solver , 2008, TACAS.

[54]  Murali Krishna Ramanathan,et al.  Synthesizing tests for detecting atomicity violations , 2015, ESEC/SIGSOFT FSE.

[55]  Koushik Sen,et al.  Randomized active atomicity violation detection in concurrent programs , 2008, SIGSOFT '08/FSE-16.

[56]  Shan Lu,et al.  Efficient Detection of Thread Safety Violations via Coverage-Guided Generation of Concurrent Tests , 2017, 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE).

[57]  Xiangyu Zhang,et al.  Analyzing multicore dumps to facilitate concurrency bug reproduction , 2010, ASPLOS XV.