Randomized active atomicity violation detection in concurrent programs

Atomicity is an important specification that enables programmers to understand atomic blocks of code in a multi-threaded program as if they are sequential. This significantly simplifies the programmer's job to reason about correctness. Several modern multithreaded programming languages provide no built-in support to ensure atomicity; instead they rely on the fact that programmers would use locks properly in order to guarantee that atomic code blocks are indeed atomic. However, improper use of locks can sometimes fail to ensure atomicity. Therefore, we need tools that can check atomicity properties of lock-based code automatically. We propose a randomized dynamic analysis technique to detect a special, but important, class of atomicity violations that are often found in real-world programs. Specifically, our technique modifies the existing Java thread scheduler behavior to create atomicity violations with high probability. Our approach has several advantages over existing dynamic analysis tools. First, we can create a real atomicity violation and see if an exception can be thrown. Second, we can replay an atomicity violating execution by simply using the same seed for random number generation---we do not need to record the execution. Third, we give no false warnings unlike existing dynamic atomicity checking techniques. We have implemented the technique in a prototype tool for Java and have experimented on a number of large multi-threaded Java programs and libraries. We report a number of previously known and unknown bugs and atomicity violations in these Java programs.

[1]  Richard J. Lipton,et al.  Reduction: a method of proving properties of parallel programs , 1975, CACM.

[2]  Gerard J. Holzmann,et al.  The Model Checker SPIN , 1997, IEEE Trans. Software Eng..

[3]  David E. Culler,et al.  The nesC language: A holistic approach to networked embedded systems , 2003, PLDI '03.

[4]  Scott D. Stoller,et al.  Accurate and efficient runtime detection of atomicity errors in concurrent programs , 2006, PPoPP '06.

[5]  Alexander Aiken,et al.  Conditional must not aliasing for static race detection , 2007, POPL '07.

[6]  Satish Narayanasamy,et al.  Automatically classifying benign and harmful data races using replay analysis , 2007, PLDI '07.

[7]  Richard H. Carver,et al.  A General Model for Reachability Testing of Concurrent Programs , 2004, ICFEM.

[8]  Koen De Bosschere,et al.  TRaDe: A Topological Approach to On-the-Fly Race Detection in Java Programs , 2001, Java Virtual Machine Research and Technology Symposium.

[9]  Koushik Sen,et al.  Race directed random testing of concurrent programs , 2008, PLDI '08.

[10]  Jeffrey S. Foster,et al.  LOCKSMITH: context-sensitive correlation analysis for race detection , 2006, PLDI '06.

[11]  Gerard J. Holzmann,et al.  The SPIN Model Checker , 2003 .

[12]  Jong-Deok Choi,et al.  Techniques for debugging parallel programs with flowback analysis , 1991, TOPL.

[13]  Cormac Flanagan,et al.  A type and effect system for atomicity , 2003, PLDI.

[14]  Nicholas Sterling,et al.  WARLOCK - A Static Data Race Analysis Tool , 1993, USENIX Winter.

[15]  Sebastian Burckhardt,et al.  CheckFence: checking consistency of concurrent data types on relaxed memory models , 2007, PLDI '07.

[16]  Stephen N. Freund,et al.  Checking Concise Specifications for Multithreaded Software , 2004, J. Object Technol..

[17]  Azadeh Farzan,et al.  Causal Atomicity , 2006, CAV.

[18]  George S. Avrunin,et al.  Using model checking with symbolic execution to verify parallel numerical programs , 2006, ISSTA '06.

[19]  Derek L. Bruening Systematic testing of multithreaded Java programs , 1999 .

[20]  Scott D. Stoller,et al.  Run-Time Analysis for Atomicity , 2003, Electron. Notes Theor. Comput. Sci..

[21]  Klaus Havelund,et al.  Model checking JAVA programs using JAVA PathFinder , 2000, International Journal on Software Tools for Technology Transfer.

[22]  Radu Grosu,et al.  Monte Carlo Model Checking , 2005, TACAS.

[23]  Koushik Sen,et al.  A Race-Detection and Flipping Algorithm for Automated Testing of Multi-threaded Programs , 2006, Haifa Verification Conference.

[24]  Martin C. Rinard,et al.  A parameterized type system for race-free Java programs , 2001, OOPSLA '01.

[25]  Cormac Flanagan,et al.  Verifying Commit-Atomicity Using Model-Checking , 2004, SPIN.

[26]  Michael Burrows,et al.  Eraser: a dynamic data race detector for multithreaded programs , 1997, TOCS.

[27]  Thomas R. Gross,et al.  Object race detection , 2001, OOPSLA '01.

[28]  Robert H. B. Netzer,et al.  Detecting data races on weak memory systems , 1991, [1991] Proceedings. The 18th Annual International Symposium on Computer Architecture.

[29]  Stephen N. Freund,et al.  Atomizer: A dynamic atomicity checker for multithreaded programs , 2008, Sci. Comput. Program..

[30]  Matthew B. Dwyer,et al.  Verifying Atomicity Specifications for Concurrent Object-Oriented Software Using Model-Checking , 2004, VMCAI.

[31]  Stephan Merz,et al.  Model Checking , 2000 .

[32]  Stephen N. Freund,et al.  Detecting race conditions in large programs , 2001, PASTE '01.

[33]  Koushik Sen,et al.  Effective random testing of concurrent programs , 2007, ASE.

[34]  Klaus Havelund,et al.  Model checking programs , 2000, Proceedings ASE 2000. Fifteenth IEEE International Conference on Automated Software Engineering.

[35]  Laurie Hendren,et al.  Soot---a java optimization framework , 1999 .

[36]  John M. Mellor-Crummey,et al.  On-the-fly detection of data races for programs with nested fork-join parallelism , 1991, Proceedings of the 1991 ACM/IEEE Conference on Supercomputing (Supercomputing '91).

[37]  Matthew B. Dwyer,et al.  Exploiting Object Escape and Locking Information in Partial-Order Reductions for Concurrent Object-Oriented Programs , 2004, Formal Methods Syst. Des..

[38]  Koen De Bosschere,et al.  RecPlay: a fully integrated practical record/replay system , 1999, TOCS.

[39]  Dawson R. Engler,et al.  RacerX: effective, static detection of race conditions and deadlocks , 2003, SOSP '03.

[40]  David Gay,et al.  Barrier inference , 1998, POPL '98.

[41]  Madan Musuvathi,et al.  Iterative context bounding for systematic testing of multithreaded programs , 2007, PLDI '07.

[42]  Eitan Farchi,et al.  Multithreaded Java program test generation , 2002, IBM Syst. J..

[43]  David F. Bacon,et al.  Guava: a dialect of Java without data races , 2000, OOPSLA '00.

[44]  Stephen N. Freund,et al.  Type-based race detection for Java , 2000, PLDI '00.

[45]  Dinghao Wu,et al.  KISS: keep it simple and sequential , 2004, PLDI '04.

[46]  Alexander Aiken,et al.  Effective static race detection for Java , 2006, PLDI '06.

[47]  Hiroyasu Nishiyama,et al.  Detecting Data Races Using Dynamic Escape Analysis Based on Read Barrier , 2004, Virtual Machine Research and Technology Symposium.

[48]  Jong-Deok Choi,et al.  Efficient and precise datarace detection for multithreaded object-oriented programs , 2002, PLDI '02.

[49]  Rahul Agarwal,et al.  Optimized run-time race detection and atomicity checking using partial discovered types , 2005, ASE.

[50]  Patrice Godefroid,et al.  Model checking for programming languages using VeriSoft , 1997, POPL '97.

[51]  Stephen N. Freund,et al.  Velodrome: a sound and complete dynamic atomicity checker for multithreaded programs , 2008, PLDI '08.

[52]  Scott D. Stoller,et al.  Testing Concurrent Java Programs using Randomized Scheduling , 2002, RV@FLoC.

[53]  Edith Schonberg,et al.  Detecting access anomalies in programs with critical sections , 1991, PADD '91.

[54]  Thomas A. Henzinger,et al.  Race checking by context inference , 2004, PLDI '04.

[55]  Matthew B. Dwyer,et al.  Parallel Randomized State-Space Search , 2007, 29th International Conference on Software Engineering (ICSE'07).