2ndStrike: toward manifesting hidden concurrency typestate bugs

Concurrency bugs are becoming increasingly prevalent in the multi-core era. Recently, much research has focused on data races and atomicity violation bugs, which are related to low-level memory accesses. However, a large number of concurrency typestate bugs such as "invalid reads to a closed file from a different thread" are under-studied. These concurrency typestate bugs are important yet challenging to study since they are mostly relevant to high-level program semantics. This paper presents 2ndStrike, a method to manifest hidden concurrency typestate bugs in software testing. Given a state machine describing correct program behavior on certain object typestates, 2ndStrike profiles runtime events related to the typestates and thread synchronization. Based on the profiling results, 2ndStrike then identifies bug candidates, each of which is a pair of runtime events that would cause typestate violation if the event order is reversed. Finally, 2ndStrike re-executes the program with controlled thread interleaving to manifest bug candidates. We have implemented a prototype of 2ndStrike on Linux and have illustrated our idea using three types of concurrency typestate bugs, including invalid file operation, invalid pointer dereference, and invalid lock operation. We have evaluated 2ndStrike with six real world bugs (including one previously unknown bug) from three open-source server and desktop programs (i.e., MySQL, Mozilla, pbzip2). Our experimental results show that 2ndStrike can effectively and efficiently manifest all six software bugs, most of which are difficult or impossible to manifest using stress testing or active testing techniques that are based on data race/atomicity violation. Additionally, 2ndStrike reports no false positives, provides detailed bug reports for each manifested bug, and can consistently reproduce the bug after manifesting it once.

[1]  Mark Weiser,et al.  Programmers use slices when debugging , 1982, CACM.

[2]  Robert E. Strom,et al.  Typestate: A programming language concept for enhancing software reliability , 1986, IEEE Transactions on Software Engineering.

[3]  Edith Schonberg,et al.  An empirical comparison of monitoring algorithms for access anomaly detection , 2011, PPOPP '90.

[4]  Barton P. Miller,et al.  Improving the accuracy of data race detection , 1991, PPOPP '91.

[5]  Eugene H. Spafford,et al.  An execution-backtracking approach to debugging , 1991, IEEE Software.

[6]  Brian A. Malloy,et al.  Data flow testing of parallelized code , 1992, Proceedings Conference on Software Maintenance 1992.

[7]  Richard N. Taylor,et al.  Structural Testing of Concurrent Programs , 1992, IEEE Trans. Software Eng..

[8]  Nicholas Sterling,et al.  WARLOCK - A Static Data Race Analysis Tool , 1993, USENIX Winter.

[9]  Kuo-Chung Tai,et al.  An incremental approach to structural testing of concurrent software , 1996, ISSTA '96.

[10]  Peter J. Keleher,et al.  Online data-race detection via coherency guarantees , 1996, OSDI '96.

[11]  Michael Burrows,et al.  Eraser: a dynamic data race detector for multi-threaded programs , 1997, TOCS.

[12]  K. Rustan M. Leino,et al.  Extended static checking , 1998, PROCOMET.

[13]  Lori L. Pollock,et al.  All-du-path coverage for parallel programs , 1998, ISSTA '98.

[14]  Andreas Zeller,et al.  Yesterday, my program worked. Today, it does not. Why? , 1999, ESEC/FSE-7.

[15]  Andreas Zeller,et al.  Finding Failure Causes through Automated Testing , 2000, AADEBUG.

[16]  Jong-Deok Choi,et al.  Efficient and precise datarace detection for multithreaded object-oriented programs , 2002, PLDI '02.

[17]  Alexander Aiken,et al.  Flow-sensitive type qualifiers , 2002, PLDI '02.

[18]  Samuel T. King,et al.  ReVirt: enabling intrusion analysis through virtual-machine logging and replay , 2002, OPSR.

[19]  Jong-Deok Choi,et al.  Isolating failure-inducing thread schedules , 2002, ISSTA '02.

[20]  Sorin Lerner,et al.  ESP: path-sensitive program verification in polynomial time , 2002, PLDI '02.

[21]  Sorin Lerner.  Path-Sensitive Program Veri cation in Polynomial Time , 2002 .

[22]  James R. Larus,et al.  Mining specifications , 2002, POPL '02.

[23]  Jong-Deok Choi,et al.  Hybrid dynamic data race detection , 2003, PPoPP '03.

[24]  Dawson R. Engler,et al.  RacerX: effective, static detection of race conditions and deadlocks , 2003, SOSP '03.

[25]  Assaf Schuster,et al.  Efficient on-the-fly data race detection in multithreaded C++ programs , 2003, PPoPP '03.

[26]  Xiangyu Zhang,et al.  Precise dynamic slicing algorithms , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..

[27]  R. Bodík,et al.  A "flight data recorder" for enabling full-system multiprocessor deterministic replay , 2003, 30th Annual International Symposium on Computer Architecture, 2003. Proceedings..

[28]  Min Xu,et al.  A "flight data recorder" for enabling full-system multiprocessor deterministic replay , 2003, ISCA '03.

[29]  Srikanth Kandula,et al.  Flashback: A Lightweight Extension for Rollback and Deterministic Replay for Software Debugging , 2004, USENIX Annual Technical Conference, General Track.

[30]  Robert DeLine,et al.  Typestates for Objects , 2004, ECOOP.

[31]  Vikram S. Adve,et al.  LLVM: a compilation framework for lifelong program analysis & transformation , 2004, International Symposium on Code Generation and Optimization, 2004. CGO 2004..

[32]  Satish Narayanasamy,et al.  BugNet: continuously recording program execution for deterministic replay debugging , 2005, 32nd International Symposium on Computer Architecture (ISCA'05).

[33]  Min Xu,et al.  A serializability violation detector for shared-memory server programs , 2005, PLDI '05.

[34]  Wei Chen,et al.  RaceTrack: efficient detection of data race conditions via adaptive tracking , 2005, SOSP '05.

[35]  Yasushi Saito,et al.  Jockey: a user-space library for record-replay debugging , 2005, AADEBUG'05.

[36]  Eitan Farchi,et al.  Applications of synchronization coverage , 2005, PPoPP.

[37]  Koushik Sen,et al.  Automated Systematic Testing of Open Distributed Programs , 2006, FASE.

[38]  Paolina Centonze,et al.  Role-Based access control consistency validation , 2006, ISSTA '06.

[39]  Satish Narayanasamy,et al.  Recording shared memory dependencies using strata , 2006, ASPLOS XII.

[40]  Min Xu,et al.  A regulated transitive reduction (RTR) for longer memory race recording , 2006, ASPLOS XII.

[41]  James R. Larus,et al.  Transactional Memory , 2006, Transactional Memory.

[42]  Richard A. DeMillo,et al.  An Execution Backtracking Approach to Program Debugging , 2007 .

[43]  Grigore Rosu,et al.  Mop: an efficient and generic runtime verification framework , 2007, OOPSLA.

[44]  Oege de Moor,et al.  Making trace monitors feasible , 2007, OOPSLA.

[45]  Madan Musuvathi,et al.  Iterative context bounding for systematic testing of multithreaded programs , 2007, PLDI '07.

[46]  Yuanyuan Zhou,et al.  AVIO: Detecting Atomicity Violations via Access-Interleaving Invariants , 2007, IEEE Micro.

[47]  Shan Lu,et al.  A study of interleaving coverage criteria , 2007, ESEC-FSE '07.

[48]  Xiao Ma,et al.  MUVI: automatically inferring multi-variable access correlations and detecting related semantic and concurrency bugs , 2007, SOSP.

[49]  Yuanyuan Zhou,et al.  Learning from mistakes: a comprehensive study on real world concurrency bug characteristics , 2008, ASPLOS.

[50]  Thomas Ball,et al.  Finding and Reproducing Heisenbugs in Concurrent Programs , 2008, OSDI.

[51]  Eran Yahav,et al.  QVM: an efficient runtime for detecting defects in deployed systems , 2008, OOPSLA '08.

[52]  Patrice Godefroid,et al.  Automated Whitebox Fuzz Testing , 2008, NDSS.

[53]  Koushik Sen,et al.  Randomized active atomicity violation detection in concurrent programs , 2008, SIGSOFT '08/FSE-16.

[54]  Horatiu Jula,et al.  Deadlock Immunity: Enabling Systems to Defend Against Deadlocks , 2008, OSDI.

[55]  Koushik Sen,et al.  Predictive Typestate Checking of Multithreaded Java Programs , 2008, 2008 23rd IEEE/ACM International Conference on Automated Software Engineering.

[56]  Dawson R. Engler,et al.  KLEE: Unassisted and Automatic Generation of High-Coverage Tests for Complex Systems Programs , 2008, OSDI.

[57]  Anna Gringauze,et al.  Detecting Data Race and Atomicity Violation via Typestate-Guided Static Analysis , 2008 .

[58]  Peter M. Chen,et al.  Execution replay of multiprocessor virtual machines , 2008, VEE '08.

[59]  Koushik Sen,et al.  Race directed random testing of concurrent programs , 2008, PLDI '08.

[60]  Stephen N. Freund,et al.  Atomizer: A dynamic atomicity checker for multithreaded programs , 2008, Sci. Comput. Program..

[61]  Eran Yahav,et al.  Static Specification Mining Using Automata-Based Abstractions , 2008, IEEE Trans. Software Eng..

[62]  Satish Narayanasamy,et al.  A case for an interleaving constrained shared-memory multi-processor , 2009, ISCA '09.

[63]  Yuanyuan Zhou,et al.  CTrigger: exposing atomicity violation bugs from their hiding places , 2009, ASPLOS.

[64]  Darko Kirovski,et al.  Detecting and tolerating asymmetric races , 2009, PPoPP '09.

[65]  Yuanyuan Zhou,et al.  PRES: probabilistic replay with execution sketching on multiprocessors , 2009, SOSP '09.

[66]  Sriram K. Rajamani,et al.  ISOLATOR: dynamically ensuring isolation in comcurrent programs , 2009, ASPLOS.

[67]  Brandon Lucia,et al.  Finding concurrency bugs with context-aware communication graphs , 2009, 2009 42nd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO).

[68]  Brandon Lucia,et al.  Atom-Aid: Detecting and Surviving Atomicity Violations , 2009, IEEE Micro.

[69]  Zhiqiang Ma,et al.  Ad Hoc Synchronization Considered Harmful , 2010, OSDI.

[70]  Tal Garfinkel,et al.  Multi-stage replay with crosscut , 2010, VEE '10.

[71]  Pravesh Kothari,et al.  A randomized scheduler with probabilistic guarantees of finding bugs , 2010, ASPLOS XV.

[72]  George Candea,et al.  Execution synthesis: a technique for automated software debugging , 2010, EuroSys '10.

[73]  Richard W. Vuduc,et al.  Falcon: fault localization in concurrent programs , 2010, 2010 ACM/IEEE 32nd International Conference on Software Engineering.

[74]  Shan Lu,et al.  ConMem: detecting severe concurrency bugs through an effect-oriented approach , 2010, ASPLOS XV.

[75]  Chupoo Alafonté,et al.  Yesterday , 2023, Calabash. A journal of Caribbean arts and letters.