Verifying finite-state properties of large-scale programs

Designers of software components can use finite-state properties to denote behavioral interface specifications which enforce client-side programming rules that state how the components ought to be used. This allows users of these components to check their client code for compliance with these rules, both statically and at runtime. In this dissertation we explain the design and implementation of C LARA, a framework for specifying and verifying finite-state properties of large-scale programs. With CLARA, programmers specify finite-state properties together with runtime monitors, using a syntactic extension to the aspect-oriented programming language AspectJ. CLARA then uses a sequence of three increasingly detailed static analyses to determine if the program satisfies the finite-state properties, i.e., is free of property violations. CLARA produces a list of program points at which the program may violate the properties, ranked by a confidence value. If violations are possible, CLARA also instruments the program with the supplied runtime monitor, which will capture property violations when the program executes. Due to its static analyses, CLARA can omit the instrumentation at program locations which the analyses proved safe, and so optimize the instrumented program. When much instrumentation remains, CLARA partitions the instrumentation into subsets, so that one can distribute multiple partially instrumented program versions that each run with a low overhead. We validated the approach by applying CLARA to finite-state properties denoted in multiple formalisms over several large-scale Java programs. CLARA proved that most of the programs fulfill our example properties. For most other programs, CLARA could remove the monitoring overhead to below 10%. We also found multiple property violations by manually inspecting the top entries in CLARA's ranked result list.

[1]  Ondrej Lhoták,et al.  A Staged Static Program Analysis to Improve the Performance of Runtime Monitoring , 2007, ECOOP.

[2]  Alexander Aiken,et al.  Relational queries over program traces , 2005, OOPSLA '05.

[3]  Jeffrey S. Foster,et al.  A comparison of bug finding tools for Java , 2004, 15th International Symposium on Software Reliability Engineering.

[4]  Heidi S. Kramer,et al.  Other publications , 2007, The English Historical Review.

[5]  Eric Bodden The design and implementation of formal monitoring techniques , 2007, OOPSLA '07.

[6]  Zhenmin Li,et al.  PR-Miner: automatically extracting implicit programming rules and detecting violations in large software code , 2005, ESEC/FSE-13.

[7]  Grigore Rosu,et al.  Mop: an efficient and generic runtime verification framework , 2007, OOPSLA.

[8]  Jack Greenfield,et al.  Software factories: assembling applications with patterns, models, frameworks and tools , 2004, OOPSLA '03.

[9]  Eric Bodden,et al.  Aspect-Oriented Race Detection in Java , 2010, IEEE Transactions on Software Engineering.

[10]  Eric Bodden Concern specific languages and their implementation with abc , 2005 .

[11]  Eric Bodden,et al.  Avoiding Infinite Recursion with Stratified Aspects , 2006, NODe/GSEM.

[12]  Jonathan Aldrich,et al.  Modular typestate checking of aliased objects , 2007, OOPSLA.

[13]  Gary T. Leavens,et al.  Design by Contract with JML , 2006 .

[14]  Robert DeLine,et al.  Typestates for Objects , 2004, ECOOP.

[15]  Hans Vangheluwe,et al.  Transforming Timeline Specifications into Automata for Runtime Monitoring , 2007, AGTIVE.

[16]  Ondrej Lhoták,et al.  Aspects for Trace Monitoring , 2006, FATES/RV.

[17]  Eric Bodden A lightweight LTL runtime verification tool for java , 2004, OOPSLA '04.

[18]  David Holmes,et al.  Java Concurrency in Practice , 2006 .

[19]  Eric Bodden,et al.  Object representatives: a uniform abstraction for pointer information , 2008, BCS Int. Acad. Conf..

[20]  Kathryn S. McKinley,et al.  Free-Me: a static analysis for automatic individual object reclamation , 2006, PLDI '06.

[21]  Eric Bodden Stateful breakpoints: a practical approach to defining parameterized runtime monitors , 2011, ESEC/FSE '11.

[22]  Ian H. Witten,et al.  Data mining - practical machine learning tools and techniques, Second Edition , 2005, The Morgan Kaufmann series in data management systems.

[23]  M. Clavel,et al.  Principles of Maude , 1996, WRLA.

[24]  William G. Griswold,et al.  Dynamically discovering likely program invariants to support program evolution , 1999, Proceedings of the 1999 International Conference on Software Engineering (IEEE Cat. No.99CB37002).

[25]  James R. Larus,et al.  Mining specifications , 2002, POPL '02.

[26]  Eric Bodden Continuation equivalence: a correctness criterion for static optimizations of dynamic analyses , 2011, WODA '11.

[27]  Ondrej Lhoták,et al.  Optimising aspectJ , 2005, PLDI '05.

[28]  Grigore Rosu,et al.  Parametric Trace Slicing and Monitoring , 2009, TACAS.

[29]  Jeannette M. Wing,et al.  A behavioral notion of subtyping , 1994, TOPL.

[30]  Stefan Hanenberg,et al.  Evolvable Pattern Implementations Need Generic Aspects , 2004, RAM-SE.

[31]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques, 3rd Edition , 1999 .

[32]  Amir Pnueli,et al.  The temporal logic of programs , 1977, 18th Annual Symposium on Foundations of Computer Science (sfcs 1977).

[33]  Jim Hugunin,et al.  Advice weaving in AspectJ , 2004, AOSD '04.

[34]  Amer Diwan,et al.  The DaCapo benchmarks: java benchmarking development and analysis , 2006, OOPSLA '06.

[35]  Naoyasu Ubayashi,et al.  Association aspects , 2004, AOSD '04.

[36]  Mira Mezini,et al.  Sicher fahren: Absicherung moderner Fahrzeugsoftware , 2010 .

[37]  Eran Yahav,et al.  Typestate verification: Abstraction techniques and complexity results , 2005, Sci. Comput. Program..

[38]  Zhendong Su,et al.  Javert: fully automatic mining of general temporal properties from dynamic traces , 2008, SIGSOFT '08/FSE-16.

[39]  Matthew B. Dwyer,et al.  Residual dynamic typestate analysis exploiting static analysis: results to reformulate and reduce the cost of dynamic analysis , 2007, ASE.

[40]  Volker Stolz,et al.  Runtime Verification of Concurrent Haskell Programs , 2005, RV@ETAPS.

[41]  Hidehiko Masuhara,et al.  Dataflow Pointcut in Aspect-Oriented Programming , 2003, APLAS.

[42]  Eric Bodden,et al.  Finding programming errors earlier by evaluating runtime monitors ahead-of-time , 2008, SIGSOFT '08/FSE-16.

[43]  Robert E. Strom,et al.  Typestate: A programming language concept for enhancing software reliability , 1986, IEEE Transactions on Software Engineering.

[44]  Andreas Zeller,et al.  Why Programs Fail: A Guide to Systematic Debugging , 2005 .

[45]  Ondrej Lhoták,et al.  Adding trace matching with free variables to AspectJ , 2005, OOPSLA '05.

[46]  Ron Kohavi,et al.  A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection , 1995, IJCAI.

[47]  Gary T. Leavens,et al.  Modular verification of higher-order methods with mandatory calls specified by model programs , 2007, OOPSLA.

[48]  Hans Vangheluwe,et al.  Domain-Specific Modelling With Atom3 , 2007, ICSOFT.

[49]  Eric Bodden,et al.  Join point interfaces for modular reasoning in aspect-oriented programs , 2011, ESEC/FSE '11.

[50]  Michael I. Jordan,et al.  Bug isolation via remote program sampling , 2003, PLDI.

[51]  Manu Sridharan,et al.  Refinement-based context-sensitive points-to analysis for Java , 2006, PLDI '06.

[52]  Eric Bodden,et al.  Effective API navigation and reuse , 2010, 2010 IEEE International Conference on Information Reuse & Integration.

[53]  K. Rustan M. Leino,et al.  Houdini, an Annotation Assistant for ESC/Java , 2001, FME.

[54]  Rajkishore Barik,et al.  Efficient Computation of May-Happen-in-Parallel Information for Concurrent Java Programs , 2005, LCPC.

[55]  Monica S. Lam,et al.  Finding Application Errors Using PQL: a Program Query Language , 2004 .

[56]  Mira Mezini,et al.  IDE 2.0: collective intelligence in software development , 2010, FoSER '10.

[57]  Eric Bodden,et al.  Racer: effective race detection using aspectj , 2008, ISSTA '08.

[58]  Eric Bodden Closure joinpoints: block joinpoints without surprises , 2011, AOSD '11.

[59]  Eric Bodden,et al.  Relational aspects as tracematches , 2008, AOSD.

[60]  Eran Yahav,et al.  QVM: an efficient runtime for detecting defects in deployed systems , 2008, OOPSLA '08.

[61]  Volker Stolz,et al.  Tracechecks: Defining Semantic Interfaces with Temporal Logic , 2006, SC@ETAPS.

[62]  Ondrej Lhoták,et al.  The Soot framework for Java program analysis: a retrospective , 2011 .

[63]  Madeline Diep,et al.  Reducing the Cost of Path Property Monitoring Through Sampling , 2008, 2008 23rd IEEE/ACM International Conference on Automated Software Engineering.

[64]  Eric Bodden Efficient hybrid typestate analysis by determining continuation-equivalent states , 2010, 2010 ACM/IEEE 32nd International Conference on Software Engineering.

[65]  Eric Bodden,et al.  Clara: Partially Evaluating Runtime Monitors at Compile Time - Tutorial Supplement , 2010, RV.

[66]  J. Brzozowski Canonical regular expressions and minimal state graphs for definite events , 1962 .

[67]  Mayuram S. Krishnan,et al.  Evaluating the cost of software quality , 1998, CACM.

[68]  Hidehiko Masuhara,et al.  SCoPE: an AspectJ compiler for supporting user-defined analysis-based pointcuts , 2007, AOSD.

[69]  Eric Bodden Efficient and Precise Typestate Analysis by Determining Continuation-Equivalent States , 2009 .

[70]  Grigore Rosu,et al.  Towards Monitoring-Oriented Programming: A Paradigm Combining Specification and Implementation , 2003, RV@CAV.

[71]  Mira Mezini,et al.  Taming reflection: Aiding static analysis in the presence of reflection and custom class loaders , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[72]  George S. Avrunin,et al.  Patterns in property specifications for finite-state verification , 1999, Proceedings of the 1999 International Conference on Software Engineering (IEEE Cat. No.99CB37002).

[73]  Robert J. Walker,et al.  Implementing protocols via declarative event patterns , 2004, SIGSOFT '04/FSE-12.

[74]  Peter Linz,et al.  An Introduction to Formal Languages and Automata , 1997 .

[75]  Mark Lillibridge,et al.  Extended static checking for Java , 2002, PLDI '02.

[76]  Ondrej Lhoták,et al.  abc: an extensible AspectJ compiler , 2005, AOSD '05.

[77]  Eran Yahav,et al.  Effective typestate verification in the presence of aliasing , 2006, TSEM.

[78]  Grigore Rosu,et al.  Dependent advice: a general approach to optimizing history-based aspects , 2009, AOSD '09.

[79]  Andreas Zeller,et al.  Detecting object usage anomalies , 2007, ESEC-FSE '07.

[80]  Clark Verbrugge,et al.  Component-Based Lock Allocation , 2007, 16th International Conference on Parallel Architecture and Compilation Techniques (PACT 2007).

[81]  William F. Clocksin,et al.  Programming in Prolog , 1987, Springer Berlin Heidelberg.

[82]  Eric Bodden,et al.  Clara: a Framework for Statically Evaluating Finite-state Runtime Monitors , 2010, Runtime Verification.

[83]  Ondrej Lhoták,et al.  Scaling Java Points-to Analysis Using SPARK , 2003, CC.

[84]  David Harel,et al.  From multi-modal scenarios to code: compiling LSCs into aspectJ , 2006, SIGSOFT '06/FSE-14.

[85]  Joseph Y. Halpern,et al.  “Sometimes” and “not never” revisited: on branching versus linear time temporal logic , 1986, JACM.

[86]  Eric Bodden MOPBox: A Library Approach to Runtime Verification - (Tool Demonstration) , 2011, RV.

[87]  Ondrej Lhoták,et al.  Typestate-like analysis of multiple interacting objects , 2008, OOPSLA.

[88]  Hidehiko Masuhara,et al.  A Compilation and Optimization Model for Aspect-Oriented Programs , 2003, CC.

[89]  David Hovemeyer,et al.  Finding bugs is easy , 2004, SIGP.

[90]  Martin Leucker,et al.  The Good, the Bad, and the Ugly, But How Ugly Is Ugly? , 2007, RV.

[91]  Ingolf H. Krüger,et al.  Automating software architecture exploration with M2Aspects , 2006, SCESM '06.

[92]  Ondrej Lhoták,et al.  Efficient trace monitoring , 2006, OOPSLA '06.

[93]  Sudheendra Hangal,et al.  Tracking down software bugs using automatic anomaly detection , 2002, ICSE '02.

[94]  David Harel,et al.  Statecharts: A Visual Formalism for Complex Systems , 1987, Sci. Comput. Program..

[95]  Eric Bodden,et al.  abc: How to implement your own tools for AOP research , 2006 .

[96]  Volker Stolz,et al.  Temporal Assertions using AspectJ , 2006, Electron. Notes Theor. Comput. Sci..

[97]  David Harel,et al.  LSCs: Breathing Life into Message Sequence Charts , 1999, Formal Methods Syst. Des..

[98]  Guy L. Steele,et al.  Java(TM) Language Specification , 2005 .

[99]  Eric Bodden,et al.  A high-level view of Java applications , 2003, OOPSLA '03.

[100]  Mira Mezini,et al.  Expressive Pointcuts for Increased Modularity , 2005, ECOOP.

[101]  Ondrej Lhoták,et al.  Collaborative Runtime Verification with Tracematches , 2010, J. Log. Comput..

[102]  Koushik Sen,et al.  Race directed random testing of concurrent programs , 2008, PLDI '08.

[103]  Sarfraz Khurshid,et al.  Reducing Configurations to Monitor in a Software Product Line , 2010, RV.

[104]  Elnar Hajiyev,et al.  Aspects and Data Refinement , 2006, MPC.

[105]  Eric Bodden,et al.  The Clara framework for hybrid typestate analysis , 2010, International Journal on Software Tools for Technology Transfer.

[106]  Eric Bodden Specifying and Exploiting Advice-Execution Ordering using Dependency State Machines , 2010 .

[107]  Volker Stolz,et al.  Efficient temporal pointcuts through dynamic advice deployment , 2006 .

[108]  Lori A. Clarke,et al.  Flow analysis for verifying properties of concurrent software systems , 2004, TSEM.