Solution Enumeration Abstraction: A Modeling Idiom to Enhance a Lightweight Formal Method

Formal methods are a key to engineering more reliable systems. In this paper, we focus on an important application of formal methods — enumerating solutions to logical formulas that encode properties of interest. Solution enumeration has many uses, e.g., in systematic software testing, model counting, or hardware analysis. We introduce solution enumeration abstraction, a novel idiom that allows users to define data abstractions to enhance solution enumeration by specifying how the solutions must differ, so enumeration creates a high quality set of solutions of a manageable size. We embody the idiom as a technique built on top of Alloy, a well-known lightweight formal method, which is comprised of a first-order relational logic with transitive closure, and a SAT-based analysis engine. Experimental results show that our technique supports a variety of data abstractions, and can substantially reduce the number of solutions enumerated and the time to enumerate them.

[1]  Niklas Sörensson,et al.  An Extensible SAT-solver , 2003, SAT.

[2]  Shriram Krishnamurthi,et al.  CompoSAT: Specification-Guided Coverage for Model Finding , 2018, FM.

[3]  Albert L. Baker,et al.  Preliminary design of JML: a behavioral interface specification language for java , 2006, SOEN.

[4]  Sarfraz Khurshid,et al.  Specification-Based Program Repair Using SAT , 2011, TACAS.

[5]  David Notkin,et al.  Rostra: a framework for detecting redundant object-oriented unit tests , 2004, Proceedings. 19th International Conference on Automated Software Engineering, 2004..

[6]  Sharad Malik,et al.  Constrained Sampling and Counting: Universal Hashing Meets SAT Solving , 2015, AAAI Workshop: Beyond NP.

[7]  Daniel Jackson,et al.  Software Abstractions - Logic, Language, and Analysis , 2006 .

[8]  Michael D. Ernst,et al.  Randoop: feedback-directed random testing for Java , 2007, OOPSLA '07.

[9]  Ronald L. Rivest,et al.  Introduction to Algorithms , 1990 .

[10]  Sarfraz Khurshid,et al.  Korat: automated testing based on Java predicates , 2002, ISSTA '02.

[11]  K. Rustan M. Leino,et al.  A Verification Methodology for Model Fields , 2006, ESOP.

[12]  Sarfraz Khurshid,et al.  TestEra: a novel framework for automated testing of Java programs , 2001, Proceedings 16th Annual International Conference on Automated Software Engineering (ASE 2001).

[13]  Marcelo F. Frias,et al.  TACO: Efficient SAT-Based Bounded Verification Using Symmetry Breaking and Tight Bounds , 2013, IEEE Transactions on Software Engineering.

[14]  Ilya Shlyakhter.  Generating effective symmetry-breaking predicates for search problems , 2007, Discret. Appl. Math..

[15]  A. Jefferson Offutt,et al.  Introduction to Software Testing , 2008 .

[16]  Ronald L. Rivest,et al.  Introduction to Algorithms, third edition , 2009 .

[17]  Sarfraz Khurshid,et al.  Contract-Based Data Structure Repair Using Alloy , 2010, ECOOP.

[18]  Stephen J. Garland,et al.  Larch: Languages and Tools for Formal Specification , 1993, Texts and Monographs in Computer Science.

[19]  Corina S. Pasareanu,et al.  Concrete Model Checking with Abstract Matching and Refinement , 2005, CAV.

[20]  Kathi Fisler,et al.  Aluminum: Principled scenario exploration through minimality , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[21]  Daniel Jackson,et al.  Finding bugs with a constraint solver , 2000, ISSTA '00.

[22]  James Mackenzie Crawford.  A theoretical analysis of reasoning by symmetry in first-order logic (extended abstract) , 1992 .

[23]  Dan Grossman,et al.  A solver-aided language for test input generation , 2017, Proc. ACM Program. Lang..

[24]  Pamela Zave,et al.  Reasoning About Identifier Spaces: How to Make Chord Correct , 2016, IEEE Transactions on Software Engineering.

[25]  Kevin J. Sullivan,et al.  COM revisited: tool-assisted modelling of an architectural framework , 2000, SIGSOFT '00/FSE-8.

[26]  Derek Rayside,et al.  Synthesizing iterators from abstraction functions , 2012, GPCE '12.

[27]  Sarfraz Khurshid,et al.  Automated Test Generation and Mutation Testing for Alloy , 2017, 2017 IEEE International Conference on Software Testing, Verification and Validation (ICST).

[28]  Viktor Kuncak,et al.  Programming with enumerable sets of structures , 2015, OOPSLA.

[29]  Laurie J. Hendren,et al.  Is it a tree, a DAG, or a cyclic graph? A shape analysis for heap-directed pointers in C , 1996, POPL '96.

[30]  Barbara Liskov,et al.  Program Development in Java - Abstraction, Specification, and Object-Oriented Design , 1986 .

[31]  Corina S. Pasareanu,et al.  Reliability analysis in Symbolic PathFinder , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[32]  Margaret Martonosi,et al.  Security Verification via Automatic Hardware-Aware Exploit Synthesis: The CheckMate Approach , 2019, IEEE Micro.

[33]  Koushik Sen,et al.  SMTSampler: Efficient Stimulus Generation from Complex SMT Constraints , 2018, 2018 IEEE/ACM International Conference on Computer-Aided Design (ICCAD).

[34]  Sarfraz Khurshid,et al.  A Case for Efficient Solution Enumeration , 2003, SAT.

[35]  Margaret Martonosi,et al.  CheckMate: Automated Synthesis of Hardware Exploits and Security Litmus Tests , 2018, 2018 51st Annual IEEE/ACM International Symposium on Microarchitecture (MICRO).

[36]  John Wickerson,et al.  The semantics of transactions and weak memory in x86, Power, ARM, and C++ , 2017, PLDI.

[37]  Emina Torlak,et al.  Kodkod: A Relational Model Finder , 2007, TACAS.

[38]  Nazareno Aguirre,et al.  Field-exhaustive testing , 2016, SIGSOFT FSE.

[39]  Eran Yahav,et al.  Predicate Abstraction and Canonical Abstraction for Singly-Linked Lists , 2005, VMCAI.

[40]  Sarfraz Khurshid,et al.  Test generation through programming in UDITA , 2010, 2010 ACM/IEEE 32nd International Conference on Software Engineering.

[41]  Todd D. Millstein,et al.  Falling Back on Executable Specifications , 2010, ECOOP.

[42]  Nikolaj Bjørner,et al.  Z3: An Efficient SMT Solver , 2008, TACAS.

[43]  Sam Malek,et al.  A formal approach for detection of security flaws in the android permission system , 2017, Formal Aspects of Computing.

[44]  Derek Rayside,et al.  Equality and hashing for (almost) free: Generating implementations from abstraction functions , 2009, 2009 IEEE 31st International Conference on Software Engineering.

[45]  George A. Constantinides,et al.  Automatically comparing memory consistency models , 2017, POPL.

[46]  Sarfraz Khurshid,et al.  Exploring the design of an intentional naming scheme with an automatic constraint analyzer , 2000, Proceedings ASE 2000. Fifteenth IEEE International Conference on Automated Software Engineering.