Automated Verification of Functional Correctness of Race-Free GPU Programs

We study an automated verification method for functional correctness of parallel programs running on graphics processing units (GPUs). Our method is based on Kojima and Igarashi’s Hoare logic for GPU programs. Our algorithm generates verification conditions (VCs) from a program annotated by specifications and loop invariants, and passes them to off-the-shelf SMT solvers. It is often impossible, however, to solve naively generated VCs in reasonable time. A main difficulty stems from quantifiers over threads due to the parallel nature of GPU programs. To overcome this difficulty, we additionally apply several transformations to simplify VCs before calling SMT solvers. Our implementation successfully verifies correctness of several GPU programs, including matrix multiplication optimized by using shared memory. In contrast to many existing verification tools for GPU programs, our verifier succeeds in verifying fully parameterized programs: parameters such as the number of threads and the sizes of matrices are all symbolic. We empirically confirm that our simplification heuristics is highly effective for improving efficiency of the verification procedure.

[1]  François Bobot,et al.  Why3: Shepherd Your Herd of Provers , 2011 .

[2]  Hubert Nguyen,et al.  GPU Gems 3 , 2007 .

[3]  Joël Ouaknine,et al.  On the Complexity of Linear Arithmetic with Divisibility , 2015, 2015 30th Annual ACM/IEEE Symposium on Logic in Computer Science.

[4]  Cormac Flanagan,et al.  Avoiding exponential explosion: generating compact verification conditions , 2001, POPL '01.

[5]  Atsushi Igarashi,et al.  A Hoare Logic for SIMT Programs , 2013, APLAS.

[6]  Peng Li,et al.  Practical Symbolic Race Checking of GPU Programs , 2014, SC14: International Conference for High Performance Computing, Networking, Storage and Analysis.

[7]  David Detlefs,et al.  Simplify: a theorem prover for program checking , 2005, JACM.

[8]  Nikolaj Bjørner,et al.  Compositional verification of procedural programs using horn clauses over integers and arrays , 2015, 2015 Formal Methods in Computer-Aided Design (FMCAD).

[9]  Peng Li,et al.  GKLEE: concolic verification and test generation for GPUs , 2012, PPoPP '12.

[10]  Atsushi Igarashi,et al.  Automated Verification of Functional Correctness of Race-Free GPU Programs , 2016, VSTTE.

[11]  Atsushi Igarashi,et al.  A Hoare Logic for GPU Kernels , 2017, TOCL.

[12]  Stephen McCamant,et al.  The Daikon system for dynamic detection of likely invariants , 2007, Sci. Comput. Program..

[13]  Christof Löding,et al.  ICE: A Robust Framework for Learning Invariants , 2014, CAV.

[14]  George C. Necula,et al.  CIL: Intermediate Language and Tools for Analysis and Transformation of C Programs , 2002, CC.

[15]  Kenneth L. McMillan,et al.  Quantified Invariant Generation Using an Interpolating Saturation Prover , 2008, TACAS.

[16]  Guodong Li,et al.  Parameterized Verification of GPU Kernel Programs , 2012, 2012 IEEE 26th International Parallel and Distributed Processing Symposium Workshops & PhD Forum.

[17]  James C. King,et al.  Symbolic execution and program testing , 1976, CACM.

[18]  K. Rustan M. Leino,et al.  Houdini, an Annotation Assistant for ESC/Java , 2001, FME.

[19]  Andrei Voronkov,et al.  Finding Loop Invariants for Programs over Arrays Using a Theorem Prover , 2009, FASE.

[20]  Hidehiko Masuhara,et al.  Proof of Soundness of Concurrent Separation Logic for GPGPU in Coq , 2016, J. Inf. Process..

[21]  Paul H. J. Kelly,et al.  Symbolic Testing of OpenCL Code , 2011, Haifa Verification Conference.

[22]  John Wickerson,et al.  The Design and Implementation of a Verification Technique for GPU Kernels , 2015, TOPL.

[23]  Marieke Huisman,et al.  Specification and verification of GPGPU programs , 2013, Sci. Comput. Program..

[24]  Marius Bozga,et al.  On Decidability Within the Arithmetic of Addition and Divisibility , 2005, FoSSaCS.

[25]  Anna Philippou,et al.  Tools and Algorithms for the Construction and Analysis of Systems , 2018, Lecture Notes in Computer Science.

[26]  Guodong Li,et al.  Scalable SMT-based verification of GPU kernel functions , 2010, FSE '10.

[27]  Peng Li,et al.  Parametric flows: Automated behavior equivalencing for symbolic analysis of races in CUDA programs , 2012, 2012 International Conference for High Performance Computing, Networking, Storage and Analysis.

[28]  David Cachera,et al.  Inference of polynomial invariants for imperative programs: A farewell to Gröbner bases , 2014, Sci. Comput. Program..