Many-core compiler fuzzing

We address the compiler correctness problem for many-core systems through novel applications of fuzz testing to OpenCL compilers. Focusing on two methods from prior work, random differential testing and testing via equivalence modulo inputs (EMI), we present several strategies for random generation of deterministic, communicating OpenCL kernels, and an injection mechanism that allows EMI testing to be applied to kernels that otherwise exhibit little or no dynamically-dead code. We use these methods to conduct a large, controlled testing campaign with respect to 21 OpenCL (device, compiler) configurations, covering a range of CPU, GPU, accelerator, FPGA and emulator implementations. Our study provides independent validation of claims in prior work related to the effectiveness of random differential testing and EMI testing, proposes novel methods for lifting these techniques to the many-core setting and reveals a significant number of OpenCL compiler bugs in commercial implementations.

[1]  Tsong Yueh Chen,et al.  Fault-based testing without the need of oracles , 2003, Inf. Softw. Technol..

[2]  Kevin Skadron,et al.  Rodinia: A benchmark suite for heterogeneous computing , 2009, 2009 IEEE International Symposium on Workload Characterization (IISWC).

[3]  Alfred V. Aho,et al.  Compilers: Principles, Techniques, and Tools (2nd Edition) , 2006 .

[4]  Xuejun Yang,et al.  Test-case reduction for C compiler bugs , 2012, PLDI.

[5]  Alastair F. Donaldson,et al.  A sound and complete abstraction for reasoning about parallel prefix sums , 2014, POPL.

[6]  Simon McIntosh-Smith,et al.  Oclgrind: an extensible OpenCL device simulator , 2015, IWOCL.

[7]  Alex Groce,et al.  Taming compiler fuzzers , 2013, ACM-SIGPLAN Symposium on Programming Language Design and Implementation.

[8]  Adam Betts,et al.  Engineering a Static Verification Tool for GPU Kernels , 2014, CAV.

[9]  Wei Wu,et al.  An Automatic Testing Approach for Compiler Based on Metamorphic Testing Technique , 2010, 2010 Asia Pacific Software Engineering Conference.

[10]  Xuejun Yang,et al.  Finding and understanding bugs in C compilers , 2011, PLDI '11.

[11]  Eric Eide,et al.  Volatiles are miscompiled, and what to do about it , 2008, EMSOFT '08.

[12]  Wen-mei W. Hwu,et al.  Parboil: A Revised Benchmark Suite for Scientific and Commercial Throughput Computing , 2012 .

[13]  Liang Guo,et al.  Automated test program generation for an industrial optimizing compiler , 2009, 2009 ICSE Workshop on Automation of Software Test.

[14]  Alfred V. Aho,et al.  Compilers: Principles, Techniques, and Tools , 1986, Addison-Wesley series in computer science / World student series edition.

[15]  Francesco Zappa Nardelli,et al.  Compiler testing via a theory of sound optimisations in the C11/C++11 memory model , 2013, PLDI.

[16]  Aaftab Munshi,et al.  The OpenCL specification , 2009, 2009 IEEE Hot Chips 21 Symposium (HCS).

[17]  Andreas Zeller,et al.  Fuzzing with Code Fragments , 2012, USENIX Security Symposium.

[18]  Flash Sheridan,et al.  Practical testing of a C99 compiler using output comparison , 2007, Softw. Pract. Exp..

[19]  W. M. McKeeman,et al.  Differential Testing for Software , 1998, Digit. Tech. J..

[20]  Zhendong Su,et al.  Compiler validation via equivalence modulo inputs , 2014, PLDI.