Poster: Pairika—A Failure Diagnosis Benchmark for C++ Programs

Empirical studies in software testing require realistic benchmarks which are able to mimic industry-like environments. For evaluating automated failure diagnosis techniques, one needs real reproducible bugs with at least one associated failing test. Extracting such bugs is challenging and time-consuming. This paper presents Pairika, a failure diagnosis benchmark for C++ programs. Pairika contains 40 bugs extracted from 7 modules of OpenCV project with more than 490 KLoC and 11129 tests. Each bug is accompanied by at least one failing test. We publish Pairika to facilitate and stimulate further research on automated failure diagnosis techniques. Pairika is available at: https://github.com/tum-i22/Pairika