Testing Error Handling Code in Device Drivers Using Characteristic Fault Injection

Device drivers may encounter errors when communicating with OS kernel and hardware. However, error handling code often gets insufficient attention in driver development and testing, because these errors rarely occur in real execution. For this reason, many bugs are hidden in error handling code. Previous approaches for testing error handling code often neglect the characteristics of device drivers, so their efficiency and accuracy are limited. In this paper, we first study the source code of Linux drivers to find useful characteristics of error handling code. Then we use these characteristics in fault injection testing, and propose a novel approach named EH-Test, which can efficiently test error handling code in drivers. To improve the representativeness of injected faults, we design a pattern-based extraction strategy to automatically and accurately extract target functions which can actually fail and trigger error handling code. During execution, we use a monitor to record runtime information and pair checkers to check resource usages. We have evaluated EH-Test on 15 real Linux device drivers and found 50 new bugs in Linux 3.17.2. The code coverage is also effectively increased. Comparison experiments to previous related approaches also show the effectiveness of EH-Test.

[1]  Daniel Kroening,et al.  Model checking concurrent linux device drivers , 2007, ASE.

[2]  George Candea,et al.  Efficient Testing of Recovery Code Using Fault Injection , 2011, TOCS.

[3]  Neeraj Suri,et al.  An empirical study of injected versus actual interface errors , 2014, ISSTA 2014.

[4]  Nuno Ferreira Neves,et al.  Robustness Testing of the Windows DDK , 2007, 37th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN'07).

[5]  Neeraj Suri,et al.  On the Selection of Error Model(s) for OS Robustness Evaluation , 2007, 37th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN'07).

[6]  George Candea,et al.  Fast black-box testing of system recovery code , 2012, EuroSys '12.

[7]  Brian N. Bershad,et al.  Recovering device drivers , 2004, TOCS.

[8]  George Candea,et al.  Testing Closed-Source Binary Device Drivers with DDT , 2010, USENIX Annual Technical Conference.

[9]  П. Довгалюк,et al.  Два способа организации механизма полносистемного детерминированного воспроизведения в симуляторе QEMU , 2012 .

[10]  Sebastian G. Elbaum,et al.  Amplifying tests to validate exception handling code , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[11]  Daniel Kroening,et al.  A Tool for Checking ANSI-C Programs , 2004, TACAS.

[12]  Tao Xie,et al.  Iterative mining of resource-releasing specifications , 2011, 2011 26th IEEE/ACM International Conference on Automated Software Engineering (ASE 2011).

[13]  B B Meshram,et al.  Device Driver Fault Simulation Using KEDR , 2012 .

[14]  Sidney Amani,et al.  Static analysis of device drivers: we can do better! , 2011, APSys.

[15]  Domenico Cotroneo,et al.  On Fault Representativeness of Software Fault Injection , 2013, IEEE Transactions on Software Engineering.

[16]  Li Lei,et al.  Automatic fault injection for driver robustness testing , 2015, ISSTA.

[17]  Jan Obdrzálek,et al.  STANSE: Bug-Finding Framework for C Programs , 2011, MEMICS.

[18]  Manuvir Das,et al.  Perracotta: mining temporal API rules from imperfect traces , 2006, ICSE.

[19]  Suman Saha,et al.  An approach to improving the structure of error-handling code in the linux kernel , 2011, LCTES '11.

[20]  Fabrice Bellard,et al.  QEMU, a Fast and Portable Dynamic Translator , 2005, USENIX ATC, FREENIX Track.

[21]  Chen Fu,et al.  Testing of java web services for robustness , 2004, ISSTA '04.

[22]  Takeshi Yoshimura,et al.  Do Injected Faults Cause Real Failures? A Case Study of Linux , 2014, 2014 IEEE International Symposium on Software Reliability Engineering Workshops.

[23]  Vladimir V. Rubanov,et al.  Runtime Verification of Linux Kernel Modules Based on Call Interception , 2011, 2011 Fourth IEEE International Conference on Software Testing, Verification and Validation.

[24]  Zhenmin Li,et al.  PR-Miner: automatically extracting implicit programming rules and detecting violations in large software code , 2005, ESEC/FSE-13.

[25]  Asim Kadav,et al.  SymDrive: Testing Drivers without Devices , 2012, OSDI.

[26]  Domenico Cotroneo,et al.  Representativeness analysis of injected software faults in complex software , 2010, 2010 IEEE/IFIP International Conference on Dependable Systems & Networks (DSN).

[27]  George Candea,et al.  S2E: a platform for in-vivo multi-path analysis of software systems , 2011, ASPLOS XVI.

[28]  Brian N. Bershad,et al.  Improving the reliability of commodity operating systems , 2005, TOCS.

[29]  Cristiano Giuffrida,et al.  EDFI: A Dependable Fault Injection Tool for Dependability Benchmarking Experiments , 2013, 2013 IEEE 19th Pacific Rim International Symposium on Dependable Computing.

[30]  Henrique Madeira,et al.  Emulation of Software Faults: A Field Data Study and a Practical Approach , 2006, IEEE Transactions on Software Engineering.

[31]  Greg Kroah-Hartman,et al.  Linux Device Drivers, 3rd Edition , 2005 .

[32]  Rob Williams,et al.  Linux device drivers , 2006 .

[33]  Wolfgang Küchlin,et al.  Integrated Static Analysis for Linux Device Driver Verification , 2007, IFM.

[34]  Sriram K. Rajamani,et al.  Thorough static analysis of device drivers , 2006, EuroSys.

[35]  Suman Saha,et al.  Hector: Detecting Resource-Release Omission Faults in error-handling code for systems software , 2013, 2013 43rd Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN).

[36]  George Candea,et al.  LFI: A practical and general library-level fault injector , 2009, 2009 IEEE/IFIP International Conference on Dependable Systems & Networks.

[37]  Julia L. Lawall,et al.  WYSIWIB: A declarative approach to finding API protocols and bugs in Linux code , 2009, DSN.

[38]  Yuping Wang,et al.  PF-Miner: A New Paired Functions Mining Method for Android Kernel in Error Paths , 2014, 2014 IEEE 38th Annual Computer Software and Applications Conference.

[39]  Li Lei,et al.  Symbolic Execution of Virtual Devices , 2013, 2013 13th International Conference on Quality Software.

[40]  Neeraj Suri,et al.  simFI: From single to simultaneous software fault injections , 2013, 2013 43rd Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN).

[41]  Koushik Sen,et al.  PREFAIL: a programmable tool for multiple-failure injection , 2011, OOPSLA '11.