Towards optimization-safe systems: analyzing the impact of undefined behavior

This paper studies an emerging class of software bugs called optimization-unstable code: code that is unexpectedly discarded by compiler optimizations due to undefined behavior in the program. Unstable code is present in many systems, including the Linux kernel and the Postgres database. The consequences of unstable code range from incorrect functionality to missing security checks. To reason about unstable code, this paper proposes a novel model, which views unstable code in terms of optimizations that leverage undefined behavior. Using this model, we introduce a new static checker called Stack that precisely identifies unstable code. Applying Stack to widely used systems has uncovered 160 new bugs that have been confirmed and fixed by developers.

[1]  Peng Li,et al.  Understanding integer overflow in C/C++ , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[2]  Isil Dillig,et al.  Static error detection using semantic inconsistency inference , 2007, PLDI '07.

[3]  Vikram S. Adve,et al.  LLVM: a compilation framework for lifelong program analysis & transformation , 2004, International Symposium on Code Generation and Optimization, 2004. CGO 2004..

[4]  Chucky Ellison,et al.  An executable formal semantics of C with applications , 2011, POPL '12.

[5]  Eddie Kohler,et al.  Making information flow explicit in HiStar , 2006, OSDI '06.

[6]  Alberto Griggio,et al.  Computing Small Unsatisfiable Cores in Satisfiability Modulo Theories , 2014, J. Artif. Intell. Res..

[7]  Chucky Ellison,et al.  Defining the undefinedness of C , 2015, PLDI.

[8]  Hari Balakrishnan,et al.  Mosh: An Interactive Remote Shell for Mobile Clients , 2012, USENIX Annual Technical Conference.

[9]  David A. Padua,et al.  Gated SSA-based demand-driven symbolic analysis for parallelizing compilers , 1995, ICS '95.

[10]  Benjamin Monate,et al.  A Value Analysis for C Programs , 2009, 2009 Ninth IEEE International Working Conference on Source Code Analysis and Manipulation.

[11]  Armin Biere,et al.  Boolector: An Efficient SMT Solver for Bit-Vectors and Arrays , 2009, TACAS.

[12]  Dawson R. Engler,et al.  KLEE: Unassisted and Automatic Generation of High-Coverage Tests for Complex Systems Programs , 2008, OSDI.

[13]  Robbert Krebbers,et al.  Subtleties of the ANSI / ISO C standard , 2012 .

[14]  Alvin Cheung,et al.  Undefined behavior: what happened to my code? , 2012, APSys.

[15]  Xi Wang,et al.  Improving Integer Security for Systems with KINT , 2012, OSDI.

[16]  Steven S. Muchnick,et al.  Advanced Compiler Design and Implementation , 1997 .

[17]  Hans-Juergen Boehm,et al.  HP Laboratories , 2006 .

[18]  Xi Wang,et al.  Linux kernel vulnerabilities: state-of-the-art defenses and open problems , 2011, APSys.

[19]  Christoforos E. Kozyrakis,et al.  Usenix Association 10th Usenix Symposium on Operating Systems Design and Implementation (osdi '12) 335 Dune: Safe User-level Access to Privileged Cpu Features , 2022 .

[20]  Mark Stephenson,et al.  Bidwidth analysis with application to silicon compilation , 2000, PLDI '00.

[21]  No License,et al.  Intel ® 64 and IA-32 Architectures Software Developer ’ s Manual Volume 3 A : System Programming Guide , Part 1 , 2006 .