Profile-guided static typing for dynamic scripting languages

Many popular scripting languages such as Ruby, Python, and Perl include highly dynamic language constructs, such as an eval method that evaluates a string as program text. While these constructs allow terse and expressive code, they have traditionally obstructed static analysis. In this paper we present PRuby, an extension to Diamondback Ruby (DRuby), a static type inference system for Ruby. PRuby augments DRuby with a novel dynamic analysis and transformation that allows us to precisely type uses of highly dynamic constructs. PRuby's analysis proceeds in three steps. First, we use run-time instrumentation to gather per-application profiles of dynamic feature usage. Next, we replace dynamic features with statically analyzable alternatives based on the profile. We also add instrumentation to safely handle cases when subsequent runs do not match the profile. Finally, we run DRuby's static type inference on the transformed code to enforce type safety. We used PRuby to gather profiles for a benchmark suite of sample Ruby programs. We found that dynamic features are pervasive throughout the benchmarks and the libraries they include, but that most uses of these features are highly constrained and hence can be effectively profiled. Using the profiles to guide type inference, we found that DRuby can generally statically type our benchmarks modulo some refactoring, and we discovered several previously unknown type errors. These results suggest that profiling and transformation is a lightweight but highly effective approach to bring static typing to highly dynamic languages.

[1]  Peter Thiemann Towards a Type System for Analyzing JavaScript Programs , 2005, ESOP.

[2]  Jeremy G. Siek Gradual Typing for Functional Languages , 2006 .

[3]  Ralph E. Johnson,et al.  A type system for Smalltalk , 1989, POPL '90.

[4]  Jeffrey S. Foster,et al.  Profile-guided static typing for dynamic scripting languages , 2009, OOPSLA 2009.

[5]  Cormac Flanagan,et al.  Space-efficient gradual typing , 2010, High. Order Symb. Comput..

[6]  Walid Taha,et al.  Gradual Typing for Objects , 2007, ECOOP.

[7]  Michael Salib,et al.  Starkiller: A Static Type Inferencer and Compiler for Python , 2004 .

[8]  Sam Tobin-Hochstadt,et al.  The design and implementation of typed scheme , 2008, POPL '08.

[9]  Atanas Rountev,et al.  Improved Static Resolution of Dynamic Class Loading in Java , 2007, Seventh IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM 2007).

[10]  Stephen N. Freund,et al.  Sage: Hybrid Checking for Flexible Specifications , 2006 .

[11]  Davide Ancona,et al.  RPython: a step towards reconciling dynamically and statically typed OO languages , 2007, DLS '07.

[12]  Michael D. Bond,et al.  Tracking bad apples: reporting the origin of null and undefined value errors , 2007, OOPSLA.

[13]  Jeffrey S. Foster,et al.  Static type inference for Ruby , 2009, SAC '09.

[14]  Martín Abadi,et al.  Dynamic typing in a statically-typed language , 1989, POPL '89.

[15]  Dave Thomas,et al.  Programming Ruby , 2004 .

[16]  Aske Simon Christensen,et al.  Precise Analysis of String Expressions , 2003, SAS.

[17]  David Flanagan,et al.  The Ruby Programming Language , 2007 .

[18]  Vivek Sarkar,et al.  Dynamic optimistic interprocedural analysis: a framework and an application , 2001, OOPSLA '01.

[19]  Satish R. Thatte Quasi-static typing , 1989, POPL '90.

[20]  Amer Diwan,et al.  Pointer Analysis in the Presence of Dynamic Class Loading , 2004, ECOOP.

[21]  Frank Tip,et al.  Practical experience with an application extractor for Java , 1999, OOPSLA '99.

[22]  Sorin Lerner,et al.  Staged information flow for javascript , 2009, PLDI '09.

[23]  Robert Cartwright,et al.  Soft typing , 1991, PLDI '91.

[24]  Benjamin Livshits,et al.  Reflection Analysis for Java , 2005, APLAS.

[25]  Michael Hicks,et al.  The ruby intermediate language , 2009, DLS '09.

[26]  Sophia Drossopoulou,et al.  Towards Type Inference for JavaScript , 2005, ECOOP.

[27]  Premkumar T. Devanbu,et al.  Static checking of dynamically generated queries in database applications , 2004, Proceedings. 26th International Conference on Software Engineering.

[28]  Jong-Deok Choi,et al.  A framework for interprocedural optimization in the presence of dynamic class loading , 2000, PLDI '00.