Transparent runtime parallelization of the R scripting language

Scripting languages such as R and Matlab are widely used in scientific data processing. As the data volume and the complexity of analysis tasks both grow, sequential data processing using these tools often becomes the bottleneck in scientific workflows. We describe pR, a runtime framework for automatic and transparent parallelization of the popular R language used in statistical computing. Recognizing scripting languages' interpreted nature and data analysis codes' use pattern, we propose several novel techniques: (1) applying parallelizing compiler technology to runtime, whole-program dependence analysis of scripting languages, (2) incremental code analysis assisted with evaluation results, and (3) runtime parallelization of file accesses. Our framework does not require any modification to either the source code or the underlying R implementation. Experimental results demonstrate that pR can exploit both task and data parallelism transparently and overall has better performance as well as scalability compared to an existing parallel R package that requires code modification.

[1]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[2]  Manish Gupta,et al.  Techniques for Speculative Run-Time Parallelization of Loops , 1998, Proceedings of the IEEE/ACM SC98 Conference.

[3]  David F. Bacon,et al.  Compiler transformations for high-performance computing , 1994, CSUR.

[4]  Chen Yang,et al.  A cost-driven compilation framework for speculative parallelization of sequential programs , 2004, PLDI '04.

[5]  James Demmel,et al.  Models and Scheduling Algorithms for Mixed Data and Task Parallel Programs , 1997, J. Parallel Distributed Comput..

[6]  Arie Shoshani,et al.  High performance statistical computing with parallel R: Applications to biology and climate modelling , 2006 .

[7]  Yutao Zhong,et al.  Predicting whole-program locality through reuse distance analysis , 2003, PLDI.

[8]  Na Li,et al.  Simple Parallel Statistical Computing in R , 2007 .

[9]  Dennis Koelma,et al.  A Software Architecture for User Transparent Parallel Image Processing on MIMD Computers , 2001, Euro-Par.

[10]  Markus Mock,et al.  A retrospective on: "an evaluation of staged run-time optimizations in DyC" , 2004, SIGP.

[11]  Michael F. P. O'Boyle,et al.  A transformational approach to compiling Sisal for distributed memory architectures , 1992, ICS '92.

[12]  Kamesh Munagala,et al.  Suppression and failures in sensor networks: a Bayesian approach , 2007, VLDB 2007.

[13]  Paul Feautrier,et al.  Automatic Parallelization of Fortran Programs in the Presence of Procedure Calls , 1986, ESOP.

[14]  Tao Yang,et al.  PYRROS: static task scheduling and code generation for message passing multiprocessors , 1992 .

[15]  Jack Dongarra,et al.  ScaLAPACK: a scalable linear algebra library for distributed memory concurrent computers , 1992, [Proceedings 1992] The Fourth Symposium on the Frontiers of Massively Parallel Computation.

[16]  Michael J. Quinn,et al.  Preliminary results from a parallel MATLAB compiler , 1998, Proceedings of the First Merged International Parallel Processing Symposium and Symposium on Parallel and Distributed Processing.

[17]  Robert Wilson,et al.  Compiling Java just in time , 1997, IEEE Micro.

[18]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[19]  David A. Padua,et al.  MaJIC: compiling MATLAB for speed and responsiveness , 2002, PLDI '02.

[20]  Lawrence Rauchwerger,et al.  Run-Time Parallelization: Its Time Has Come , 1998, Parallel Comput..

[21]  Alan Edelman,et al.  Parallel MATLAB: Doing it Right , 2005, Proceedings of the IEEE.

[22]  Alok N. Choudhary,et al.  Match virtual machine: an adaptive runtime system to execute MATLAB in parallel , 2000, Proceedings 2000 International Conference on Parallel Processing.

[23]  Nagiza F. Samatova,et al.  RScaLAPACK: High-Performance Parallel Statistical Computing with R and ScaLAPACK , 2005, ISCA PDCS.

[24]  Mayer D. Schwartz,et al.  Incremental compilation in Magpie , 1984, SIGPLAN '84.

[25]  Kunle Olukotun,et al.  The Jrpm system for dynamically parallelizing Java programs , 2003, ISCA '03.

[26]  P J Miller.  Parallel, Distributed Scripting with Python , 2002 .

[27]  Anne Rogers,et al.  Compiling for Distributed Memory Architectures , 1994, IEEE Trans. Parallel Distributed Syst..

[28]  Ken Kennedy,et al.  A technique for summarizing data access and its use in parallelism enhancing transformations , 1989, PLDI '89.

[29]  Rudolf Eigenmann,et al.  Automatic program parallelization , 1993, Proc. IEEE.

[30]  Joel H. Saltz,et al.  Run-time parallelization and scheduling of loops , 1989, SPAA '89.

[31]  José Rolim,et al.  Parallel and distributed processing : 10 IPPS/SPDP'98 Workshops held in conjunction with the 12th International Parallel Processing Symposium and 9th Symposium on Parallel and Distrubuted Processing, Orlando, Florida, USA, March 30-April 3, 1998, proceedings , 1998 .

[32]  John R. Gilbert,et al.  Star-P: High Productivity Parallel Computing , 2004 .

[33]  Josep Torrellas,et al.  An efficient algorithm for the run-time parallelization of DOACROSS loops , 1994, Proceedings of Supercomputing '94.

[34]  Sally A. McKee,et al.  Machine learning based online performance prediction for runtime parallelization and task scheduling , 2009, 2009 IEEE International Symposium on Performance Analysis of Systems and Software.

[35]  Vikram S. Adve,et al.  High Performance Fortran Compilation Techniques for Parallelizing Scientific Codes , 1998, Proceedings of the IEEE/ACM SC98 Conference.

[36]  Lawrence Rauchwerger,et al.  Polaris: The Next Generation in Parallelizing Compilers , 2000 .

[37]  Monica S. Lam,et al.  Interprocedural parallelization analysis in SUIF , 2005, TOPL.

[38]  William Pugh,et al.  Uniform techniques for loop optimization , 1991, ICS '91.

[39]  Nagiza F. Samatova,et al.  Automatic Parallelization of Scripting Languages: Toward Transparent Desktop Parallel Computing , 2007, 2007 IEEE International Parallel and Distributed Processing Symposium.

[40]  Monica S. Lam,et al.  Efficient and exact data dependence analysis , 1991, PLDI '91.

[41]  John Garvin.  RCC: A compiler for the R language for statistical computing , 2004 .