Optimizing aggregate array computations in loops

An aggregate array computation is a loop that computes accumulated quantities over array elements. Such computations are common in programs that use arrays, and the array elements involved in such computations often overlap, especially across iterations of loops, resulting in significant redundancy in the overall computations. This article presents a method and algorithms that eliminate such overlapping aggregate array redundancies and shows analytical and experimental performance improvements. The method is based on incrementalization, that is, updating the values of aggregate array computations from iteration to iteration rather than computing them from scratch in each iteration. This involves maintaining additional values not maintained in the original program. We reduce various analysis problems to solving inequality constraints on loop variables and array subscripts, and we apply results from work on array data dependence analysis. For aggregate array computations that have significant redundancy, incrementalization produces drastic speedup compared to previous optimizations; when there is little redundancy, the benefit might be offset by cache effects and other factors. Previous methods for loop optimizations of arrays do not perform incrementalization, and previous techniques for loop incrementalization do not handle arrays.

[1]  William Pugh,et al.  A practical algorithm for exact array dependence analysis , 1992, CACM.

[2]  Richard C. Waters,et al.  Automatic transformation of series expressions into loops , 1991, TOPL.

[3]  R. Paige Symbolic finite differencing, part I (invited lecture) , 1990 .

[4]  Ken Kennedy,et al.  An algorithm for reduction of operator strength , 1977, Commun. ACM.

[5]  V. K. Garg,et al.  An Efficient Algorithm for Detecting Conjunctions of General Global Predicate , 1996 .

[6]  William M. Wells,et al.  Efficient Synthesis of Gaussian Filters by Cascaded Uniform Filters , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  K. Li,et al.  Interferometric strain/slope rosette for static and dynamic measurements , 1997 .

[8]  Vivek Sarkar,et al.  A general framework for iteration-reordering loop transformations , 1992, PLDI '92.

[9]  Jon A. Webb Steps toward architecture-independent image processing , 1992, Computer.

[10]  Vadim Maslov,et al.  Lazy array data-flow dependence analysis , 1994, POPL '94.

[11]  Allan L. Fisher,et al.  Communication and code optimization in SIMD programs , 1988, ICPP.

[12]  Richard S. Bird The promotion and accumulation strategies in transformational programming , 1984, TOPL.

[13]  Ramin Zabih,et al.  Non-parametric Local Transforms for Computing Visual Correspondence , 1994, ECCV.

[14]  Ben Wegbreit,et al.  Goal-Directed Program Transformation , 1976, IEEE Transactions on Software Engineering.

[15]  Hans Langmaack,et al.  Translation of Algol 60 , 1967 .

[16]  Charles Consel,et al.  A general approach for run-time specialization and its application to C , 1996, POPL '96.

[17]  Jay Earley,et al.  High Level Iterators and a Method for Automatically Designing Data Structure Representation , 1976, Comput. Lang..

[18]  Yanhong A. Liu,et al.  Systematic Derivation of Incremental Programs , 1995, Sci. Comput. Program..

[19]  LiNing,et al.  Optimizing aggregate array computations in loops , 2005 .

[20]  Yanhong A. Liu,et al.  Static caching for incremental computation , 1998, TOPL.

[21]  Siddhartha Chatterjee,et al.  Exact analysis of the cache behavior of nested loops , 2001, PLDI '01.

[22]  William Pugh,et al.  Finding Legal Reordering Transformations Using Mappings , 1994, LCPC.

[23]  Jeffrey D. Ullman,et al.  Induction variables in very high level languages , 1976, POPL.

[24]  Philip Wadler,et al.  Deforestation: Transforming Programs to Eliminate Trees , 1990, Theor. Comput. Sci..

[25]  Robert Paige,et al.  Program Derivation by Fixed Point Computation , 1989, Sci. Comput. Program..

[26]  Leonidas J. Guibas,et al.  Compilation and delayed evaluation in APL , 1978, POPL.

[27]  Nicholas J. Higham,et al.  INVERSE PROBLEMS NEWSLETTER , 1991 .

[28]  John Darlington,et al.  A Transformation System for Developing Recursive Programs , 1977, J. ACM.

[29]  Yanhong A. Liu,et al.  Efficiency by Incrementalization: An Introduction , 2000, High. Order Symb. Comput..

[30]  Donald E. Knuth,et al.  An empirical study of FORTRAN programs , 1971, Softw. Pract. Exp..

[31]  Keshav Pingali,et al.  Synthesizing Transformations for Locality Enhancement of Imperfectly-Nested Loop Nests , 2001, International Journal of Parallel Programming.

[32]  Vicki H. Allan,et al.  Software pipelining , 1995, CSUR.

[33]  Charles N. Fischer Proceedings of the ACM SIGPLAN 1996 conference on Programming language design and implementation , 1990, PLDI 1996.

[34]  William Pugh,et al.  Constraint-based array dependence analysis , 1998, TOPL.

[35]  Joe D. Warren,et al.  A hierarchical basis for reordering transformations , 1984, POPL '84.

[36]  R. Zabih,et al.  Individuating unknown objects by combining motion and stereo , 1994 .

[37]  Martin Griebl,et al.  Loop-Carried Code Placement , 2001, Euro-Par.

[38]  Shmuel Katz Program Optimization Using Invariants , 1978, IEEE Transactions on Software Engineering.

[39]  David Gries,et al.  A Note on a Standard Strategy for Developing Loop Invariants and Loops , 1982, Sci. Comput. Program..

[40]  Edsger W. Dijkstra,et al.  A Discipline of Programming , 1976 .

[41]  Paul Feautrier,et al.  Dataflow analysis of array and scalar references , 1991, International Journal of Parallel Programming.

[42]  John Randal Allen,et al.  Dependence analysis for subscripted variables and its application to program transformations , 1983 .

[43]  Yanhong A. Liu,et al.  Strengthening invariants for efficient computation , 2001, Sci. Comput. Program..

[44]  Amelia C. Fong Inductively computable constructs in very high level languages , 1979, POPL.

[45]  Yanhong A. Liu,et al.  Program optimization using indexed and recursive data structures , 2002, PEPM '02.

[46]  Eric Goubault,et al.  Static Analyses of the Precision of Floating-Point Operations , 2001, SAS.

[47]  A.L. Fisher,et al.  Design and performance of an optimizing SIMD compiler , 1990, [1990 Proceedings] The Third Symposium on the Frontiers of Massively Parallel Computation.

[48]  Daniel M. Yellin,et al.  INC: a language for incremental computations , 1988, PLDI '88.

[49]  Guy L. Steele,et al.  Fortran at ten gigaflops: the connection machine convolution compiler , 1991, PLDI '91.

[50]  Herman H. Goldstine,et al.  The Computer from Pascal to von Neumann , 1973 .

[51]  Ole Immanuel Franksen Mr. Babbage's Secret: The Tale of a Cypher and Apl , 1985 .

[52]  Thomas Reps,et al.  The Synthesizer Generator: A System for Constructing Language-Based Editors , 1988 .

[53]  Yanhong A. Liu,et al.  Principled strength reduction , 1997, Algorithmic Languages and Calculi.

[54]  Yanhong A. Liu,et al.  Dynamic Programming via Static Incrementalization , 2003, High. Order Symb. Comput..

[55]  William Pugh,et al.  The Omega test: A fast and practical integer programming algorithm for dependence analysis , 1991, Proceedings of the 1991 ACM/IEEE Conference on Supercomputing (Supercomputing '91).

[56]  Peter Thomas Highnam Systems and programming issues in the design and use of a SIMD linear array for image processing , 1991 .

[57]  Ken Kennedy,et al.  Scalar replacement in the presence of conditional control flow , 1994, Softw. Pract. Exp..

[58]  P. Feautrier Parametric integer programming , 1988 .

[59]  Wei-Ngan Chin Safe fusion of functional expressions , 1992, LFP '92.

[60]  Robert Paige,et al.  Symbolic Finite Differencing - Part I , 1990, ESOP.

[61]  Douglas R. Smith,et al.  KIDS: A Semiautomatic Program Development System , 1990, IEEE Trans. Software Eng..

[62]  Michael D. Ernst Serializing Parallel Programs by Removing Redundant Computation , 1994 .

[63]  Ben Wegbreit Goal-Directed Program Transformation , 1976, IEEE Trans. Software Eng..

[64]  Manfred Broy,et al.  Algebraic Methods for Program Construction: The Project CIP , 1984 .

[65]  Robert Paige,et al.  Finite Differencing of Computable Expressions , 1982, TOPL.

[66]  Friedrich L. Bauer,et al.  Formal Program Construction by Transformations-Computer-Aided, Intuition-Guided Programming , 1989, IEEE Trans. Software Eng..

[67]  William Pugh,et al.  Uniform techniques for loop optimization , 1991, ICS '91.