Transformational programming: applications to algorithms and systems

Ten years ago Cheatham and Wegbreit [4] proposed atransformational program development methodology based on notionsof top-down stepwise program refinement first expressed by Dijkstra[10] and Wirth [45]. A schema describing the process of thismethodology is given in fig. 1. To develop a program bytransformation, we first specify the program in as high a level ofabstraction and as great a degree of clarity as our programminglanguage admits. This high level problem statement program P isproved correct semimechanically according to some standard approach(see Flovd and Hoare [15, 21]), Next, using an interactive systemequipped with a library of encoded transformations, each of whichmaps a correct program into another equivalent program, we selectand apply transformations one at a time to successive versions ofthe program until we obtain a concrete, low level, effecientimplementation version P'. The goals of transformationalprogramming are to reduce programming labor, improve programreliability, and upgrade program performance. In order for labor tobe reduced, the effort required to obtain P, prove it correct, andderive P' by transformation should be less than the effort requiredto code P from scratch, and also to debug it. Program reliabilitywill be improved if P can be certified correct, and if eachtransformation preserves program meaning. Finally, programperformance will be upgraded if transformations are directedtowards increased efficiency. Experimental transformational systems that emphasize one or moreaspects of the methodology outlined above have been implemented byCheatham [5], Darlington [3], Loveman [27], Standish [41], Feather[14] Huet and Lang [11], and others. However, all of these systemsfall short of the goals, because of a number of reasons thatinclude, 1 inability to mechanize the checking of transformationapplicability conditions 2 reliance on large, unmanageable collections of low leveltransformations, and long arduous derivation sequences 3 dependency on transformations whose potential for improvingprogram performance is unpredictable 4 use of source languages insufficiently high level toaccommodate perspicuous initial program specifications and powerfulalgorithmic transformations Yet, convincing evidence that this new methodology will succeedhas come from recent advances in verification, programtransformations, syntax directed editting systems, and high levellanguages. These advances, discussed below, represent partialsolution to the problems stated above, and could eventually beintegrated into a single system 1 The transformational approach to verification was pioneered byGerhart [19] and strengthened by the results of Schwartz [39],Scherlis [36], Broy et al [2], Koenig and Paige [26.31] Blaustein[1], and others. Due mainly to improved technology for themechanization of proofs of enabling conditions that justifyapplication of transformations, this approach is now at a pointwhere it can be effectively used in a system. Such mechanizationdepends strongly on program analysis, and, in particular, onreanalyses after a program is modified. Attribute grammars [24]have been shown to be especially useful in facilitating programanalysis [23]. Moreover, Reps [34] has discovered algorithm thatreevaluates attributes in optimal time after a program undergoessyntax directed editing changes (as are allowed on the CornellSynthesizer [43]). He has implemented his algorithm recently, andhas reported initial success 2 There are encouraging indications that a transformationalsystem can be made to depend mainly on a small but powerfulcollection of transformations applied top-down fashion to programsspecified at various levels of abstraction from logic down toassembler. We envision such a system as a fairly conventionalsemiautomatic compiler which classes of transformations areselected semimechanically in a predetermined order, and arejustified by predicates supplied mechanically but provedsemimanually. Of particular importance is nondeterminism removalwhich has formulated by Sharir [40] could lead to a technique forturning naive, nondeterministic programs into deterministicprograms with emergent strategies. Such programs could then betransformed automatically by finite differencing [13, 16, 17, 18,29, 30, 31] and jamming [28, 31, 20] (which we have implemented)into programs whose data access paths are fully determined. TheSETL optimizer could improve these programs further byautomatically choosing efficient data structure representations andaggregations 3 Of fundamental importance to the transformations justmentioned is the fact that they can be associated with speeduppredictions Fong and Ullman [16] were the first to characterize animportant class of algorithmic differencing transformations interms of accurate asymptotic speedup predictions, eg, they gaveconditions under which repeated calculation of a set former {x ins|k(x)} could be computed on O(#s) + cost(k) steps. By consideringstronger conditions and special cases for the boolean valuedsubpart k, Paige [31] later gave sharper speedup predictions (eg,either O(1) steps for each encounter of the set former or acumulative cost of O(#s) steps for every encounter) associated withanother differencing method. Both Morgenstern [28] and Paige [31]prove constant factor improvements due to their jammingtransformations (implemented by Morgenstern for the improvement offile processing, and by Paige for the optimization of programs).Constant factor speedup has also been observed for data structureselection by the method of basings but a supporting analytic studyhas not been presented [8, 37] 4 Essential to the whole transformational process is a widespectrum programming language (or set of languages) that canexpress a program at every stage of development from the initialabstract specification down to its concrete implementationrealization. Since transformations applied to programs written atthe highest levels of abstraction are likely to make the mostfundamental algorithmic changes, it is important to stress abstractfeatures in our language. In addition to supportingtransformations, the highest level language dictions should supportlucid initial specifications, verification, and even programanalysts. Of special importance is SETL [38, 9], because itsabstract set theoretic dictions can model data structures andalgorithms easily, because its philosophy of avoiding hidden asymptotic costs facilitates program analysis, because its semanticsconforms to finite set theory and can accommodate a set theoreticprogram logic, and because it is wide spectrum. As is evidenced bythe work of Schwartz, Fong, Paige, and Sharir, SETL is also a richmedium for transformation.