Computational power of pipelined memory hierarchies

We define a model of computation, called the <i>Pipelined Hierarchical Random Access Machine</i> with <i>access function a</i> (<i>x</i>), denoted the <i>a</i>(<i>x</i>)-PH-RAM. In this model, a processor interacts with a memory which can accept requests at a constant rate and satisfy each of the requests to the location <i>x</i> within a(<i>x</i>) units of time. We investigate memory management strategies that lead to time efficient implementations of arbitrary computations on a PH-RAM. We begin by developing the so called <i>pipeline d decomposition-tree</i>memory management strategy, which can be tuned to the memory access function. Specifically, for a linear or sublinear access function <i>a</i>(<i>x</i>), w e define the concept of <i>latency-hiding depth d<subscrpt>a</subscrpt></i>(<i>x</i>) and show ho w an y computation of <i>N</i> operations can be implemented on an <i>a</i>(<i>x</i>)-PH-RAM in time <i>T</i>(<i>N</i>) = <i>&Ogr;</i>(<i>Nd<subscrpt>a</subscrpt></i>(<i>N</i>)). In particular, <i>T</i>(<i>N</i>) = <i>&Ogr;</i>(<i>N</i> log <i>N</i>) if <i>a</i>(<i>x</i>) = <i>&Ogr;</i>(<i>x</i>), <i>T</i>(<i>N</i>) = <i>&Ogr;</i>(<i>N</i> log log <i>N</i>) if <i>a</i>(<i>x</i>) = <i>&Ogr;</i>(<i>x^Β</i>) with 0 < <i>Β</i> < 1, and <i>T</i>(<i>N</i>) = O(<i>N</i> log* <i>N</i>) if <i>a</i>(<i>x</i>) = <i>&Ogr;</i>(log <i>x</i>). We develop lower bound techniques that allow to establish existential lower bounds on PH-RAMs. In particular, we exhibit computations for which <i>T</i>(<i>N</i>) = &OHgr;(<i>N</i>log <i>N</i>/ log log <i>N</i>) when <i>a</i>(<i>x</i>) = &OHgr;(<i>x</i>), <i>T</i>(<i>N</i>) = &OHgr;(<i>N</i>log log<i>N</i>) when <i>a</i>(<i>x</i>) = &OHgr;(<i>x^Β</i>) with 0 < <i>Β</i> < 1, and <i>T</i>(<i>N</i>) = &OHgr;(<i>N</i> log* <i>N</i>) when <i>a</i>(<i>x</i>) = &OHgr;(log <i>x</i>). The stated lower bounds show that the pipelined decomposition-tree strategy is existentially optimal for the latter case but indicates the potential for a modest, <i>&Ogr;</i>(log log <i>N</i>) improvement for linear access functions. To realize this potential, a <i>superpipelined</i> decomposition-tree memory manager is proposed, which achieves <i>T</i>(<i>N</i>) = <i>&Ogr;</i>(<i>N</i> log <i>N</i>/log log <i>N</i>). The pipelined decomposition-tree strategy can also be tuned to the computation, in order to exploit its <i>temporal locality</i> as characterized by the width parameters [9]. When the latter are suitably bounded, then <i>T</i>(<i>N</i>) = <i>&Ogr;</i>(<i>N</i>) on any PH-RAM with linear or sublinear access function. Finally, we discuss how performance could benefit from <i>parallelism</i> in the data-dependence dag of the computation or from architectural enhancements, such as <i>block-transfer</i> primitives, and formulate various questions that deserve further investigation.