Optimal prefetching via data compression

A form of the competitive philosophy is applied to the problem of prefetching to develop an optimal universal prefetcher in terms of fault ratio, with particular applications to large-scale databases and hypertext systems. The algorithms are novel in that they are based on data compression techniques that are both theoretically optimal and good in practice. Intuitively, in order to compress data effectively, one has to be able to predict feature data well, and thus good data compressors should be able to predict well for purposes of prefetching. It is shown for powerful models such as Markov sources and mth order Markov sources that the page fault rates incurred by the prefetching algorithms presented are optimal in the limit for almost all sequences of page accesses.<<ETX>>

[1]  R. Gallager.  Information Theory and Reliable Communication , 1968 .

[2]  Abraham Lempel,et al.  On the Complexity of Finite Sequences , 1976, IEEE Trans. Inf. Theory.

[3]  Thomas M. Cover,et al.  Compound Bayes Predictors for Sequences with Apparent Markov Structure , 1977, IEEE Transactions on Systems, Man, and Cybernetics.

[4]  Abraham Lempel,et al.  Compression of individual sequences via variable-rate coding , 1978, IEEE Trans. Inf. Theory.

[5]  Glen G. Langdon,et al.  A note on the Ziv-Lempel model for compressing individual sequences , 1983, IEEE Trans. Inf. Theory.

[6]  Ian H. Witten,et al.  Data Compression Using Adaptive Coding and Partial String Matching , 1984, IEEE Trans. Commun..

[7]  Glen G. Langdon,et al.  An Introduction to Arithmetic Coding , 1984, IBM J. Res. Dev..

[8]  S. Natarajan,et al.  Large deviations, hypotheses testing, and source coding for finite Markov chains , 1985, IEEE Trans. Inf. Theory.

[9]  Robert E. Tarjan,et al.  Amortized efficiency of list update and paging rules , 1985, CACM.

[10]  James T. Brady,et al.  A Theory of Productivity in the Creative Process , 1986, IEEE Computer Graphics and Applications.

[11]  David Haussler,et al.  Occam's Razor , 1987, Inf. Process. Lett..

[12]  Ian H. Witten,et al.  Arithmetic coding for data compression , 1987, CACM.

[13]  David Haussler,et al.  Learnability and the Vapnik-Chervonenkis dimension , 1989, JACM.

[14]  Leonard Pitt,et al.  On the necessity of Occam algorithms , 1990, STOC '90.

[15]  Ian H. Witten,et al.  Text Compression , 1990, 125 Problems in Text Algorithms.

[16]  M. Luby,et al.  On ~ competitive algorithms for paging problems , 1991 .

[17]  Allan Borodin,et al.  Competitive paging with locality of reference , 1991, STOC '91.

[18]  Amos Fiat,et al.  Competitive Paging Algorithms , 1991, J. Algorithms.

[19]  Stanley B. Zdonik,et al.  Fido: A Cache That Learns to Fetch , 1991, VLDB.

[20]  Jeffrey Scott Vitter,et al.  Analysis of arithmetic coding for data compression , 1991, [1991] Proceedings. Data Compression Conference.

[21]  Anoop Gupta,et al.  Design and evaluation of a compiler algorithm for prefetching , 1992, ASPLOS V.

[22]  Jean-Loup Baer,et al.  Reducing memory latency via non-blocking and prefetching caches , 1992, ASPLOS V.

[23]  Neri Merhav,et al.  Universal prediction of individual sequences , 1992, IEEE Trans. Inf. Theory.

[24]  Anne Rogers,et al.  Software support for speculative loads , 1992, ASPLOS V.

[25]  Sandy Irani,et al.  Strongly competitive algorithms for paging with locality of reference , 1992, SODA '92.

[26]  Anna R. Karlin,et al.  Markov paging , 1992, Proceedings., 33rd Annual Symposium on Foundations of Computer Science.

[27]  Yali Amit,et al.  Large deviations for coding Markov chains and Gibbs random fields , 1993, IEEE Trans. Inf. Theory.