Experimentally Evaluating In-Place Delta Reconstruction

In-place reconstruction of delta compressed data allows information on devices with limited storage capability to be updated efficiently over low-bandwidth channels. Delta compression encodes a version of data compactly as a small set of changes from a previous version. Transmitting updates to data as delta versions saves both time and bandwidth. In-place reconstruction rebuilds the new version of the data in the storage or memory space the current version occupies – no additional scratch space is needed. By combining these technologies, we support large-scale, highly-mobile applications on inexpensive hardware. In this paper, we present an experimental study of in-place reconstruction algorithms, characterizing the performance of several in-place reconstruction algorithms. We take a data-driven approach to determine important performance features, classifying files distributed on the Internet based on their in-place properties, and exploring the scaling relationship between files and data structures used by in-place algorithms. We conclude that in-place algorithms are I/O bound and that the performance of algorithms is most sensitive to the size of inputs and outputs, rather than asymptotic bounds.

[1]  Peter Weiner,et al.  Linear Pattern Matching Algorithms , 1973, SWAT.

[2]  Marc J. Rochkind,et al.  The source code control system , 1975, IEEE Transactions on Software Engineering.

[3]  Edward M. McCreight,et al.  A Space-Economical Suffix Tree Construction Algorithm , 1976, JACM.

[4]  Guy M. Lohman,et al.  Differential files: their application to the maintenance of large databases , 1976, TODS.

[5]  Walter F. Tichy,et al.  The string-to-string correction problem with block moves , 1984, TOCS.

[6]  Eugene W. Myers,et al.  A file comparison program , 1985, Softw. Pract. Exp..

[7]  Walter F. Tichy,et al.  Rcs — a system for version control , 1985, Softw. Pract. Exp..

[8]  Christoph Reichenberger,et al.  Delta storage for arbitrary non-text files , 1991, SCM '91.

[9]  Paul Mackerras,et al.  The rsync algorithm , 1996 .

[10]  Walter F. Tichy,et al.  An Empirical Study of Delta Algorithms , 1996, SCM.

[11]  Darrell D. E. Long,et al.  Efficient distributed backup and restore with delta compression , 1997 .

[12]  Potential Benefits of Delta Encoding and Data Compression for HTTP , 1997, SIGCOMM.

[13]  Hector Garcia-Molina,et al.  Meaningful change detection in structured data , 1997, SIGMOD '97.

[14]  Fred Douglis,et al.  Optimistic deltas for WWW latency reduction , 1997 .

[15]  Anja Feldmann,et al.  Potential benefits of delta encoding and data compression for HTTP , 1997, SIGCOMM '97.

[16]  Randal C. Burns,et al.  In-place reconstruction of delta compressed files , 1998, PODC '98.

[17]  Paul N. Hilfinger,et al.  PRCS: The Project Revision Control System , 1998, SCM.

[18]  Walter F. Tichy,et al.  Delta algorithms: an empirical analysis , 1998, TSEM.

[19]  Mun Choon Chan,et al.  Cache-based compaction: a new technique for optimizing Web transfer , 1999, IEEE INFOCOM '99. Conference on Computer Communications. Proceedings. Eighteenth Annual Joint Conference of the IEEE Computer and Communications Societies. The Future is Now (Cat. No.99CH36320).

[20]  Ronald Fagin,et al.  Compactly encoding unstructured inputs with differential compression , 2002, JACM.