Adaptive on-the-fly compression

We present a system called the adaptive compression environment (ACE) that automatically and transparently applies compression (on-the-fly) to a communication stream to improve network transfer performance. ACE uses a series of estimation techniques to make short-term forecasts of compressed and uncompressed transfer time at the 32 Kb block level. ACE considers underlying networking technology, available resource performance, and data characteristics as part of its estimations to determine which compression algorithm to apply (if any). Our empirical evaluation shows that, on average, ACE improves transfer performance given changing network types and performance characteristics by 8 to 93 percent over using the popular compression techniques that we studied (Bzip, Zlib, LZO, and no compression) alone.

[1]  Emmanuel Jeannot,et al.  Adaptive online data compression , 2002, Proceedings 11th IEEE International Symposium on High Performance Distributed Computing.

[2]  Chandra Krintz,et al.  Reducing delay with dynamic selection of compression formats , 2001, Proceedings 10th IEEE International Symposium on High Performance Distributed Computing.

[3]  Thomas R. Gross,et al.  ReMoS: A Resource Monitoring System for Network-Aware Applications , 1997 .

[4]  Mats Björkman,et al.  Adaptive end-to-end compression for variable-bandwidth communication , 1999, Comput. Networks.

[5]  Ian Foster,et al.  The Grid 2 - Blueprint for a New Computing Infrastructure, Second Edition , 1998, The Grid 2, 2nd Edition.

[6]  Ami Marowka,et al.  The GRID: Blueprint for a New Computing Infrastructure , 2000, Parallel Distributed Comput. Pract..

[7]  Chandra Krintz,et al.  ACE: a resource-aware adaptive compression environment , 2003, Proceedings ITCC 2003. International Conference on Information Technology: Coding and Computing.

[8]  Richard Wolski,et al.  The network weather service: a distributed resource performance forecasting service for metacomputing , 1999, Future Gener. Comput. Syst..

[9]  Richard Wolski,et al.  Experiences with predicting resource performance on-line in computational grid settings , 2003, PERV.

[10]  Francine Berman,et al.  Overview of the Book: Grid Computing – Making the Global Infrastructure a Reality , 2003 .

[11]  James M. Stichnoth,et al.  Practicing JUDO: Java under dynamic optimizations , 2000, PLDI '00.

[12]  Richard Wolski,et al.  Dynamically forecasting network performance using the Network Weather Service , 1998, Cluster Computing.

[13]  Ali-Reza Adl-Tabatabai,et al.  Fast, effective code generation in a just-in-time Java compiler , 1998, PLDI.

[14]  William Pugh,et al.  Compressing Java class files , 1999, PLDI '99.

[15]  Amar Mukherjee,et al.  Network conscious text compression system (NCTCSys) , 2001, Proceedings International Conference on Information Technology: Coding and Computing.