Tailored data compression using stream partitioning and prediction: application to Gaia

Gaia is the most ambitious space astrometry mission currently envisaged and it will be a technological challenge in all its aspects. Here we describe a proposal for the data compression system of Gaia, specifically designed for this mission but based on concepts that can be applied to other missions and systems as well. Realistic simulations have been performed with our Telemetry CODEC software, which performs a stream partitioning and pre-compression to the science data. In this way, standard compressors such as bzip2 or szip boost their performance and decrease their processing requirements when applied to such pre-processed data. These simulations have shown that a lossless compression factor of 3 can be achieved, whereas standard compression systems were unable to reach a factor of 2.

[1]  M. Perryman,et al.  The Three-Dimensional Universe with Gaia , 2005 .

[2]  Leis J.M,et al.  Proceedings of the Symposium , 1997 .

[3]  Wilfried Philips,et al.  Exploiting interframe redundancies in the lossless compression of 3D medical images , 2000, Proceedings DCC 2000. Data Compression Conference.

[4]  H. Wechsler,et al.  Micro-Doppler effect in radar: phenomenon, model, and simulation study , 2006, IEEE Transactions on Aerospace and Electronic Systems.

[5]  David Salomon,et al.  Data Compression: The Complete Reference , 2006 .

[6]  J.D. Gibson,et al.  Adaptive prediction in speech differential encoding systems , 1980, Proceedings of the IEEE.

[7]  E. Garcia-Berro,et al.  High-performance payload data handling system for Gaia , 2006, IEEE Transactions on Aerospace and Electronic Systems.

[8]  Luis J. Goicoechea,et al.  Highlights of Spanish Astrophysics V , 2001 .

[9]  M. G. Lattanzi,et al.  GAIA: Composition, formation and evolution of the Galaxy , 2001, astro-ph/0101235.

[10]  G.G. Langdon,et al.  On prediction error coding methods for lossless image compression , 1997, Conference Record of the Thirty-First Asilomar Conference on Signals, Systems and Computers (Cat. No.97CB36136).