Compression challenges in large scale PDE solvers

Solvers for partial differential equations (PDE) are one of the cornerstones of computational science. For large problems, they involve huge amounts of data that needs to be stored and transmitted on all levels of the memory hierarchy. Often, bandwidth is the limiting factor due to relatively small arithmetic intensity, and increasingly so due to the growing disparity between computing power and bandwidth. Consequently, data compression techniques have been investigated and tailored towards the specific requirements of PDE solvers during the last decades. This paper surveys data compression challenges and corresponding solution approaches for PDE problems, covering all levels of the memory hierarchy from mass storage up to main memory. Exemplarily, we illustrate concepts at particular methods, and give references to alternatives.

[1]  Rajeev Balasubramonian,et al.  MemZip: Exploring unconventional benefits from memory compression , 2014, 2014 IEEE 20th International Symposium on High Performance Computer Architecture (HPCA).

[2]  Karl Kunisch,et al.  Lossy Compression in Optimal Control of Cardiac Defibrillation , 2014, J. Sci. Comput..

[3]  Michel Schanen,et al.  Large-scale lossy data compression based on an a priori error estimator in a Spectral Element Code , 2016 .

[4]  Sebastian Götschel,et al.  Lossy data compression reduces communication time in hybrid time-parallel integrators , 2018, Comput. Vis. Sci..

[5]  Franck Cappello,et al.  Error-Controlled Lossy Compression Optimized for High Compression Ratios of Scientific Datasets , 2018, 2018 IEEE International Conference on Big Data (Big Data).

[6]  W. Hackbusch A Sparse Matrix Arithmetic Based on $\Cal H$-Matrices. Part I: Introduction to ${\Cal H}$-Matrices , 1999, Computing.

[7]  Rolf Rannacher,et al.  Adaptive Finite Element Methods for Optimal Control of Partial Differential Equations: Basic Concept , 2000, SIAM J. Control. Optim..

[8]  Peter Lindstrom,et al.  Fixed-Rate Compressed Floating-Point Arrays , 2014, IEEE Transactions on Visualization and Computer Graphics.

[9]  Franck Cappello,et al.  Improving performance of iterative methods by lossy checkponting , 2018, HPDC.

[10]  Peter Lindstrom,et al.  Error Analysis of ZFP Compression for Floating-Point Data , 2018, SIAM J. Sci. Comput..

[11]  Martin Burtscher,et al.  SPDP: An Automatically Synthesized Lossless Compression Algorithm for Floating-Point Data , 2018, 2018 Data Compression Conference.

[12]  Tamara G. Kolda,et al.  Parallel Tensor Compression for Large-Scale Scientific Data , 2015, 2016 IEEE International Parallel and Distributed Processing Symposium (IPDPS).

[13]  Rob Stevenson,et al.  Locally Supported, Piecewise Polynomial Biorthogonal Wavelets on Nonuniform Meshes , 2000 .

[14]  Layne T. Watson,et al.  Mixed-Precision Preconditioners in Parallel Domain Decomposition Solvers , 2008 .

[15]  Peter Lindstrom,et al.  TTHRESH: Tensor Compression for Multidimensional Visual Data , 2018, IEEE Transactions on Visualization and Computer Graphics.

[16]  Christoph von Tycowicz,et al.  Reducing Memory Requirements in Scientific Computing and Optimal Control , 2013 .

[17]  A. Toselli Domain Decomposition Methods , 2004 .

[18]  Peter Deuflhard,et al.  Concepts of an adaptive hierarchical finite element code , 1989, IMPACT Comput. Sci. Eng..

[19]  Martin Isenburg,et al.  Fast and Efficient Compression of Floating-Point Data , 2006, IEEE Transactions on Visualization and Computer Graphics.

[20]  Franck Cappello,et al.  Exploring the feasibility of lossy compression for PDE simulations , 2019, Int. J. High Perform. Comput. Appl..

[21]  Allison H. Baker,et al.  A Statistical Analysis of Compressed Climate Model Data , 2018 .

[22]  J. Strikwerda Finite Difference Schemes and Partial Differential Equations , 1989 .

[23]  Onur Mutlu,et al.  Linearly compressed pages: A low-complexity, low-latency main memory compression framework , 2013, 2013 46th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO).

[24]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[25]  Jack J. Dongarra,et al.  GPU-Accelerated Asynchronous Error Correction for Mixed Precision Iterative Refinement , 2011, Euro-Par.

[26]  G. Nigel Martin,et al.  * Range encoding: an algorithm for removing redundancy from a digitised message , 1979 .

[27]  Idoia Ochoa,et al.  QualComp: a new lossy compressor for quality scores based on rate distortion theory , 2013, BMC Bioinformatics.

[28]  Scott B. Baden,et al.  A Method of Adaptive Coarsening for Compressing Scientific Datasets , 2006, PARA.

[29]  Scott Klasky,et al.  Multilevel techniques for compression and reduction of scientific data—the univariate case , 2018, Comput. Vis. Sci..

[30]  Wolfgang Dahmen,et al.  Compression Techniques for Boundary Integral Equations - Asymptotically Optimal Complexity Estimates , 2006, SIAM J. Numer. Anal..

[31]  Scott B. Baden,et al.  An Adaptive Sub-sampling Method for In-memory Compression of Scientific Data , 2009, 2009 Data Compression Conference.

[32]  G. M. Ostrovskii,et al.  Automatic computation of derivatives with the use of the multilevel differentiating technique—1. Algorithmic basis , 1985 .

[33]  Samuel Williams,et al.  Roofline: an insightful visual performance model for multicore architectures , 2009, CACM.

[34]  Valerio Pascucci,et al.  A Study of the Trade-off Between Reducing Precision and Reducing Resolution for Data Analysis and Visualization , 2019, IEEE Transactions on Visualization and Computer Graphics.

[35]  Florian Wende,et al.  Impact of mixed precision and storage layout on additive Schwarz smoothers , 2021, Numer. Linear Algebra Appl..

[36]  Robert Latham,et al.  ISABELA for effective in situ compression of scientific data , 2013, Concurr. Comput. Pract. Exp..

[37]  Andreas Griewank,et al.  Evaluating derivatives - principles and techniques of algorithmic differentiation, Second Edition , 2000, Frontiers in applied mathematics.

[38]  Andreas Griewank,et al.  Achieving logarithmic growth of temporal and spatial complexity in reverse automatic differentiation , 1992 .

[39]  Francesco De Simone,et al.  Evaluating lossy data compression on climate simulation data within a large ensemble , 2016, Geoscientific Model Development.

[40]  Céline Hudelot,et al.  3D Mesh Compression , 2015, ACM Comput. Surv..

[41]  Tamara G. Kolda,et al.  TuckerMPI , 2019, ACM Trans. Math. Softw..

[42]  Jesús Carretero,et al.  Adaptive-Compi: Enhancing Mpi-Based Applications’ Performance and Scalability by using Adaptive Compression , 2011, Int. J. High Perform. Comput. Appl..

[43]  Martin J. Gander,et al.  50 Years of Time Parallel Time Integration , 2015 .

[44]  Thomas Maugey,et al.  A Geometry-aware Framework for Compressing 3D Mesh Textures , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[45]  Sebastian Götschel,et al.  Adaptive Lossy Trajectory Compression for Optimal Control of Parabolic PDEs , 2015 .

[46]  Franck Cappello,et al.  Significantly Improving Lossy Compression for Scientific Data Sets Based on Multidimensional Prediction and Error-Controlled Quantization , 2017, 2017 IEEE International Parallel and Distributed Processing Symposium (IPDPS).

[47]  Christoph von Tycowicz,et al.  Context‐Based Coding of Adaptive Multiresolution Meshes , 2011, Comput. Graph. Forum.

[48]  Andrea Toselli,et al.  Domain decomposition methods : algorithms and theory , 2005 .

[49]  Peter Lindstrom,et al.  Reducing disk storage of full-3D seismic waveform tomography (F3DT) through lossy online compression , 2016, Comput. Geosci..

[50]  Martin Burtscher,et al.  FPC: A High-Speed Compressor for Double-Precision Floating-Point Data , 2009, IEEE Transactions on Computers.

[51]  HackbuschW. A sparse matrix arithmetic based on H-matrices. Part I , 1999 .

[52]  P. Solin,et al.  On Scientific Data and Image Compression Based on Adaptive Higher-Order FEM , 2009 .

[53]  Julian M. Kunkel,et al.  Data Compression for Climate Data , 2016, Supercomput. Front. Innov..

[54]  Martin Weiser,et al.  On goal-oriented adaptivity for elliptic optimal control problems , 2013, Optim. Methods Softw..

[55]  Francesc Alted,et al.  Why Modern CPUs Are Starving and What Can Be Done about It , 2010, Computing in Science & Engineering.

[56]  N. Ahmed,et al.  Discrete Cosine Transform , 1996 .

[57]  Karl Kunisch,et al.  Numerical solution for optimal control of the reaction-diffusion equations in cardiac electrophysiology , 2011, Comput. Optim. Appl..

[58]  John N. Shadid,et al.  Towards efficient backward-in-time adjoint computations using data compression techniques , 2015 .

[59]  Peter Deuflhard,et al.  Adaptive Numerical Solution of PDEs , 2012 .

[60]  Scott Klasky,et al.  Multilevel Techniques for Compression and Reduction of Scientific Data - The Multivariate Case , 2019, SIAM J. Sci. Comput..

[61]  Scott A. Mahlke,et al.  Concise loads and stores: The case for an asymmetric compute-memory architecture for approximation , 2016, 2016 49th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO).

[62]  Tong Liu,et al.  Understanding and Modeling Lossy Compression Schemes on HPC Scientific Data , 2018, 2018 IEEE International Parallel and Distributed Processing Symposium (IPDPS).

[63]  Sebastian Götschel,et al.  Lossy compression for PDE-constrained optimization: adaptive error control , 2015, Comput. Optim. Appl..

[64]  Marc Antonini,et al.  HexaShrink, an exact scalable framework for hexahedral meshes with attributes and discontinuities: multiresolution rendering and storage of geoscience models , 2019, ArXiv.

[65]  Jianqin Zhou,et al.  On discrete cosine transform , 2011, ArXiv.

[66]  Nira Dyn,et al.  Adaptive Thinning for Terrain Modelling and Image Compression , 2005, Advances in Multiresolution for Geometric Modelling.

[67]  Sebastian Götschel,et al.  State Trajectory Compression for Optimal Control with Parabolic PDEs , 2012, SIAM J. Sci. Comput..

[68]  Franck Cappello,et al.  Toward an Optimal Online Checkpoint Solution under a Two-Level HPC Checkpoint Model , 2017, IEEE Transactions on Parallel and Distributed Systems.

[69]  Long Chen FINITE ELEMENT METHOD , 2013 .

[70]  Jens Lang,et al.  Konrad-zuse-zentrum F ¨ Ur Informationstechnik Berlin Adaptivity in Space and Time for Reaction-diffusion Systems in Electrocardiology Adaptivity in Space and Time for Reaction-diffusion Systems in Electrocardiology , 2022 .

[71]  Jack J. Dongarra,et al.  Exascale computing and big data , 2015, Commun. ACM.

[72]  Peter Bauer,et al.  New Methods for Data Storage of Model Output from Ensemble Simulations , 2019, Monthly Weather Review.

[73]  Franck Cappello,et al.  Fast Error-Bounded Lossy HPC Data Compression with SZ , 2016, 2016 IEEE International Parallel and Distributed Processing Symposium (IPDPS).

[74]  Erwin Laure,et al.  Lossy Data Compression Effects on Wall-bounded Turbulence: Bounds on Data Reduction , 2018 .

[75]  Julien Langou,et al.  Accelerating scientific computations with mixed precision algorithms , 2008, Comput. Phys. Commun..

[76]  Rolf Rannacher,et al.  On the adaptive discretization of PDE-based optimization problems , 2006 .

[77]  Rolf Rannacher,et al.  A Feed-Back Approach to Error Control in Finite Element Methods: Basic Analysis and Examples , 1996 .

[78]  Alfio Borzì,et al.  Computational Optimization of Systems Governed by Partial Differential Equations , 2012, Computational science and engineering.

[79]  J. Oden,et al.  Goal-oriented error estimation and adaptivity for the finite element method , 2001 .

[80]  Michael L. Minion,et al.  TOWARD AN EFFICIENT PARALLEL IN TIME METHOD FOR PARTIAL DIFFERENTIAL EQUATIONS , 2012 .

[81]  Sally A. McKee,et al.  Reflections on the memory wall , 2004, CF '04.

[82]  Guillaume Lavoué,et al.  Progressive compression of arbitrary textured meshes , 2016, Comput. Graph. Forum.

[83]  James Demmel,et al.  IEEE Standard for Floating-Point Arithmetic , 2008 .

[84]  Wim Sweldens,et al.  The lifting scheme: a construction of second generation wavelets , 1998 .

[85]  John T. Daly,et al.  A higher order estimate of the optimum checkpoint interval for restart dumps , 2006, Future Gener. Comput. Syst..

[86]  John W. Young,et al.  A first order approximation to the optimum checkpoint interval , 1974, CACM.

[87]  Clemens-August Thole,et al.  Predictive Principal Component Analysis as a Data Compression Core in a Simulation Data Management System , 2015, 2015 Data Compression Conference.

[88]  Mauricio Hanzich,et al.  Wavefield compression for adjoint methods in full-waveform inversion , 2016 .

[89]  Franck Cappello,et al.  Optimizing Lossy Compression Rate-Distortion from Automatic Online Selection between SZ and ZFP , 2018, IEEE Transactions on Parallel and Distributed Systems.

[90]  Moinuddin K. Qureshi,et al.  DICE: Compressing DRAM caches for bandwidth and capacity , 2017, 2017 ACM/IEEE 44th Annual International Symposium on Computer Architecture (ISCA).

[91]  William A. Pearlman,et al.  A new, fast, and efficient image codec based on set partitioning in hierarchical trees , 1996, IEEE Trans. Circuits Syst. Video Technol..

[92]  Franck Cappello,et al.  Use cases of lossy compression for floating-point data in scientific data sets , 2019, Int. J. High Perform. Comput. Appl..

[93]  Jeffrey S. Vetter,et al.  A Survey Of Architectural Approaches for Data Compression in Cache and Main Memory Systems , 2016 .

[94]  J. J. Moré,et al.  Quasi-Newton Methods, Motivation and Theory , 1974 .

[95]  Enrique S. Quintana-Ortí,et al.  Adaptive precision in block‐Jacobi preconditioning for iterative sparse linear system solvers , 2019, Concurr. Comput. Pract. Exp..

[96]  Jeremy Iverson,et al.  Fast and Effective Lossy Compression Algorithms for Scientific Datasets , 2012, Euro-Par.