How Will Astronomy Archives Survive the Data Tsunami?

Astronomy is already awash with data: currently 1 PB of public data is electronically accessible, and this volume is growing at 0.5 PB per year. The availability of this data has already transformed research in astronomy, and the STScI now reports that more papers are published with archived data sets than with newly acquired data. This growth in data size and anticipated usage will accelerate in the coming few years as new projects such as the LSST, ALMA, and SKA move into operation. These new projects will use much larger arrays of telescopes and detectors or much higher data acquisition rates than are now used. Projections indicate that by 2020, more than 60 PB of archived data will be accessible to astronomers.

[1]  P. Napier National Radio Astronomy Observatory , 1992 .

[2]  Paul F. Dubois Software Carpentry , 2006, Computing in Science & Engineering.

[3]  Ivan Zolotukhin,et al.  The True Bottleneck of Modern Scientific Computing in Astronomy , 2010, 1012.4119.

[4]  G. Bruce Berriman,et al.  The Application of Cloud Computing to Astronomy: A Study of Cost and Performance , 2010, 2010 Sixth IEEE International Conference on e-Science Workshops.

[5]  Brian Major,et al.  CANFAR: the Canadian Advanced Network for Astronomical Research , 2010, Astronomical Telescopes + Instrumentation.

[6]  S. Deustua,et al.  2010 Space Telescope Science Institute Calibration Workshop - Hubble after SM4. Preparing JWST , 2010 .

[7]  Z. Merali Computational science: ...Error , 2010, Nature.

[8]  Sebastien Fabbro,et al.  The Canadian Advanced Network For Astronomical Research , 2011 .

[9]  Amr H. Hassan,et al.  Astrophysical Supercomputing with GPUs: Critical Decisions for Early Adopters* , 2010, Publications of the Astronomical Society of Australia.

[10]  Ewa Deelman,et al.  Ten years of software sustainability at the Infrared Processing and Analysis Center , 2011, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[11]  Shantenu Jha,et al.  Using the TeraGrid to teach scientific computing , 2011 .

[12]  Ewa Deelman,et al.  The application of cloud computing to scientific workflows: a study of cost and performance , 2013, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.