Astronomical data formats: What we have and how we got here

Despite almost all being acquired as photons, astronomical data from different instruments and at different stages in its life may exist in different formats to serve different purposes. Beyond the data itself, descriptive information is associated with it as metadata, either included in the data format or in a larger multi-format data structure. Those formats may be used for the acquisition, processing, exchange, and archiving of data. It has been useful to use similar formats, or even a single standard to ease interaction with data in its various stages using familiar tools. Knowledge of the evolution and advantages of present standards is useful before we discuss the future of how astronomical data is formatted. The evolution of the use of world coordinates in FITS is presented as an example.

[1]  Ole Streicher,et al.  Learning from FITS: Limitations in use in modern astronomical research , 2015, Astron. Comput..

[2]  Michael J. Kurtz,et al.  The NASA Astrophysics Data System: Overview , 2000, astro-ph/0002104.

[3]  Mark R. Calabretta,et al.  Representations of distortions in FITS world coordinate systems , 2004 .

[4]  M. Servillat,et al.  Improved Photometry for the DASCH Pipeline , 2013, 1304.7504.

[5]  Nelson R. Zarate,et al.  A FITS Image Extension Kernel for IRAF , 1996 .

[6]  William Pence,et al.  Fv: Interactive FITS file editor , 2012 .

[7]  Douglas J. Mink.  WCSTools: an Image Astrometry Toolkit , 1998 .

[8]  Barry M. Lasker,et al.  The guide star catalog , 1987 .

[9]  Douglas J. Mink,et al.  DASCH to Measure (and preserve) the Harvard Plates: Opening the ̃100-year Time Domain Astronomy Window , 2009 .

[10]  David Taubman,et al.  Astronomical imagery: Considerations for a contemporary approach with JPEG2000 , 2014 .

[11]  M. R. Calabretta.  Wcslib and Pgsbox , 2011 .

[12]  J. W. Palmer,et al.  AIPS -- Astronomical Image Processing System , 1996 .

[13]  Brian McLean,et al.  The Guide Star Catalog. II. Photometric and Astrometric Models and Solutions , 1990 .

[14]  Mark Taylor,et al.  TOPCAT: Tool for OPerations on Catalogues And Tables , 2011 .

[15]  Rodney F. Warren-Smith,et al.  AST: World Coordinate Systems in Astronomy , 2014 .

[16]  C. Beichman,et al.  Infrared astronomical satellite (IRAS) catalogs and atlases. Volume 1: Explanatory supplement , 1988 .

[17]  D. Wells,et al.  Fits: a flexible image transport system , 1981 .

[18]  R. H. Harten,et al.  The FITS Tape Formats: Flexible Image Transport Systems , 1980, Photonics West - Lasers and Applications in Science and Engineering.

[19]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[20]  C. Keller.  NATIONAL OPTICAL ASTRONOMY OBSERVATORIES , 1999 .

[21]  Mark R. Calabretta,et al.  Representations of world coordinates in FITS , 2002, astro-ph/0207407.

[22]  Tim Jenness.  Reimplementing the Hierarchical Data System using HDF5 , 2015 .

[23]  J. D. Ponz,et al.  The FITS image extension , 1994 .

[24]  Doug Tody,et al.  The Simple Spectral Access protocol , 2004, SPIE Astronomical Telescopes + Instrumentation.

[25]  Keith Shortridge,et al.  Learning from 25 years of the extensible N-Dimensional Data Format , 2014, Astron. Comput..

[26]  Brian Thomas,et al.  Significant problems in FITS limit its use in modern astronomical research , 2014 .

[27]  Jing Li,et al.  The SIP Convention for Representing Distortion in FITS Image Headers , 2005 .

[28]  William D. Pence.  CFITSIO, v2.0: A New Full-Featured Data Interface , 1999 .

[30]  Emmanuel Bertin,et al.  SCAMP: Automatic Astrometric and Photometric Calibration , 2010 .

[31]  Jessica Mink,et al.  The Past, Present and Future of Astronomical Data Formats , 2014 .