DES science portal: Creating science-ready catalogs

We present a novel approach for creating science-ready catalogs through a software infrastructure developed for the Dark Energy Survey (DES). We integrate the data products released by the DES Data Management and additional products created by the DES collaboration in an environment known as DES Science Portal. Each step involved in the creation of a science-ready catalog is recorded in a relational database and can be recovered at any time. We describe how the DES Science Portal automates the creation and characterization of lightweight catalogs for DES Year 1 Annual Release, and show its flexibility in creating multiple catalogs with different inputs and configurations. Finally, we discuss the advantages of this infrastructure for large surveys such as DES and the Large Synoptic Survey Telescope. The capability of creating science-ready catalogs efficiently and with full control of the inputs and configurations used is an important asset for supporting science analysis using data from large astronomical surveys.

Michael Schubnell | Eric Suchyta | Elizabeth Buckley-Geer | Klaus Honscheid | Peter Doel | David James | Felipe Menanteau | David Brooks | Tim Abbott | Guilherme Soares | Matias Carrasco Kind | Alex Drlica-Wagner | David Gerdes | Gregory Tarle | August Evrard | Vic Scarpine | Kyler Kuehn | Ofer Lahav | Riccardo Campisano | Pablo Fosalba | Filipe Abdalla | Robert Gruendl | Alistair Walker | Jorge Carretero | Eusebio Sanchez | Tesla Jeltema | Ignacio Sevilla | Molly Swanson | Rafe Schindler | Chris D'Andrea | Marcio Maia | Peter Melchior | Juan Garcia-Bellido | R. Chris Smith | Rogerio Rosenfeld | Sahar Allam | Nikolay Kuropatkin | Julia Gschwend | Aurelio Carnero Rosell | Ricardo Ogando | Angelo Fausti Neto | Luiz da Costa | Marcos Lima | Cristiano Singulani | Carlos Adean | Shantanu Desai | Gaston Gutierrez | Basilio Santiago | Mathew Smith | Flavia Sobreira | Lucas Nunes | Rafael Brito | Glauber C. Vila-Verde | Aur'elien Benoit-L'evy | Diego Capozzi | Steve Kuhlmann | Jennifer Marshall | Andr'es Plazas Malag'on | D. Gerdes | O. Lahav | P. Fosalba | F. Abdalla | J. García-Bellido | D. Capozzi | L. Costa | K. Honscheid | M. Maia | R. Ogando | F. Sobreira | M. Swanson | Peter Melchior | S. Kuhlmann | R. Gruendl | S. Allam | J. Gschwend | I. Sevilla-Noarbe | T. Abbott | E. Buckley-Geer | J. Carretero | C. D'Andrea | S. Desai | P. Doel | A. Drlica-Wagner | A. Evrard | G. Gutiérrez | D. James | T. Jeltema | K. Kuehn | N. Kuropatkin | F. Menanteau | A. Plazas | V. Scarpine | R. Schindler | M. Schubnell | E. Suchyta | G. Tarlé | A. Walker | E. Sánchez | A. Benoit-Lévy | M. Lima | Robert C. Smith | R. Rosenfeld | M. C. Kind | A. C. Rosell | D. Brooks | J. Marshall | B. Santiago | A. F. Neto | R. Campisano | L. Nunes | C. Singulani | C. Adean | R. Brito | G. Soares | G. Vila-Verde | Matthew C. Smith | M. Swanson

[1]  Jordan Raddick,et al.  SciServer: An Online Collaborative Environment for Big Data in Research and Education , 2017 .

[2]  Mark Taylor,et al.  Astrophysics Source Code Library, version 3.1 , 2016 .

[3]  William E. Harris,et al.  A Catalog of Parameters for Globular Clusters in the Milky Way , 1996 .

[4]  Nolan Li,et al.  CasJobs and MyDB: A Batch Query Workbench , 2008, Computing in Science & Engineering.

[5]  C. Benoist,et al.  ESO imaging survey - Deep public survey: Multi-color optical data for the Chandra Deep Field South , 2001, astro-ph/0103071.

[6]  Cosmological Constraints from Measurements of Type Ia , 2014 .

[7]  D. Schlegel,et al.  Maps of Dust IR Emission for Use in Estimation of Reddening and CMBR Foregrounds , 1997, astro-ph/9710327.

[8]  D. Schiminovich,et al.  The First Release COSMOS Optical and Near-IR Data and Catalog , 2007, 0704.2430.

[9]  M. Skrutskie,et al.  The Two Micron All Sky Survey (2MASS) , 2006 .

[10]  R. Keisler,et al.  Assessing Galaxy Limiting Magnitudes in Large Optical Surveys , 2015, 1509.00870.

[11]  Peter Z. Kunszt,et al.  The SDSS skyserver: public access to the sloan digital sky server data , 2001, SIGMOD '02.

[12]  R. J. Brunner,et al.  TPZ: photometric redshift PDFs and ancillary information by using prediction trees and random forests , 2013, 1303.7269.

[13]  M. Sullivan,et al.  THE DIFFERENCE IMAGING PIPELINE FOR THE TRANSIENT SEARCH IN THE DARK ENERGY SURVEY , 2015, 1507.05137.

[14]  A. Mazure,et al.  The VIMOS VLT deep survey First epoch VVDS-deep survey : 11 564 spectra with 17 . 5 ≤ IAB ≤ 24 , and the redshift distribution over 0 ≤ z ≤ 5 , 2005 .

[15]  J. Frieman,et al.  Star-galaxy classification in the Dark Energy Survey Y1 dataset , 2018, Monthly Notices of the Royal Astronomical Society.

[16]  Adrian T. Lee,et al.  The 10 Meter South Pole Telescope , 2009, 0907.4445.

[17]  D. Gerdes,et al.  The Dark Energy Survey view of the Sagittarius stream: discovery of two faint stellar system candidates , 2016, 1608.04033.

[18]  Karl Glazebrook,et al.  An imaging K-band survey - I: The catalogue, star and galaxy counts , 1994 .

[19]  D. Gerdes,et al.  SDSS-IV eBOSS emission-line galaxy pilot survey , 2015, 1509.05045.

[20]  J. Vicente,et al.  DNF – Galaxy photometric redshift by Directional Neighbourhood Fitting , 2015, 1511.07623.

[21]  R. Nichol,et al.  Digging deeper into the Southern skies: a compact Milky Way companion discovered in first-year Dark Energy Survey data , 2015, 1508.02381.

[22]  L. Moscardini,et al.  Measuring the Redshift Evolution of Clustering: the Hubble Deep Field South , 2002 .

[23]  E. Hatziminaoglou,et al.  Star counts in the Galaxy - Simulating from very deep to very shallow photometric surveys with the TRILEGAL code , 2005, astro-ph/0504047.

[24]  B. Yanny,et al.  The Dark Energy Survey Image Processing Pipeline , 2018, 1801.03177.

[25]  A. Myers,et al.  THE SDSS-IV EXTENDED BARYON OSCILLATION SPECTROSCOPIC SURVEY: LUMINOUS RED GALAXY TARGET SELECTION , 2015, 1508.04478.

[26]  Gavin Dalton,et al.  The Oxford–Dartmouth Thirty Degree Survey – I. Observations and calibration of a wide-field multiband survey , 2004 .

[27]  B. Flaugher.  The Dark Energy Survey , 2005 .

[28]  Eric Suchyta,et al.  DES science portal: Computing photometric redshifts , 2017, Astron. Comput..

[29]  Richard Grunzke,et al.  Using Science Gateways for Bridging the Differences between Research Infrastructures , 2016, Journal of Grid Computing.

[30]  Edwin A. Valentijn,et al.  The Kilo-Degree Survey , 2012, Experimental Astronomy.

[31]  D. Schlegel,et al.  Maps of Dust Infrared Emission for Use in Estimation of Reddening and Cosmic Microwave Background Radiation Foregrounds , 1998 .

[32]  J. Mohr,et al.  The Photometric Calibration of the Dark Energy Survey , 2007 .

[33]  Alan R. Duffy,et al.  THE THEORETICAL ASTROPHYSICAL OBSERVATORY: CLOUD-BASED MOCK GALAXY CATALOGS , 2014, 1403.5270.

[34]  R. Ellis,et al.  COSMOS: Hubble Space Telescope Observations , 2006, astro-ph/0612306.

[35]  Napp,et al.  SDSS data management and photometric quality assessment , 2008 .

[36]  J. Mohr,et al.  THE BLANCO COSMOLOGY SURVEY: DATA ACQUISITION, PROCESSING, CALIBRATION, QUALITY DIAGNOSTICS, AND DATA RELEASE , 2012, 1204.1210.

[37]  E. Bertin,et al.  SExtractor: Software for source extraction , 1996 .

[38]  E. al.,et al.  The Sloan Digital Sky Survey: Technical summary , 2000, astro-ph/0006396.

[39]  A. Pastorello,et al.  SYSTEMATIC UNCERTAINTIES ASSOCIATED WITH THE COSMOLOGICAL ANALYSIS OF THE FIRST PAN-STARRS1 TYPE Ia SUPERNOVA SAMPLE , 2013, 1310.3824.

[40]  Caltech,et al.  Weighing the Giants – II. Improved calibration of photometry from stellar colours and accurate photometric redshifts , 2012, 1208.0602.

[41]  Larry Denneau,et al.  The Pan-STARRS wide-field optical/NIR imaging survey , 2010, Astronomical Telescopes + Instrumentation.

[42]  Mauro Barbieri,et al.  TRILEGAL, a TRIdimensional modeL of thE GALaxy: Status and Future , 2012 .

[43]  K. Gorski,et al.  HEALPix: A Framework for High-Resolution Discretization and Fast Analysis of Data Distributed on the Sphere , 2004, astro-ph/0409513.

[44]  B. Yanny,et al.  Dark Energy Survey Year 1 Results: The Photometric Data Set for Cosmology , 2017, 1708.01531.

[45]  Donald W. Sweeney,et al.  Large Synoptic Survey Telescope: From Science Drivers to Reference Design , 2008 .

[46]  E. Bertin,et al.  MODELING THE TRANSFER FUNCTION FOR THE DARK ENERGY SURVEY , 2014, 1411.0032.

[47]  R. Nichol,et al.  MAPPING AND SIMULATING SYSTEMATICS DUE TO SPATIALLY VARYING OBSERVING CONDITIONS IN DES SCIENCE VERIFICATION DATA , 2015, 1507.05647.

[48]  G. Bruzual,et al.  Stellar population synthesis at the resolution of 2003 , 2003, astro-ph/0309134.

[49]  Robert J. Brunner,et al.  MLZ: Machine Learning for photo-Z , 2014 .

[50]  Nancy Wilkins-Diehr,et al.  Authoring a Science Gateway Cookbook , 2013, 2013 IEEE International Conference on Cluster Computing (CLUSTER).

[51]  Emmanuel Bertin,et al.  The Dark Energy Survey data processing and calibration system , 2012, Other Conferences.