The data processing pipeline for the MUSE instrument

Processing of raw data from modern astronomical instruments is nowadays often carried out using dedicated software, so-called "pipelines" which are largely run in automated operation. In this paper we describe the data reduction pipeline of the Multi Unit Spectroscopic Explorer (MUSE) integral field spectrograph operated at ESO's Paranal observatory. This spectrograph is a complex machine: it records data of 1152 separate spatial elements on detectors in its 24 integral field units. Efficiently handling such data requires sophisticated software, a high degree of automation and parallelization. We describe the algorithms of all processing steps that operate on calibrations and science data in detail, and explain how the raw science data gets transformed into calibrated datacubes. We finally check the quality of selected procedures and output data products, and demonstrate that the pipeline provides datacubes ready for scientific analysis.

[1]  B. Edĺen.  The Refractive Index of Air , 1966 .

[2]  J. Owens,et al.  Optical refractive index of air: dependence on pressure, temperature and composition. , 1967, Applied optics.

[3]  A. V. Filippenko,et al.  THE IMPORTANCE OF ATMOSPHERIC DIFFERENTIAL REFRACTION IN SPECTROPHOTOMETRY. , 1982 .

[4]  P. Stetson.  DAOPHOT: A COMPUTER PROGRAM FOR CROWDED-FIELD STELLAR PHOTOMETRY , 1987 .

[5]  Robert J. Renka,et al.  Multivariate interpolation of large sets of scattered data , 1988, TOMS.

[6]  K. P. Birch,et al.  An Updated Edln Equation for the Refractive Index of Air , 1993 .

[7]  E. Greisen,et al.  Representations of celestial coordinates in FITS , 2002, astro-ph/0207413.

[8]  P. Ciddor.  Refractive index of air: new equations for the visible and near infrared. , 1996, Applied optics.

[9]  Francisco Prada,et al.  New parametrizations of non-Gaussian line-of-sight velocity distribution , 1996 .

[10]  W. C. Martin,et al.  Atomic Spectra Database , 1999 .

[11]  R. Davies,et al.  The SAURON project – I. The panoramic integral-field spectrograph , 2001, astro-ph/0103451.

[12]  P. Dokkum.  Cosmic-Ray Rejection by Laplacian Edge Detection , 2001, astro-ph/0108003.

[13]  I. Hook,et al.  Integral Field Spectroscopy with the Gemini Multiobject Spectrograph.I. Design, Construction, and Testing , 2002 .

[14]  C. Goessl,et al.  Image reduction pipeline for the detection of variable sources in highly crowded fields , 2001, astro-ph/0110704.

[15]  Peter J. Quinn,et al.  Observatory Operations to Optimize Scientific Return III , 2002 .

[16]  D. Kelson.  Optimal Techniques in Two‐dimensional Spectroscopy: Background Subtraction for the 21st Century , 2003, astro-ph/0303507.

[17]  K. Jahnke,et al.  Integral-field spectrophotometry of the quadruple QSO HE 0435 1223: Evidence for microlensing , 2003, astro-ph/0307147.

[18]  A. Moorwood,et al.  Instrument Design and Performance for Optical/Infrared Ground-based Telescopes, , 2003 .

[19]  David Dunlap Observatory.  A Fast Algorithm for Cosmic Rays Removal from Single Images , 2003 .

[20]  J. R. Walsh.  Euro3D Science Conference , 2004 .

[21]  Arlette Pécontal-Rousset,et al.  The Euro3D data format: A common FITS data format for integral field spectrographs , 2004 .

[22]  W. Pych.  A Fast Algorithm for Cosmic‐Ray Removal from Single Images , 2003, astro-ph/0311290.

[23]  H Germany,et al.  PMAS: The Potsdam Multi‐Aperture Spectrophotometer. I. Design, Manufacture, and Performance , 2005, astro-ph/0502581.

[24]  A. Monreal-Ibero,et al.  INTEGRAL FIELD SPECTROSCOPY OF FAINT HALOS OF PLANETARY NEBULAE , 2005 .

[25]  A. Mazure,et al.  The VIMOS Integral Field Unit: Data‐Reduction Methods and Quality Assessment , 2005, astro-ph/0509454.

[26]  Caroline van Breukelen,et al.  The luminosity function of Lyα emitters at 2.3 z , 2005, astro-ph/0502409.

[27]  Steven G. Johnson,et al.  The Design and Implementation of FFTW3 , 2005, Proceedings of the IEEE.

[28]  R.I.Davies.  A method to remove residual OH emission from near infrared spectra , 2006, astro-ph/0612257.

[29]  Steve B. Howell,et al.  Handbook of CCD Astronomy: Contents , 2006 .

[30]  Molefe Mokoene,et al.  The Messenger , 1995, Outrageous Fortune.

[31]  Martin Roth.  The Euro3D research training network , 2006 .

[32]  David L. Huestis,et al.  High‐resolution terrestrial nightglow emission line atlas from UVES/VLT: Positions, intensities, and identifications for 2808 lines at 314–1043 nm , 2006 .

[33]  R. Davies,et al.  A method to remove residual OH emission from near-infrared spectra , 2007 .

[34]  Martin M. Roth,et al.  Spatially resolved spectroscopy of planetary nebulae and their halos - I. Five galactic disk objects , 2008, 0802.3813.

[35]  William D. Pence,et al.  CFITSIO: A FITS File Subroutine Library , 2010 .

[36]  P. Weilbacher,et al.  P3D: a general data-reduction tool for fiber-fed integral-field spectrographs , 2010, 1002.4406.

[37]  M. R. Calabretta.  Wcslib and Pgsbox , 2011 .

[38]  G. Bruce Berriman,et al.  Astrophysics Source Code Library , 2012, ArXiv.

[39]  C. Sandin,et al.  PyCosmic: a robust method to detect cosmics in CALIFA and other fiber-fed integral-field spectroscopy datasets , 2012, 1208.1696.

[40]  Jeremy S. Heyl.  A fast matching algorithm for sheared stellar samples: k-d match , 2013 .

[41]  W. Freudling,et al.  Automated data reduction workflows for astronomy , 2013, 1311.5411.

[42]  Martin M. Roth,et al.  Resolving stellar populations with crowded field 3D spectroscopy , 2012, 1211.0445.

[43]  W. Kausch,et al.  Skycorr: A general tool for spectroscopic sky subtraction , 2014, 1405.3679.

[44]  P. Weilbacher,et al.  The MUSE 3D view of the Hubble Deep Field South , 2014, 1411.7667.

[45]  Benoit Epinat,et al.  Unveiling the counter-rotating nature of the kinematically distinct core in NGC 5813 with MUSE , 2015, 1505.06226.

[46]  W. Kausch,et al.  Molecfit: A general tool for telluric absorption correction - I. Method and application to ESO instruments , 2015, 1501.07239.

[47]  E. Popow,et al.  PEPSI: The high-resolution ichelle spectrograph and polarimeter for the Large Binocular Telescope , 2015, 1505.06492.

[48]  S. Ramsay,et al.  The Pillars of Creation revisited with MUSE: gas kinematics and high-mass stellar feedback traced by optical spectroscopy , 2015, 1504.03323.

[49]  J. Fensch,et al.  Ionization processes in a local analogue of distant clumpy galaxies: VLT MUSE IFU spectroscopy and FORS deep images of the TDG NGC 5291N , 2015, 1509.08873.

[50]  Andreas Kelz,et al.  A MUSE map of the central Orion Nebula (M 42) , 2015, 1507.00006.

[51]  European Southern Observatory,et al.  Towards DIB mapping in galaxies beyond 100 Mpc. A radial profile of the $\lambda$5780.5 diffuse interstellar band in AM 1353-272 B , 2015, 1502.06548.

[52]  P. Weilbacher,et al.  MUSE observations of the lensing cluster SMACSJ2031.8-4036: new constraints on the mass distribution in the cluster core. , 2014, 1409.2488.

[53]  E. Emsellem,et al.  Extended Lyman α haloes around individual high-redshift galaxies revealed by MUSE , 2015, 1509.05143.

[54]  Roland Bacon,et al.  MPDAF: MUSE Python Data Analysis Framework , 2016 .

[55]  Simon J. Lilly,et al.  UBIQUITOUS GIANT Lyα NEBULAE AROUND THE BRIGHTEST QUASARS AT z ∼ 3.5 REVEALED WITH MUSE , 2016, 1605.01422.

[56]  Martin M. Roth,et al.  MUSE crowded field 3D spectroscopy of over 12 000 stars in the globular cluster NGC 6397 - I. The first comprehensive HRD of a globular cluster , 2016, 1602.01649.

[57]  Eric Emsellem,et al.  Exploring the mass assembly of the early-type disc galaxy NGC 3115 with MUSE , 2016, 1605.07667.

[58]  Simon J. Lilly,et al.  ZAP -- Enhanced PCA Sky Subtraction for Integral Field Spectroscopy , 2016, 1602.08037.

[59]  Martin M. Roth,et al.  On the Origin of Diffuse Ionized Gas in the Antennae Galaxy , 2017, 1712.04450.

[60]  Lutz Wisotzki,et al.  LSDCat: Detection and cataloguing of emission-line sources in integral-field spectroscopy datacubes , 2017, 1703.05166.

[61]  David Mary,et al.  The MUSE Hubble Ultra Deep Field Survey: I. Survey description, data reduction and source detection , 2017, 1710.03002.

[62]  Guy Monnet,et al.  Optical 3D-Spectroscopy for Astronomy , 2017 .

[63]  B. Guiderdoni,et al.  Nearly all the sky is covered by Lyman-α emission around high-redshift galaxies , 2018, Nature.

[64]  Jonathan Tennyson,et al.  Analysis of gaseous ammonia (NH3) absorption in the visible spectrum of Jupiter - Update , 2018, Icarus.

[65]  J. J. González-Vidal,et al.  Gaia Data Release 2 , 2018, Astronomy & Astrophysics.

[66]  Christer Sandin,et al.  Lyman-continuum leakage as dominant source of diffuse ionized gas in the Antennae galaxy , 2018 .

[67]  Matthew Colless,et al.  The SAMI Galaxy Survey: Data Release Two with absorption-line physics value-added products , 2018, Monthly Notices of the Royal Astronomical Society.

[68]  R. Pello,et al.  Kinematics, turbulence, and star formation of z ∼ 1 strongly lensed galaxies seen with MUSE , 2018, 1802.08451.

[69]  M. Steinmetz,et al.  Cluster kinematics and stellar rotation in NGC 419 with MUSE and adaptive optics , 2018, Monthly Notices of the Royal Astronomical Society.

[70]  A. Kelz,et al.  MUSE crowded field 3D spectroscopy in NGC 300 , 2018, Astronomy & Astrophysics.

[71]  Martin M. Roth,et al.  A stellar census in globular clusters with MUSE: The contribution of rotation to cluster dynamics studied with 200 000 stars , 2017, 1710.07257.

[72]  T. Nanayakkara,et al.  The MUSE-Wide Survey: survey description and first data release , 2018, Astronomy & Astrophysics.

[73]  J. H. Knapen,et al.  MUSE-AO view of the starburst–AGN connection: NGC 7130 , 2018, Astronomy & Astrophysics.

[74]  C. U. Keller,et al.  Two accreting protoplanets around the young star PDS 70 , 2019, Nature Astronomy.

[75]  Gaspare Lo Curto,et al.  Rotational and Rotational-Vibrational Raman Spectroscopy of Air to Characterize Astronomical Spectrographs. , 2019, Physical review letters.

[76]  I. Smail,et al.  The MUSE Ultra Deep Field (MUDF). II. Survey design and the gaseous properties of galaxy groups at 0.5 < z < 1.5 , 2019, Monthly Notices of the Royal Astronomical Society.

[77]  T. Contini,et al.  Three-Dimensional Optimal Spectral Extraction (TDOSE) from integral field spectroscopy , 2019, Astronomy & Astrophysics.

[78]  G. Orton,et al.  Latitudinal variation in the abundance of methane (CH4) above the clouds in Neptune's atmosphere from VLT/MUSE Narrow Field Mode Observations , 2019, Icarus.

[79]  Sylvain Oberti,et al.  Physics-based model of the adaptive-optics-corrected point spread function , 2019, Astronomy & Astrophysics.