Machine Learning Classification of SDSS Transient Survey Images

We show that multiple machine learning algorithms can match human performance in classifying transient imaging data from the Sloan Digital Sky Survey (SDSS) supernova survey into real objects and artefacts. This is a first step in any transient science pipeline and is currently still done by humans, but future surveys such as the Large Synoptic Survey Telescope (LSST) will necessitate fully machine-enabled solutions. Using features trained from eigenimage analysis (principal component analysis, PCA) of single-epoch g, r and i-difference images, we can reach a completeness (recall) of 96 per cent, while only incorrectly classifying at most 18 per cent of artefacts as real objects, corresponding to a precision (purity) of 84 per cent. In general, random forests performed best, followed by the k-nearest neighbour and the SkyNet artificial neural net algorithms, compared to other methods such as na\"ive Bayes and kernel support vector machine. Our results show that PCA-based machine learning can match human success levels and can naturally be extended by including multiple epochs of data, transient colours and host galaxy information which should allow for significant further improvements, especially at low signal-to-noise.

[1]  Fionn Murtagh,et al.  Multilayer perceptrons for classification and regression , 1991, Neurocomputing.

[2]  J. Fleiss,et al.  Statistical methods for rates and proportions , 1973 .

[3]  M. Wainwright,et al.  Using machine learning for discovery in synoptic survey imaging data , 2012, 1209.3775.

[4]  Radford M. Neal Pattern Recognition and Machine Learning , 2007, Technometrics.

[5]  Ashutosh Kumar Singh,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2010 .

[6]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[7]  D. Signorini,et al.  Neural networks , 1995, The Lancet.

[8]  R. Nichol,et al.  COSMOLOGY WITH PHOTOMETRICALLY CLASSIFIED TYPE Ia SUPERNOVAE FROM THE SDSS-II SUPERNOVA SURVEY , 2012, 1211.4480.

[9]  N. S. Philip,et al.  Results from the Supernova Photometric Classification Challenge , 2010, 1008.1024.

[10]  E. O. Ofek,et al.  Automating Discovery and Classification of Transients and Variable Stars in the Synoptic Survey Era , 2011, 1106.5491.

[11]  Sabrina S Wilson Radiology , 1938, Glasgow Medical Journal.

[12]  S. Danziger,et al.  Extraneous factors in judicial decisions , 2011, Proceedings of the National Academy of Sciences.

[13]  A. Cooray,et al.  COSMOLOGY WITH PHOTOMETRIC SURVEYS OF TYPE Ia SUPERNOVAE , 2009, 0909.2692.

[14]  J. Kaplan,et al.  THE SLOAN DIGITAL SKY SURVEY-II SUPERNOVA SURVEY: TECHNICAL SUMMARY , 2007, 0708.2749.

[15]  J. Prieto,et al.  THE SLOAN DIGITAL SKY SURVEY-II SUPERNOVA SURVEY: SEARCH ALGORITHM AND FOLLOW-UP OBSERVATIONS , 2007, 0708.2750.

[16]  Michael E. Tipping,et al.  Probabilistic Principal Component Analysis , 1999 .

[17]  Douglas G. Altman,et al.  Practical statistics for medical research , 1990 .

[18]  Jacob Cohen A Coefficient of Agreement for Nominal Scales , 1960 .

[19]  M. Sullivan,et al.  Galaxy Zoo Supernovae , 2010, 1011.2199.

[20]  Ryan Chornock,et al.  Observed Fractions of Core-Collapse Supernova Types and Initial Masses of their Single and Binary Progenitor Stars , 2010, 1006.3899.

[21]  Thomas G. Dietterich What is machine learning? , 2020, Archives of Disease in Childhood.

[22]  Ethem Alpaydin,et al.  Machine learning , 2011 .

[23]  R. C. Wolf,et al.  AUTOMATED TRANSIENT IDENTIFICATION IN THE DARK ENERGY SURVEY , 2015, 1504.02936.

[24]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[25]  Melvin M. Varughese,et al.  Parameter estimation with Bayesian estimation applied to multiple species in the presence of biases and correlations , 2012 .

[26]  Shlomo Geva,et al.  A constructive method for multivariate function approximation by multilayer perceptrons , 1992, IEEE Trans. Neural Networks.

[27]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.

[28]  R. J. Brunner,et al.  TPZ: photometric redshift PDFs and ancillary information by using prediction trees and random forests , 2013, 1303.7269.

[29]  M. Kunz,et al.  Bayesian estimation applied to multiple species , 2006, astro-ph/0611004.

[30]  S. Bailey,et al.  How to Find More Supernovae with Less Work: Object Classification Techniques for Difference Imaging , 2006, 0705.0493.

[31]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[32]  M. Kunz,et al.  Extending BEAMS to incorporate correlated systematic uncertainties , 2012, 1205.3493.

[33]  Bruce A. Bassett,et al.  Machine Classification of Transient Images , 2014 .

[34]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[35]  J. Hanley,et al.  The meaning and use of the area under a receiver operating characteristic (ROC) curve. , 1982, Radiology.

[36]  Farhan Feroz,et al.  SKYNET: an efficient and robust neural network training tool for machine learning in astronomy , 2013, ArXiv.

[37]  Pieter Reitsma,et al.  Educational and Psychological Measurement , 2003 .

[38]  B. Everitt,et al.  Statistical methods for rates and proportions , 1973 .

[39]  Melvin M. Varughese,et al.  PHOTOMETRIC SUPERNOVA COSMOLOGY WITH BEAMS AND SDSS-II , 2011, 1111.5328.

[40]  E. Bertin,et al.  SExtractor: Software for source extraction , 1996 .