Deep reinforcement learning for smart calibration of radio telescopes

Modern radio telescopes produce unprecedented amounts of data, which are passed through many processing pipelines before the delivery of scientific results. Hyperparameters of these pipelines need to be tuned by hand to produce optimal results. Because many thousands of observations are taken during a lifetime of a telescope and because each observation will have its unique settings, the fine tuning of pipelines is a tedious task. In order to automate this process of hyperparameter selection in data calibration pipelines, we introduce the use of reinforcement learning. We test two reinforcement learning techniques, twin delayed deep deterministic policy gradient (TD3), and soft actor-critic, to train an autonomous agent to perform this fine tuning. For the sake of generalization, we consider the pipeline to be a black-box system where the summarized state of the performance of the pipeline is used by the autonomous agent. The autonomous agent trained in this manner is able to determine optimal settings for diverse observations and is therefore able to perform smart calibration, minimizing the need for human intervention.

[1]  Tapani Raiko,et al.  International Conference on Learning Representations (ICLR) , 2016 .

[2]  Aapo Hyvärinen,et al.  Proceedings of the 13th International Conference on Artificial Intelligence and Statistics (AISTATS) , 2010 .

[3]  S. Weisberg,et al.  Residuals and Influence in Regression , 1982 .

[4]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[5]  Sarod Yatawatta,et al.  Statistical Performance of Radio Interferometric Calibration , 2019, Monthly Notices of the Royal Astronomical Society.

[6]  Stephanie Thalberg,et al.  Interferometry And Synthesis In Radio Astronomy , 2016 .

[7]  Sarod Yatawatta,et al.  Distributed Radio Interferometric Calibration , 2015, ArXiv.

[8]  Sarod Yatawatta,et al.  Stochastic Calibration of Radio Interferometers , 2020, ArXiv.

[9]  S. Zaroubi,et al.  Improved upper limits on the 21 cm signal power spectrum of neutral hydrogen at z ≈ 9.1 from LOFAR , 2020, Monthly Notices of the Royal Astronomical Society.

[10]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[11]  Krzysztof J. Geras,et al.  International Conference on Learning Representations (ICLR) 2015 , 2015 .

[12]  Eugenia Ternovska,et al.  Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence (AAAI-16) , 2016, AAAI 2016.

[13]  Elie Bienenstock,et al.  Neural Networks and the Bias/Variance Dilemma , 1992, Neural Computation.

[14]  Faruk Diblen,et al.  Data Multiplexing in Radio Interferometric Calibration , 2017, ArXiv.

[15]  J. Schaye,et al.  Upper Limits on the 21 cm Epoch of Reionization Power Spectrum from One Night with LOFAR , 2017, 1702.08679.

[16]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .

[17]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[18]  Pascal Larzabal,et al.  Parallel multi-wavelength calibration algorithm for radio astronomical arrays , 2018, Signal Process..

[19]  David Mary,et al.  A parallel and automatically tuned algorithm for multispectral image deconvolution , 2019, Monthly Notices of the Royal Astronomical Society.

[20]  Pascal Larzabal,et al.  Robust distributed calibration of radio interferometers with direction dependent distortions , 2018, Signal Process..

[21]  R. Sault,et al.  Understanding radio polarimetry. I. Mathematical foundations , 1996 .

[22]  Susan A. Murphy,et al.  Monographs on statistics and applied probability , 1990 .