Compressive Sensing with Low Precision Data Representation: Radio Astronomy and Beyond

Modern scientific instruments produce vast amounts of data, which can overwhelm the processing ability of computer systems. Lossy compression of data is an intriguing solution but comes with its own dangers, such as potential signal loss, and the need for careful parameter optimization. In this work, we focus on a setting where this problem is especially acute compressive sensing frameworks for radio astronomy and ask: Can the precision of the data representation be lowered for all input data, with recovery guarantees and good practical performance? Our first contribution is a theoretical analysis of the Iterative Hard Thresholding (IHT) algorithm when all input data, that is, the measurement matrix and the observation, are quantized aggressively, to as little as 2 bits per value. Under reasonable constraints, we show that there exists a variant of low precision IHT which can still provide recovery guarantees. The second contribution is a tailored analysis of our general quantized framework to radio astronomy, showing that its conditions are satisfied in this case. We evaluate our approach using an FPGA implementation, and show that it can achieve up to 9.19x speed up with negligible loss of recovery quality, on real telescope data

[1]  J. Högbom,et al.  APERTURE SYNTHESIS WITH A NON-REGULAR DISTRIBUTION OF INTERFEROMETER BASELINES. Commentary , 1974 .

[2]  C. Carilli,et al.  Synthesis Imaging in Radio Astronomy II , 1999 .

[3]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[4]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[5]  R. Chartrand,et al.  Restricted isometry properties and nonconvex compressive sensing , 2007 .

[6]  T. Blumensath,et al.  Iterative Thresholding for Sparse Approximations , 2008 .

[7]  J. Usón,et al.  Correcting direction-dependent gains in the deconvolution of radio interferometric images , 2008, 0805.0834.

[8]  E. Candès.  The restricted isometry property and its implications for compressed sensing , 2008 .

[9]  Richard G. Baraniuk,et al.  1-Bit compressive sensing , 2008, 2008 42nd Annual Conference on Information Sciences and Systems.

[10]  J. Tropp,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, Commun. ACM.

[11]  P. Vandergheynst,et al.  Compressed sensing imaging techniques for radio interferometry , 2008, 0812.4933.

[12]  Marcus A. Magnor,et al.  Compressed sensing for aperture synthesis imaging , 2010, 2010 IEEE International Conference on Image Processing.

[13]  Mike E. Davies,et al.  Normalized Iterative Hard Thresholding: Guaranteed Stability and Performance , 2010, IEEE Journal of Selected Topics in Signal Processing.

[14]  Robert D. Nowak,et al.  Sample complexity for 1-bit compressed sensing and sparse classification , 2010, 2010 IEEE International Symposium on Information Theory.

[15]  Deanna Needell,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, ArXiv.

[16]  Feng Li,et al.  The application of compressive sampling to radio astronomy I: Deconvolution , 2011, 1106.1711.

[17]  Volkan Cevher,et al.  On accelerated hard thresholding methods for sparse approximation , 2011, Optical Engineering + Applications.

[18]  Wotao Yin,et al.  Trust, But Verify: Fast and Accurate Signal Recovery From 1-Bit Compressive Measurements , 2011, IEEE Transactions on Signal Processing.

[19]  Yaniv Plan,et al.  One‐Bit Compressed Sensing by Linear Programming , 2011, ArXiv.

[20]  Ewout van den Berg,et al.  1-Bit Matrix Completion , 2012, ArXiv.

[21]  Mike E. Davies,et al.  Greedy algorithms for compressed sensing , 2012, Compressed Sensing.

[22]  Yaniv Plan,et al.  One-bit compressed sensing with non-Gaussian measurements , 2012, ArXiv.

[23]  Thomas Blumensath,et al.  Accelerated iterative hard thresholding , 2012, Signal Process..

[24]  Prateek Jain,et al.  One-Bit Compressed Sensing: Provable Support and Vector Recovery , 2013, ICML.

[25]  Yaniv Plan,et al.  Robust 1-bit Compressed Sensing and Sparse Logistic Regression: A Convex Programming Approach , 2012, IEEE Transactions on Information Theory.

[26]  Thomas Blumensath,et al.  Compressed Sensing With Nonlinear Observations and Related Nonlinear Optimization Problems , 2012, IEEE Transactions on Information Theory.

[27]  Alle-Jan van der Veen,et al.  Signal Processing Tools for Radio Astronomy , 2013, Handbook of Signal Processing Systems.

[28]  Laurent Jacques,et al.  Robust 1-Bit Compressive Sensing via Binary Stable Embeddings of Sparse Vectors , 2011, IEEE Transactions on Information Theory.

[29]  Dong Yu,et al.  1-bit stochastic gradient descent and its application to data-parallel distributed training of speech DNNs , 2014, INTERSPEECH.

[30]  Xiao-Tong Yuan,et al.  Gradient Hard Thresholding Pursuit for Sparsity-Constrained Optimization , 2013, ICML.

[31]  Pritish Narayanan,et al.  Deep Learning with Limited Numerical Precision , 2015, ICML.

[32]  Ke Wei,et al.  Fast Iterative Hard Thresholding for Compressed Sensing , 2015, IEEE Signal Processing Letters.

[33]  Jared Tanner,et al.  Conjugate Gradient Iterative Hard Thresholding: Observed Noise Stability for Compressed Sensing , 2015, IEEE Transactions on Signal Processing.

[34]  Kunle Olukotun,et al.  Taming the Wild: A Unified Analysis of Hogwild-Style Algorithms , 2015, NIPS.

[35]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[36]  Bin Liu,et al.  Ternary Weight Networks , 2016, ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[37]  Luca Cinquini,et al.  Scalable Data Mining, Archiving, and Big Data Management for the Next Generation Astronomical Telescopes , 2016 .

[38]  Xiao-Tong Yuan,et al.  Exact Recovery of Hard Thresholding Pursuit , 2016, NIPS.

[39]  Dan Alistarh,et al.  QSGD: Randomized Quantization for Communication-Optimal Stochastic Gradient Descent , 2016, ArXiv.

[40]  Ali Farhadi,et al.  XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks , 2016, ECCV.

[41]  Daisuke Miyashita,et al.  Convolutional Neural Networks using Logarithmic Data Representation , 2016, ArXiv.

[42]  Qingshan Liu,et al.  Dual Iterative Hard Thresholding: From Non-convex Sparse Minimization to Non-smooth Concave Maximization , 2017, ICML.

[43]  Dan Alistarh,et al.  ZipML: Training Linear Models with End-to-End Low Precision, and a Little Bit of Deep Learning , 2017, ICML.

[44]  Gustavo Alonso,et al.  FPGA-Accelerated Dense Linear Machine Learning: A Precision-Convergence Trade-Off , 2017, 2017 IEEE 25th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM).

[45]  Ran El-Yaniv,et al.  Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations , 2016, J. Mach. Learn. Res..