It was "all" for "nothing": sharp phase transitions for noiseless discrete channels

We establish a phase transition known as the “all-or-nothing” phenomenon for noiseless discrete channels. This class of models includes the Bernoulli group testing model and the planted Gaussian perceptron model. Previously, the existence of the all-or-nothing phenomenon for such models was only known in a limited range of parameters. Our work extends the results to all signals with arbitrary sublinear sparsity. Over the past several years, the all-or-nothing phenomenon has been established in various models as an outcome of two seemingly disjoint results: one positive result establishing the “all” half of all-or-nothing, and one impossibility result establishing the “nothing” half. Our main technique in the present work is to show that for noiseless discrete channels, the “all” half implies the “nothing” half, that is a proof of “all” can be turned into a proof of “nothing.” Since the “all” half can often be proven by straightforward means—for instance, by the firstmoment method—our equivalence gives a powerful and general approach towards establishing the existence of this phenomenon in other contexts.

[1]  Galen Reeves,et al.  The replica-symmetric prediction for compressed sensing with Gaussian matrices is exact , 2016, 2016 IEEE International Symposium on Information Theory (ISIT).

[2]  Editors , 2003 .

[3]  HighWire Press.  Philosophical transactions of the Royal Society of London. Series A, Containing papers of a mathematical or physical character , 1896 .

[4]  Galen Reeves,et al.  The All-or-Nothing Phenomenon in Sparse Linear Regression , 2019, COLT.

[5]  Ina Ruck,et al.  USA , 1969, The Lancet.

[6]  T. Sanders,et al.  Analysis of Boolean Functions , 2012, ArXiv.

[7]  Michael I. Jordan,et al.  Advances in Neural Information Processing Systems 30 , 1995 .

[8]  Galen Reeves,et al.  All-or-Nothing Phenomena: From Single-Letter to High Dimensions , 2019, 2019 IEEE 8th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP).

[9]  R. Lathe.  Phd by thesis , 1988, Nature.

[10]  Nicolas Macris,et al.  Optimal errors and phase transitions in high-dimensional generalized linear models , 2017, Proceedings of the National Academy of Sciences.

[11]  Ilias Zadik,et al.  The All-or-Nothing Phenomenon in Sparse Tensor PCA , 2020, NeurIPS.

[12]  Tobias J. Hagge,et al.  Physics , 1929, Nature.

[13]  A. Sterrett.  On the Detection of Defective Members of Large Populations , 1957 .

[14]  Andrea Montanari,et al.  Maxwell Construction: The Hidden Bridge Between Iterative and Maximum a Posteriori Decoding , 2005, IEEE Transactions on Information Theory.

[15]  Matthew Aldridge,et al.  Group testing: an information theory perspective , 2019, Found. Trends Commun. Inf. Theory.

[16]  Jess Banks,et al.  Information-theoretic bounds and phase transitions in clustering, sparse PCA, and submatrix localization , 2016, 2017 IEEE International Symposium on Information Theory (ISIT).

[17]  Santhosh Kumar,et al.  Reed–Muller Codes Achieve Capacity on Erasure Channels , 2015, IEEE Transactions on Information Theory.

[18]  Florent Krzakala,et al.  Statistical physics of inference: thresholds and algorithms , 2015, ArXiv.

[19]  S. Kak.  Information, physics, and computation , 1996 .

[20]  Will Perkins,et al.  Storage capacity in symmetric binary perceptrons , 2019, Journal of Physics A: Mathematical and Theoretical.

[21]  Ilias. Zadik,et al.  Computational and statistical challenges in high dimensional statistical models , 2019 .

[22]  Lan V. Truong,et al.  On the All-or-Nothing Behavior of Bernoulli Group Testing , 2020, IEEE Journal on Selected Areas in Information Theory.

[23]  Jean Barbier,et al.  0-1 Phase Transitions in Sparse Spiked Matrix Estimation , 2019, ArXiv.

[24]  Ilias Zadik,et al.  Group testing and local search: is there a computational-statistical gap? , 2020, COLT.