A Spin Glass Model for Reconstructing Nonlinearly Encrypted Signals Corrupted by Noise

We define a (symmetric key) encryption of a signal $$\mathbf{s}\in {\mathbb {R^N}}$$s∈RN as a random mapping $$\mathbf{s}\mapsto \mathbf y =(y_1,\ldots ,y_M)^T\in \mathbb {R}^M$$s↦y=(y1,…,yM)T∈RM known both to the sender and a recipient. In general the recipients may have access only to images $$\mathbf{y}$$y corrupted by an additive noise of unknown strength. Given the encryption redundancy parameter (ERP) $$\mu =M/N\ge 1$$μ=M/N≥1 and the signal strength parameter $$R=\sqrt{\sum _i {s_i^2/N}}$$R=∑isi2/N, we consider the problem of reconstructing $$\mathbf{s}$$s from its corrupted image by a Least Square Scheme for a certain class of random Gaussian mappings. The problem is equivalent to finding the configuration of minimal energy in a certain version of spherical spin glass model, with squared Gaussian random interaction potential. We use the Parisi replica symmetry breaking scheme to evaluate the mean overlap $$p_{\infty }\in [0,1]$$p∞∈[0,1] between the original signal and its recovered image (known as ’estimate’) as $$N\rightarrow \infty $$N→∞, for a given (’bare’) noise-to-signal ratio (NSR) $$\gamma \ge 0$$γ≥0. Such an overlap is a measure of the quality of the signal reconstruction. We explicitly analyze the general case of linear-quadratic family of random mappings and discuss the full $$p_{\infty } (\gamma )$$p∞(γ) curve. When nonlinearity exceeds a certain threshold but redundancy is not yet too big, the replica symmetric solution is necessarily broken in some interval of NSR. We show that encryptions with a nonvanishing linear component permit reconstructions with $$p_{\infty }>0$$p∞>0 for any $$\mu >1$$μ>1 and any $$\gamma <\infty $$γ<∞, with $$p_{\infty }\sim \gamma ^{-1/2}$$p∞∼γ-1/2 as $$\gamma \rightarrow \infty $$γ→∞. In contrast, for the case of purely quadratic nonlinearity, for any ERP $$\mu >1$$μ>1 there exists a threshold NSR value $$\gamma _c(\mu )$$γc(μ) such that $$p_{\infty }=0$$p∞=0 for $$\gamma >\gamma _c(\mu )$$γ>γc(μ) making the reconstruction impossible. The behaviour close to the threshold is given by $$p_{\infty }\sim (\gamma _c-\gamma )^{3/4}$$p∞∼(γc-γ)3/4 and is controlled by the replica symmetry breaking mechanism.

[1]  Wei-Kuo Chen,et al.  Parisi Formula, Disorder Chaos and Fluctuation for the Ground State Energy in the Spherical Mixed p-Spin Models , 2015, 1512.08492.

[2]  Thomas Blumensath,et al.  Compressed Sensing With Nonlinear Observations and Related Nonlinear Optimization Problems , 2012, IEEE Transactions on Information Theory.

[3]  P. L. Doussal,et al.  Topology Trivialization and Large Deviations for the Minimum in the Simplest Random Optimization , 2013, 1304.0024.

[4]  Characteristic polynomials of random Hermitian matrices and Duistermaat–Heckman localisation on non-compact Kähler manifolds , 2002, math-ph/0201045.

[5]  Y. Fyodorov Multifractality and freezing phenomena in random energy landscapes: An introduction , 2009, 0911.2765.

[6]  V. Marčenko,et al.  DISTRIBUTION OF EIGENVALUES FOR SOME SETS OF RANDOM MATRICES , 1967 .

[7]  G. Biroli,et al.  Complex Energy Landscapes in Spiked-Tensor and Simple Glassy Models: Ruggedness, Arrangements of Local Minima, and Phase Transitions , 2018, Physical Review X.

[8]  J. Baik,et al.  Fluctuations of the Free Energy of the Spherical Sherrington–Kirkpatrick Model , 2015, Journal of Statistical Physics.

[9]  Florent Krzakala,et al.  Statistical physics-based reconstruction in compressed sensing , 2011, ArXiv.

[10]  J. Bouchaud,et al.  Statistical mechanics of a single particle in a multiscale random potential: Parisi landscapes in finite-dimensional Euclidean spaces , 2007, 0711.4006.

[11]  A. Guionnet,et al.  A Fourier view on the R-transform and related asymptotics of spherical integrals , 2005 .

[12]  A. Crisanti,et al.  The sphericalp-spin interaction spin glass model: the statics , 1992 .

[13]  Nicolas Sourlas,et al.  Spin-glass models as error-correcting codes , 1989, Nature.

[14]  Yaniv Plan,et al.  The Generalized Lasso With Non-Linear Observations , 2015, IEEE Transactions on Information Theory.

[15]  A. Guionnet,et al.  Large Deviations Asymptotics for Spherical Integrals , 2002 .

[16]  M. Talagrand Free energy of the spherical mean field model , 2006 .

[17]  Yan V Fyodorov Negative moments of characteristic polynomials of random matrices: Ingham–Siegel integral as an alternative to Hubbard–Stratonovich transformation , 2002 .

[18]  Classical particle in a box with random potential: Exploiting rotational symmetry of replicated Hamiltonian , 2006, cond-mat/0610035.

[19]  J. Percus Dimensional reduction of integrals of orthogonal invariants , 1987 .

[20]  J. Kurchan,et al.  Statics and dynamics of infinite-dimensional liquids and glasses: a parallel and compact derivation , 2015, 1512.02186.

[21]  Eliran Subag,et al.  The complexity of spherical p-spin models - a second moment approach , 2015, 1504.02251.

[22]  Ali Bereyhi,et al.  Replica symmetry breaking in compressive sensing , 2017, 2017 Information Theory and Applications Workshop (ITA).

[23]  Ali Bereyhi,et al.  Statistical Mechanics of MAP Estimation: General Replica Ansatz , 2016, IEEE Transactions on Information Theory.

[24]  Andrea Montanari,et al.  Statistical Estimation: From Denoising to Sparse Regression and Hidden Cliques , 2014, ArXiv.

[25]  M. Mézard,et al.  Spin Glass Theory And Beyond: An Introduction To The Replica Method And Its Applications , 1986 .

[26]  Antonio Auffinger,et al.  Existence of Two-Step Replica Symmetry Breaking for the Spherical Mixed p-Spin Glass at Zero Temperature , 2018, Communications in Mathematical Physics.

[27]  On the dynamics of a spherical spin-glass in a magnetic field , 1995, cond-mat/9505149.

[28]  A. Bovier,et al.  Statistical Mechanics of Disordered Systems , 2006 .

[29]  Florent Krzakala,et al.  Statistical physics of inference: thresholds and algorithms , 2015, ArXiv.

[30]  Nicolas Macris,et al.  Phase Transitions, Optimal Errors and Optimality of Message-Passing in Generalized Linear Models , 2017, ArXiv.

[31]  D. Thouless,et al.  Spherical Model of a Spin-Glass , 1976 .

[32]  D. Thouless,et al.  Stability of the Sherrington-Kirkpatrick solution of a spin glass model , 1978 .

[33]  Wei-Kuo Chen,et al.  On the energy landscape of spherical spin glasses , 2017, 1702.08906.

[34]  Giorgio Parisi Constraint Optimization and Statistical Mechanics , 2003, ArXiv.

[35]  Permanental Processes , 2010, 1008.3522.

[36]  Ali Bereyhi,et al.  Maximum-A-Posteriori Signal Recovery with Prior Information: Applications to Compressive Sensing , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[37]  Antonio Auffinger,et al.  Complexity of random smooth functions on the high-dimensional sphere , 2011, 1110.5872.

[38]  Lenka Zdeborová,et al.  Phase diagram of matrix compressed sensing , 2016, Physical review. E.

[39]  D. Panchenko The Sherrington-Kirkpatrick Model , 2013 .

[40]  Y. Fyodorov On statistical mechanics of a single particle in high-dimensional random landscapes , 2008, 0801.0732.

[41]  Wei-Kuo Chen,et al.  Temperature Chaos in Some Spherical Mixed p-Spin Models , 2016, 1608.02478.