Decoding From Pooled Data: Phase Transitions of Message Passing

We consider the problem of decoding a discrete signal of categorical variables from the observation of several histograms of pooled subsets of it. We present an approximate message passing (AMP) algorithm for recovering the signal in the <italic>random dense</italic> setting where each observed histogram involves a random subset of entries of size proportional to <inline-formula> <tex-math notation="LaTeX">$n$ </tex-math></inline-formula>. We characterize the performance of the algorithm in the asymptotic regime where the number of observations <inline-formula> <tex-math notation="LaTeX">$m$ </tex-math></inline-formula> tends infinity proportionally to <inline-formula> <tex-math notation="LaTeX">$n$ </tex-math></inline-formula> by deriving the corresponding state evolution (SE) equations and studying their dynamics. We initiate the analysis of the multi-dimensional SE dynamics by proving their convergence to a fixed point, along with some further properties of the iterates. The analysis reveals sharp phase transition phenomena where the behavior of AMP changes from exact recovery to weak correlation with the signal, as <inline-formula> <tex-math notation="LaTeX">$m/n$ </tex-math></inline-formula> crosses a threshold. We derive formulae for the threshold in some special cases and show that they accurately match experimental behavior.

[1]  R. Palmer,et al.  Solution of 'Solvable model of a spin glass' , 1977 .

[2]  M. Mézard,et al.  Spin Glass Theory and Beyond , 1987 .

[3]  M. O’Donovan,et al.  DNA Pooling: a tool for large-scale association studies , 2002, Nature Reviews Genetics.

[4]  Sergio Verdú,et al.  Randomly spread CDMA: asymptotics via statistical physics , 2005, IEEE Transactions on Information Theory.

[5]  Andrea Montanari,et al.  Message-passing algorithms for compressed sensing , 2009, Proceedings of the National Academy of Sciences.

[6]  Andrea Montanari,et al.  The dynamics of message passing on dense graphs, with applications to compressed sensing , 2010, ISIT.

[7]  Andrea Montanari,et al.  Universality in polytope phase transitions and iterative algorithms , 2012, 2012 IEEE International Symposium on Information Theory Proceedings.

[8]  Sundeep Rangan,et al.  Hybrid generalized approximate message passing with applications to structured sparsity , 2012, 2012 IEEE International Symposium on Information Theory Proceedings.

[9]  Florent Krzakala,et al.  Statistical physics of inference: thresholds and algorithms , 2015, ArXiv.

[10]  Kwang-Cheng Chen,et al.  Data extraction via histogram and arithmetic mean queries: Fundamental limits and algorithms , 2016, 2016 IEEE International Symposium on Information Theory (ISIT).

[11]  Michael I. Jordan,et al.  Decoding from Pooled Data: Sharp Information-Theoretic Bounds , 2016, SIAM J. Math. Data Sci..

[12]  Tsuyoshi Murata,et al.  {m , 1934, ACML.