Nearly Optimal Sparse Group Testing

Group testing is the process of pooling arbitrary subsets from a set of <inline-formula> <tex-math notation="LaTeX">$ {n}$ </tex-math></inline-formula> items so as to identify, with a minimal number of tests, a “small” subset of <inline-formula> <tex-math notation="LaTeX">$ {d}$ </tex-math></inline-formula> defective items. In “classical” non-adaptive group testing, it is known that when <inline-formula> <tex-math notation="LaTeX">$ {d}$ </tex-math></inline-formula> is substantially smaller than <inline-formula> <tex-math notation="LaTeX">$ {n}$ </tex-math></inline-formula>, <inline-formula> <tex-math notation="LaTeX">$\Theta ( {d}\log ( {n}))$ </tex-math></inline-formula> tests are both information-theoretically necessary and sufficient to guarantee recovery with high probability. Group testing schemes in the literature that meet this bound require most items to be tested <inline-formula> <tex-math notation="LaTeX">$ {\Omega }(\log ( {n}))$ </tex-math></inline-formula> times, and most tests to incorporate <inline-formula> <tex-math notation="LaTeX">$ {\Omega }({{n/d}})$ </tex-math></inline-formula> items. Motivated by physical considerations, we study group testing models in which the testing procedure is constrained to be “sparse.” Specifically, we consider (separately) scenarios in which 1) items are finitely divisible and hence may participate in at most <inline-formula> <tex-math notation="LaTeX">$ {\gamma } \in {o}(\log ( {n}))$ </tex-math></inline-formula> tests; or 2) tests are size-constrained to pool no more than <inline-formula> <tex-math notation="LaTeX">$\rho \in {o}({{n/d}})$ </tex-math></inline-formula> items per test. For both scenarios, we provide information-theoretic lower bounds on the number of tests required to guarantee high probability recovery. In particular, one of our main results shows that <inline-formula> <tex-math notation="LaTeX">$ {\gamma }$ </tex-math></inline-formula>-finite divisibility of items forces <italic>any</italic> non-adaptive group testing algorithm with the probability of recovery error at most <inline-formula> <tex-math notation="LaTeX">$ {\epsilon }$ </tex-math></inline-formula> to perform at least <inline-formula> <tex-math notation="LaTeX">$ {\gamma } {d}({ {n/d}})^{({1}-{5} {\epsilon })/ {\gamma }}$ </tex-math></inline-formula> tests. Analogously, for <inline-formula> <tex-math notation="LaTeX">$ {\rho }$ </tex-math></inline-formula>-sized constrained tests, we show an information-theoretic lower bound of <inline-formula> <tex-math notation="LaTeX">$ {\Omega }( {n}/ {\rho })$ </tex-math></inline-formula> tests for high-probability recovery–hence in both settings the number of tests required grows dramatically (relative to the classical setting) as a function of <inline-formula> <tex-math notation="LaTeX">$ {n}$ </tex-math></inline-formula>. In both scenarios, we provide both randomized constructions and explicit constructions of designs with computationally efficient reconstruction algorithms that require a number of tests that is optimal up to constant or small polynomial factors in some regimes of <inline-formula> <tex-math notation="LaTeX">${{n, d,}} {\gamma }$ </tex-math></inline-formula>, and <inline-formula> <tex-math notation="LaTeX">$ {\rho }$ </tex-math></inline-formula>. The randomized design/reconstruction algorithm in the <inline-formula> <tex-math notation="LaTeX">$ {\rho }$ </tex-math></inline-formula>-sized test scenario is <italic>universal</italic>–independent of the value of <inline-formula> <tex-math notation="LaTeX">$ {d}$ </tex-math></inline-formula>, as long as <inline-formula> <tex-math notation="LaTeX">$ {\rho } \in {o}({\textbf {n/d}})$ </tex-math></inline-formula>. We also investigate the effect of unreliability/noise in test outcomes, and show that whereas the impact of noise in test outcomes can be obviated with a small (constant factor) penalty in the number of tests in the <inline-formula> <tex-math notation="LaTeX">$ {\rho }$ </tex-math></inline-formula>-sized tests scenario, there is <italic>no</italic> group-testing procedure, <italic>regardless</italic> of the number of tests, that can combat noise in the <inline-formula> <tex-math notation="LaTeX">$ {\gamma }$ </tex-math></inline-formula>-divisible scenario.