Noise-resilient group testing: Limitations and constructions

We study combinatorial group testing schemes for learning d-sparse boolean vectors using highly unreliable disjunctive measurements. We consider an adversarial noise model that only limits the number of false observations, and show that any noise-resilient scheme in this model can only approximately reconstruct the sparse vector. On the positive side, we give a general framework for construction of highly noise-resilient group testing schemes using randomness condensers. Simple randomized instantiations of this construction give non-adaptive measurement schemes, with m = O(d log n) measurements, that allow efficient reconstruction of d-sparse vectors up to O(d) false positives even in the presence of δm false positives and Ω(m/d) false negatives within the measurement outcomes, for any constant δ < 1. None of these parameters can be substantially improved without dramatically affecting the others. Furthermore, we obtain several explicit (and incomparable) constructions, in particular one matching the randomized trade-off but using m = O(d1+o(1) log n) measurements. We also obtain explicit constructions that allow fast reconstruction in time poly(m), which would be sublinear in n for sufficiently sparse vectors.

[1]  Peter Bro Miltersen,et al.  Are bitvectors optimal? , 2000, STOC '00.

[2]  Jaikumar Radhakrishnan,et al.  Tight bounds for depth-two superconcentrators , 1997, Proceedings 38th Annual Symposium on Foundations of Computer Science.

[3]  Piotr Indyk Explicit constructions for compressed sensing of sparse signals , 2008, SODA '08.

[4]  Graham Cormode,et al.  Combinatorial Algorithms for Compressed Sensing , 2006 .

[5]  Alexander Vardy,et al.  Correcting errors beyond the Guruswami-Sudan radius in polynomial time , 2005, 46th Annual IEEE Symposium on Foundations of Computer Science (FOCS'05).

[6]  Ding-Zhu Du,et al.  New Constructions of One- and Two-Stage Pooling Designs , 2008, J. Comput. Biol..

[7]  Enkatesan G Uruswami Unbalanced expanders and randomness extractors from Parvaresh-Vardy codes , 2008 .

[8]  Amnon Ta-Shma,et al.  Loss-less condensers, unbalanced expanders, and extractors , 2001, STOC '01.

[9]  Venkatesan Guruswami,et al.  List decoding of error correcting codes , 2001 .

[10]  Peter L. Hammer,et al.  Discrete Applied Mathematics , 1993 .

[11]  Venkatesan Guruswami,et al.  Hardness Amplification within NP against Deterministic Algorithms , 2008, 2008 23rd Annual IEEE Conference on Computational Complexity.

[12]  Miklós Ruszinkó,et al.  On the upper bound of the size of the r -cover-free families , 1994 .

[13]  Amnon Ta-Shma,et al.  Lossless Condensers, Unbalanced Expanders, And Extractors , 2007, Comb..

[14]  Anthony J. Macula,et al.  Error-correcting Nonadaptive Group Testing with de-disjunct Matrices , 1997, Discret. Appl. Math..

[15]  Emanuel Knill,et al.  Lower bounds for identifying subset members with subset queries , 1994, SODA '95.

[16]  Miklós Ruszinkó,et al.  On the Upper Bound of the Size of the R-Cover-Free Families , 1993, Proceedings. IEEE International Symposium on Information Theory.

[17]  Rahul Santhanam,et al.  Uniform Hardness Amplification in NP via Monotone Codes , 2006, Electron. Colloquium Comput. Complex..

[18]  Dariusz R. Kowalski,et al.  Almost Optimal Explicit Selectors , 2005, FCT.

[19]  D. Du,et al.  Combinatorial Group Testing and Its Applications , 1993 .

[20]  GuruswamiVenkatesan,et al.  Unbalanced expanders and randomness extractors from Parvaresh--Vardy codes , 2009 .

[21]  M. Sobel,et al.  Group testing to eliminate efficiently all defectives in a binomial sample , 1959 .

[22]  Venkatesan Guruswami,et al.  Concatenated codes can achieve list-decoding capacity , 2008, SODA '08.

[23]  Arkadii G. D'yachkov,et al.  A survey of superimposed code theory , 1983 .

[24]  Amnon Ta-Shma,et al.  Extractor codes , 2001, IEEE Transactions on Information Theory.

[25]  Ran Raz,et al.  Extracting all the randomness and reducing the error in Trevisan's extractors , 1999, STOC '99.

[26]  A. Robert Calderbank,et al.  Efficient and Robust Compressed Sensing using High-Quality Expander Graphs , 2008, ArXiv.

[27]  Graham Cormode,et al.  What's hot and what's not: tracking most frequent items dynamically , 2003, TODS.

[28]  A. Macula Probabilistic nonadaptive group testing in the presence of errors and DNA library screening , 1999 .

[29]  Anthony J. Macula,et al.  Probabilistic Nonadaptive and Two-Stage Group Testing with Relatively Small Pools and DNA Library Screening , 1998, J. Comb. Optim..

[30]  David Eppstein,et al.  Improved Combinatorial Group Testing Algorithms for Real-World Problem Sizes , 2005, SIAM J. Comput..

[31]  Sanjeev Arora,et al.  Computational Complexity: A Modern Approach , 2009 .

[32]  Christopher Umans,et al.  Simple extractors for all min-entropies and a new pseudorandom generator , 2005, JACM.

[33]  Annalisa De Bonis,et al.  Optimal Two-Stage Algorithms for Group Testing Problems , 2005, SIAM J. Comput..

[34]  Jack K. Wolf,et al.  Born again group testing: Multiaccess communications , 1985, IEEE Trans. Inf. Theory.

[35]  Andreas Blass,et al.  Pairwise Testing , 2002, Bull. EATCS.

[36]  Amnon Ta-Shma,et al.  Storing information with extractors , 2002, Inf. Process. Lett..

[37]  Ding-Zhu Du,et al.  A survey on combinatorial group testing algorithms with applications to DNA Library Screening , 1999, Discrete Mathematical Problems with Medical Applications.

[38]  Omer Reingold,et al.  Randomness Conductors and Constant-Degree Expansion Beyond the Degree / 2 Barrier , 2001 .

[39]  Richard C. Singleton,et al.  Nonrandom binary superimposed codes , 1964, IEEE Trans. Inf. Theory.

[40]  J. Mandell,et al.  Maximally Efficient Two‐Stage Screening , 2000, Biometrics.

[41]  Zoltán Füredi On r-Cover-free Families , 1996, J. Comb. Theory, Ser. A.

[42]  D. Du,et al.  Pooling Designs And Nonadaptive Group Testing: Important Tools For Dna Sequencing , 2006 .

[43]  Alexander Schliep,et al.  Group testing with DNA chips: generating designs and decoding experiments , 2003, Computational Systems Bioinformatics. CSB2003. Proceedings of the 2003 IEEE Bioinformatics Conference. CSB2003.

[44]  Emanuel Knill,et al.  Non-adaptive Group Testing in the Presence of Errors , 1998, Discret. Appl. Math..

[45]  Piotr Indyk,et al.  Explicit constructions of selectors and related combinatorial structures, with applications , 2002, SODA '02.

[46]  Avi Wigderson,et al.  Randomness conductors and constant-degree lossless expanders , 2002, STOC '02.

[47]  Ely Porat,et al.  k -Mismatch with Don't Cares , 2007, ESA.

[48]  Graham Cormode,et al.  What's hot and what's not: tracking most frequent items dynamically , 2003, PODS '03.

[49]  A. Sterrett On the Detection of Defective Members of Large Populations , 1957 .

[50]  Richard E. Ladner,et al.  Group testing for image compression , 2000, Proceedings DCC 2000. Data Compression Conference.

[51]  Luca Trevisan,et al.  Extractors and pseudorandom generators , 2001, JACM.

[52]  Pavel A. Pevzner,et al.  Towards DNA Sequencing Chips , 1994, MFCS.