Noise-resilient group testing: Limitations and constructions

We study combinatorial group testing schemes for learning d-sparse Boolean vectors using highly unreliable disjunctive measurements. We consider an adversarial noise model that only limits the number of false observations, and show that any noise-resilient scheme in this model can only approximately reconstruct the sparse vector. On the positive side, we take this barrier to our advantage and show that approximate reconstruction (within a satisfactory degree of approximation) allows us to break the information theoretic lower bound of @[email protected]?(d^2logn) that is known for exact reconstruction of d-sparse vectors of length n via non-adaptive measurements, by a multiplicative factor @[email protected]?(d). Specifically, we give simple randomized constructions of non-adaptive measurement schemes, with m=O(dlogn) measurements, that allow efficient reconstruction ofd-sparse vectors up to O(d) false positives even in the presence of @dm false positives and O(m/d) false negatives within the measurement outcomes, for any constant @d<1. We show that, information theoretically, none of these parameters can be substantially improved without dramatically affecting the others. Furthermore, we obtain several explicit constructions, in particular one matching the randomized trade-off but using m=O(d^1^+^o^(^1^)logn) measurements. We also obtain explicit constructions that allow fast reconstruction in time poly(m), which would be sublinear in n for sufficiently sparse vectors. The main tool used in our construction is the list-decoding view of randomness condensers and extractors. An immediate consequence of our result is an adaptive scheme that runs in only two non-adaptive rounds and exactly reconstructs any d-sparse vector using a total O(dlogn) measurements, a task that would be impossible in one round and fairly easy in O(log(n/d)) or d rounds.

[1]  Venkatesan Guruswami,et al.  Unbalanced expanders and randomness extractors from Parvaresh--Vardy codes , 2009, JACM.

[2]  D. Du,et al.  Combinatorial Group Testing and Its Applications , 1993 .

[3]  Miklós Ruszinkó,et al.  On the upper bound of the size of the r -cover-free families , 1994 .

[4]  Amnon Ta-Shma,et al.  Extractor codes , 2004, IEEE Trans. Inf. Theory.

[5]  J. Mandell,et al.  Maximally Efficient Two‐Stage Screening , 2000, Biometrics.

[6]  Shachar Lovett,et al.  Subspace Evasive Sets , 2011, Electron. Colloquium Comput. Complex..

[7]  Anthony J. Macula,et al.  Error-correcting Nonadaptive Group Testing with de-disjunct Matrices , 1997, Discret. Appl. Math..

[8]  Emanuel Knill,et al.  Lower bounds for identifying subset members with subset queries , 1994, SODA '95.

[9]  A. Macula Probabilistic nonadaptive group testing in the presence of errors and DNA library screening , 1999 .

[10]  Anthony J. Macula,et al.  Probabilistic Nonadaptive and Two-Stage Group Testing with Relatively Small Pools and DNA Library Screening , 1998, J. Comb. Optim..

[11]  Venkatesan Guruswami,et al.  Explicit Codes Achieving List Decoding Capacity: Error-Correction With Optimal Redundancy , 2005, IEEE Transactions on Information Theory.

[12]  Graham Cormode,et al.  What's hot and what's not: tracking most frequent items dynamically , 2003, TODS.

[13]  Ely Porat,et al.  Explicit Nonadaptive Combinatorial Group Testing Schemes , 2011, IEEE Trans. Inf. Theory.

[14]  Venkatesan Guruswami,et al.  Linear-Algebraic List Decoding of Folded Reed-Solomon Codes , 2011, 2011 IEEE 26th Annual Conference on Computational Complexity.

[15]  Ding-Zhu Du,et al.  A survey on combinatorial group testing algorithms with applications to DNA Library Screening , 1999, Discrete Mathematical Problems with Medical Applications.

[16]  Dariusz R. Kowalski,et al.  Almost Optimal Explicit Selectors , 2005, FCT.

[17]  R. Dorfman The Detection of Defective Members of Large Populations , 1943 .

[18]  Jaikumar Radhakrishnan,et al.  Tight bounds for depth-two superconcentrators , 1997, Proceedings 38th Annual Symposium on Foundations of Computer Science.

[19]  Arkadii G. D'yachkov,et al.  A survey of superimposed code theory , 1983 .

[20]  Salil P. Vadhan,et al.  The unified theory of pseudorandomness , 2010 .

[21]  Arkadii G. D'yachkov,et al.  New constructions of superimposed codes , 2000, IEEE Trans. Inf. Theory.

[22]  Sanjeev Arora,et al.  Computational Complexity: A Modern Approach , 2009 .

[23]  Ran Raz,et al.  Extracting all the randomness and reducing the error in Trevisan's extractors , 1999, STOC '99.

[24]  Annalisa De Bonis,et al.  Optimal Two-Stage Algorithms for Group Testing Problems , 2005, SIAM J. Comput..

[25]  Luca Trevisan,et al.  Extractors and pseudorandom generators , 2001, JACM.

[26]  Richard C. Singleton,et al.  Nonrandom binary superimposed codes , 1964, IEEE Trans. Inf. Theory.

[27]  Ely Porat,et al.  Pattern matching with don't cares and few errors , 2010, J. Comput. Syst. Sci..

[28]  Pavel A. Pevzner,et al.  Towards DNA Sequencing Chips , 1994, MFCS.

[29]  Jack K. Wolf,et al.  Born again group testing: Multiaccess communications , 1985, IEEE Trans. Inf. Theory.

[30]  Venkatesan Guruswami,et al.  Hardness amplification within NP against deterministic algorithms , 2011, J. Comput. Syst. Sci..

[31]  Alexander Schliep,et al.  Group testing with DNA chips: generating designs and decoding experiments , 2003, Computational Systems Bioinformatics. CSB2003. Proceedings of the 2003 IEEE Bioinformatics Conference. CSB2003.

[32]  Emanuel Knill,et al.  Non-adaptive Group Testing in the Presence of Errors , 1998, Discret. Appl. Math..

[33]  Venkatesan Guruswami,et al.  Concatenated codes can achieve list-decoding capacity , 2008, SODA '08.

[34]  A.G. D'yachkov,et al.  Superimposed codes for multiple accessing of the OR-channel , 1998, Proceedings. 1998 IEEE International Symposium on Information Theory (Cat. No.98CH36252).

[35]  Richard E. Ladner,et al.  Group testing for image compression , 2000, Proceedings DCC 2000. Data Compression Conference.

[36]  Venkatesan Guruswami,et al.  Linear-Time List Decoding in Error-Free Settings: (Extended Abstract) , 2004, ICALP.

[37]  Piotr Indyk,et al.  Explicit constructions of selectors and related combinatorial structures, with applications , 2002, SODA '02.

[38]  Avi Wigderson,et al.  Randomness conductors and constant-degree lossless expanders , 2002, STOC '02.

[39]  Venkatesan Guruswami,et al.  Folded codes from function field towers and improved optimal rate list decoding , 2012, STOC '12.

[40]  Zoltán Füredi On r-Cover-free Families , 1996, J. Comb. Theory, Ser. A.

[41]  D. Du,et al.  Pooling Designs And Nonadaptive Group Testing: Important Tools For Dna Sequencing , 2006 .

[42]  Mahdi Cheraghchi,et al.  Applications of Derandomization Theory in Coding , 2011, ArXiv.

[43]  Ding-Zhu Du,et al.  New Constructions of One- and Two-Stage Pooling Designs , 2008, J. Comput. Biol..

[44]  Mahdi Cheraghchi,et al.  Noise-resilient group testing: Limitations and constructions , 2008, Discret. Appl. Math..

[45]  Amnon Ta-Shma,et al.  Loss-less condensers, unbalanced expanders, and extractors , 2001, STOC '01.

[46]  Arkadii G. D'yachkov,et al.  New Applications and Results of Superimposed Code Theory Arising from the Potentialities of Molecular Biology , 2000 .

[47]  M. Sobel,et al.  Group testing to eliminate efficiently all defectives in a binomial sample , 1959 .

[48]  Atri Rudra,et al.  Efficiently decodable non-adaptive group testing , 2010, SODA '10.

[49]  Graham Cormode,et al.  Combinatorial Algorithms for Compressed Sensing , 2006 .

[50]  Alexander Vardy,et al.  Correcting errors beyond the Guruswami-Sudan radius in polynomial time , 2005, 46th Annual IEEE Symposium on Foundations of Computer Science (FOCS'05).

[51]  Atri Rudra,et al.  Efficiently Decodable Error-Correcting List Disjunct Matrices and Applications - (Extended Abstract) , 2011, ICALP.

[52]  Venkatesan Guruswami,et al.  List decoding of error correcting codes , 2001 .

[53]  Rahul Santhanam,et al.  Uniform Hardness Amplification in NP via Monotone Codes , 2006, Electron. Colloquium Comput. Complex..

[54]  David Eppstein,et al.  Improved Combinatorial Group Testing Algorithms for Real-World Problem Sizes , 2005, SIAM J. Comput..

[55]  Christopher Umans,et al.  Simple extractors for all min-entropies and a new pseudorandom generator , 2005, JACM.