Randomized Approximate Nearest Neighbor Search with Limited Adaptivity

We study the complexity of parallel data structures for approximate nearest neighbor search in d-dimensional Hamming space {0,1}d. A classic model for static data structures is the cell-probe model [27]. We consider a cell-probe model with limited adaptivity, where given a k≥1, a query is resolved by making at most k rounds of parallel memory accesses to the data structure. We give two randomized algorithms that solve the approximate nearest neighbor search using k rounds of parallel memory accesses: —a simple algorithm with O(k(logd)1/k) total number of memory accesses for all k≥1; —an algorithm with O(k+(1/klogd)O(1/k)) total number of memory accesses for all sufficiently large k. Both algorithms use data structures of polynomial size. We prove an Ω(1/k(logd)1/k) lower bound for the total number of memory accesses for any randomized algorithm solving the approximate nearest neighbor search within k ≤log logd/2log log log d rounds of parallel memory accesses on any data structures of polynomial size. This lower bound shows that our first algorithm is asymptotically optimal when k=O(1). And our second algorithm achieves the asymptotically optimal tradeoff between number of rounds and total number of memory accesses. In the extremal case, when k=O(log logd/log log log d) is big enough, our second algorithm matches the Θ(log logd/log log log d) tight bound for fully adaptive algorithms for approximate nearest neighbor search in [11].

[1]  Bernard Chazelle,et al.  A lower bound on the complexity of approximate nearest-neighbor searching on the Hamming cube , 1999, STOC '99.

[2]  Mikkel Thorup,et al.  Higher Lower Bounds for Near-Neighbor and Further Rich Problems , 2006, 2006 47th Annual IEEE Symposium on Foundations of Computer Science (FOCS'06).

[3]  Yuval Rabani,et al.  Tighter bounds for nearest neighbor search and related problems in the cell probe model , 2000, STOC '00.

[4]  Alexandr Andoni,et al.  Optimal Data-Dependent Hashing for Approximate Near Neighbors , 2015, STOC.

[5]  A. Razborov Communication Complexity , 2011 .

[6]  Andrew Chi-Chih Yao,et al.  Should Tables Be Sorted? , 1981, JACM.

[7]  Alexandr Andoni,et al.  Near-Optimal Hashing Algorithms for Approximate Nearest Neighbor in High Dimensions , 2006, 2006 47th Annual IEEE Symposium on Foundations of Computer Science (FOCS'06).

[8]  Peter Bro Miltersen,et al.  On data structures and asymmetric communication complexity , 1994, STOC '95.

[9]  Piotr Indyk,et al.  Approximate nearest neighbors: towards removing the curse of dimensionality , 1998, STOC '98.

[10]  Allan Borodin,et al.  Lower bounds for high dimensional nearest neighbor search and related problems , 1999, STOC '99.

[11]  Yuval Rabani,et al.  Cell-probe lower bounds for the partial match problem , 2004, J. Comput. Syst. Sci..

[12]  Rina Panigrahy,et al.  A Geometric Approach to Lower Bounds for Approximate Near-Neighbor Search and Partial Match , 2008, 2008 49th Annual IEEE Symposium on Foundations of Computer Science.

[13]  Yitong Yin,et al.  Certificates in Data Structures , 2014, ICALP.

[14]  Alexandr Andoni,et al.  Beyond Locality-Sensitive Hashing , 2013, SODA.

[15]  Ding Liu A strong lower bound for approximate nearest neighbor searching , 2004, Inf. Process. Lett..

[16]  Miklós Ajtai,et al.  A lower bound for finding predecessors in Yao's cell probe model , 1988, Comb..

[17]  Amit Chakrabarti,et al.  An Optimal Randomized Cell Probe Lower Bound for Approximate Nearest Neighbor Searching , 2010, SIAM J. Comput..

[18]  Piotr Indyk,et al.  Nearest Neighbors in High-Dimensional Spaces , 2004, Handbook of Discrete and Computational Geometry, 2nd Ed..

[19]  Ilan Newman,et al.  Private vs. Common Random Bits in Communication Complexity , 1991, Inf. Process. Lett..

[20]  David Eisenstat,et al.  Low-contention data structures , 2010, SPAA '10.

[21]  Rina Panigrahy,et al.  Lower Bounds on Near Neighbor Search via Metric Expansion , 2010, 2010 IEEE 51st Annual Symposium on Foundations of Computer Science.

[22]  Joshua Brody,et al.  Adapt or Die: Polynomial Lower Bounds for Non-Adaptive Dynamic Data Structures , 2015, Theory Comput..

[23]  Srinivasan Venkatesh,et al.  Lower bounds for predecessor searching in the cell probe model , 2003, J. Comput. Syst. Sci..

[24]  Yitong Yin Simple Average-Case Lower Bounds for Approximate Near-Neighbor from Isoperimetric Inequalities , 2016, ICALP.

[25]  Andrew Chi-Chih Yao,et al.  Some complexity questions related to distributive computing(Preliminary Report) , 1979, STOC.

[26]  Rafail Ostrovsky,et al.  Efficient search for approximate nearest neighbor in high dimensional spaces , 1998, STOC '98.

[27]  Yitong Yin,et al.  Randomized Approximate Nearest Neighbor Search with Limited Adaptivity , 2016, SPAA.