On Closest Pair in Euclidean Metric: Monochromatic is as Hard as Bichromatic

Given a set of n points in R^d, the (monochromatic) Closest Pair problem asks to find a pair of distinct points in the set that are closest in the l_p-metric. Closest Pair is a fundamental problem in Computational Geometry and understanding its fine-grained complexity in the Euclidean metric when d=omega(log n) was raised as an open question in recent works (Abboud-Rubinstein-Williams [FOCS'17], Williams [SODA'18], David-Karthik-Laekhanukit [SoCG'18]). In this paper, we show that for every p in R_{>= 1} cup {0}, under the Strong Exponential Time Hypothesis (SETH), for every epsilon>0, the following holds: - No algorithm running in time O(n^{2-epsilon}) can solve the Closest Pair problem in d=(log n)^{Omega_{epsilon}(1)} dimensions in the l_p-metric. - There exists delta = delta(epsilon)>0 and c = c(epsilon)>= 1 such that no algorithm running in time O(n^{1.5-epsilon}) can approximate Closest Pair problem to a factor of (1+delta) in d >= c log n dimensions in the l_p-metric. In particular, our first result is shown by establishing the computational equivalence of the bichromatic Closest Pair problem and the (monochromatic) Closest Pair problem (up to n^{epsilon} factor in the running time) for d=(log n)^{Omega_epsilon(1)} dimensions. Additionally, under SETH, we rule out nearly-polynomial factor approximation algorithms running in subquadratic time for the (monochromatic) Maximum Inner Product problem where we are given a set of n points in n^{o(1)}-dimensional Euclidean space and are required to find a pair of distinct points in the set that maximize the inner product. At the heart of all our proofs is the construction of a dense bipartite graph with low contact dimension, i.e., we construct a balanced bipartite graph on n vertices with n^{2-epsilon} edges whose vertices can be realized as points in a (log n)^{Omega_epsilon(1)}-dimensional Euclidean space such that every pair of vertices which have an edge in the graph are at distance exactly 1 and every other pair of vertices are at distance greater than 1. This graph construction is inspired by the construction of locally dense codes introduced by Dumer-Miccancio-Sudan [IEEE Trans. Inf. Theory'03].

[1]  Peter Frankl,et al.  Embedding the n-cube in Lower Dimensions , 1986, Eur. J. Comb..

[2]  Bundit Laekhanukit,et al.  On the Complexity of Closest Pair via Polar-Pair of Point-Sets , 2016, SoCG.

[3]  Nathan Linial,et al.  Monotone maps, sphericity and bounded second eigenvalue , 2005, J. Comb. Theory, Ser. B.

[4]  Gregory Valiant Finding Correlations in Subquadratic Time, with Applications to Learning Parities and the Closest Pair Problem , 2015, J. ACM.

[5]  Bingkai Lin,et al.  The Parameterized Complexity of k-Biclique , 2014, SODA.

[6]  Michael Ian Shamos,et al.  Computational geometry: an introduction , 1985 .

[7]  Udi Manber,et al.  Introduction to algorithms - a creative approach , 1989 .

[8]  Otfried Cheong,et al.  Euclidean minimum spanning trees and bichromatic closest pairs , 1990, SCG '90.

[9]  H. Stichtenoth,et al.  On the Asymptotic Behaviour of Some Towers of Function Fields over Finite Fields , 1996 .

[10]  Richard Ryan Williams,et al.  Distributed PCP Theorems for Hardness of Approximation in P , 2017, 2017 IEEE 58th Annual Symposium on Foundations of Computer Science (FOCS).

[11]  Russell Impagliazzo,et al.  Complexity of k-SAT , 1999, Proceedings. Fourteenth Annual IEEE Conference on Computational Complexity (Formerly: Structure in Complexity Theory Conference) (Cat.No.99CB36317).

[12]  Bengt J. Nilsson,et al.  Minimum Spanning Trees in d Dimensions , 1999, Nord. J. Comput..

[13]  Tomislav Hengl,et al.  Finding the right pixel size , 2006, Comput. Geosci..

[14]  Madhu Sudan,et al.  Hardness of approximating the minimum distance of a linear code , 1999, IEEE Trans. Inf. Theory.

[15]  Daniele Micciancio,et al.  Locally Dense Codes , 2014, 2014 IEEE 29th Conference on Computational Complexity (CCC).

[16]  S LuekerGeorge Improved bounds on the average length of longest common subsequences , 2009 .

[17]  Charles T. Zahn,et al.  Graph-Theoretical Methods for Detecting and Describing Gestalt Clusters , 1971, IEEE Transactions on Computers.

[18]  Edith Cohen,et al.  Approximating matrix multiplication for pattern recognition tasks , 1997, SODA '97.

[19]  Bernard Chazelle,et al.  The Fast Johnson--Lindenstrauss Transform and Approximate Nearest Neighbors , 2009, SIAM J. Comput..

[20]  Michael Ben-Or,et al.  Lower bounds for algebraic computation trees , 1983, STOC.

[21]  Lijie Chen,et al.  Toward Super-Polynomial Size Lower Bounds for Depth-Two Threshold Circuits , 2018, ArXiv.

[22]  Jon Louis Bentley,et al.  Multidimensional divide-and-conquer , 1980, CACM.

[23]  Timothy M. Chan,et al.  Polynomial Representations of Threshold Functions and Algorithmic Applications , 2016, 2016 IEEE 57th Annual Symposium on Foundations of Computer Science (FOCS).

[24]  Michael R. Fellows,et al.  Fundamentals of Parameterized Complexity , 2013 .

[25]  Yi Wu,et al.  Optimal Lower Bounds for Locality-Sensitive Hashing (Except When q is Tiny) , 2014, TOCT.

[26]  Qi Cheng,et al.  A Deterministic Reduction for the Gap Minimum Distance Problem , 2012, IEEE Transactions on Information Theory.

[27]  Raymond Chi-Wing Wong,et al.  On Efficient Spatial Matching , 2007, VLDB.

[28]  Cem Güneri Algebraic geometric codes: basic notions , 2008 .

[29]  Pasin Manurangsi,et al.  On the parameterized complexity of approximating dominating set , 2017, Electron. Colloquium Comput. Complex..

[30]  Yun Kuen Cheung,et al.  Lecture 5 : k-wise Independent Hashing and Applications , 2013 .

[31]  A. Paz Probabilistic algorithms , 2003 .

[32]  Ronald L. Rivest,et al.  Introduction to Algorithms, third edition , 2009 .

[33]  J. Pach Decomposition of multiple packing and covering , 1980 .

[34]  Michael Ian Shamos,et al.  Closest-point problems , 1975, 16th Annual Symposium on Foundations of Computer Science (sfcs 1975).

[35]  Ryan Williams,et al.  On the Difference Between Closest, Furthest, and Orthogonal Pairs: Nearly-Linear vs Barely-Subquadratic Complexity , 2017, SODA.

[36]  J. Reiterman,et al.  Embeddings of graphs in euclidean spaces , 1989, Discret. Comput. Geom..

[37]  Bingkai Lin,et al.  The Parameterized Complexity of the k-Biclique Problem , 2018, J. ACM.

[38]  Klaus Sutner Probabilistic Algorithms , 2017 .

[39]  George S. Lueker,et al.  Improved bounds on the average length of longest common subsequences , 2003, JACM.

[40]  Russell Impagliazzo,et al.  A duality between clause width and clause density for SAT , 2006, 21st Annual IEEE Conference on Computational Complexity (CCC'06).

[41]  Hiroshi Maehara Dispersed points and geometric embedding of complete bipartite graphs , 1991, Discret. Comput. Geom..

[42]  Hiroshi Maehara Contact patterns of equal nonoverlapping spheres , 1985, Graphs Comb..

[43]  Alexander Barg,et al.  Linear Codes with Exponentially Many Light Vectors , 2001, J. Comb. Theory A.

[44]  Richard C. Singleton,et al.  Maximum distance q -nary codes , 1964, IEEE Trans. Inf. Theory.

[45]  Ryan Williams,et al.  Probabilistic Polynomials and Hamming Nearest Neighbors , 2015, 2015 IEEE 56th Annual Symposium on Foundations of Computer Science.

[46]  Rajeev Motwani,et al.  Lower bounds on locality sensitive hashing , 2005, SCG '06.

[47]  Micha Sharir,et al.  Dominance Products and Faster Algorithms for High-Dimensional Closest Pair under L∞ , 2016, ArXiv.

[48]  Jon M. Kleinberg,et al.  Two algorithms for nearest-neighbor search in high dimensions , 1997, STOC '97.

[49]  Lijie Chen,et al.  On The Hardness of Approximate and Exact (Bichromatic) Maximum Inner Product , 2018, Electron. Colloquium Comput. Complex..

[50]  Henning Stichtenoth,et al.  Algebraic function fields and codes , 1993, Universitext.

[51]  Moshe Lewenstein,et al.  Closest Pair Problems in Very High Dimensions , 2004, ICALP.

[52]  Pasin Manurangsi,et al.  Parameterized Intractability of Even Set and Shortest Vector Problem from Gap-ETH , 2018, Electron. Colloquium Comput. Complex..

[53]  Luca Trevisan,et al.  From Gap-ETH to FPT-Inapproximability: Clique, Dominating Set, and More , 2017, 2017 IEEE 58th Annual Symposium on Foundations of Computer Science (FOCS).

[54]  M. Kubát An Introduction to Machine Learning , 2017, Springer International Publishing.

[55]  Piotr Indyk,et al.  Approximate nearest neighbors: towards removing the curse of dimensionality , 1998, STOC '98.

[56]  Ryan Williams,et al.  A new algorithm for optimal 2-constraint satisfaction and its implications , 2005, Theor. Comput. Sci..

[57]  Piotr Indyk Dimensionality reduction techniques for proximity problems , 2000, SODA '00.

[58]  Klaus H. Hinrichs,et al.  Plane-Sweep Solves the Closest Pair Problem Elegantly , 1988, Inf. Process. Lett..

[59]  Serge Vluaduct Lattices with exponentially large kissing numbers , 2018, Moscow Journal of Combinatorics and Number Theory.

[60]  W. B. Johnson,et al.  Extensions of Lipschitz mappings into Hilbert space , 1984 .

[61]  E. Gilbert A comparison of signalling alphabets , 1952 .

[62]  O. Antoine,et al.  Theory of Error-correcting Codes , 2022 .

[63]  Peter Frankl,et al.  On the contact dimensions of graphs , 1988, Discret. Comput. Geom..

[64]  I. Reed,et al.  Polynomial Codes Over Certain Finite Fields , 1960 .

[65]  Michael Ian Shamos,et al.  Divide-and-conquer in multidimensional space , 1976, STOC '76.

[66]  Neil J. A. Sloane,et al.  The theory of error-correcting codes (north-holland , 1977 .

[67]  Aviad Rubinstein,et al.  Hardness of approximate nearest neighbor search , 2018, STOC.

[68]  Andrew Chi-Chih Yao Lower bounds for algebraic computation trees with integer inputs , 1989, 30th Annual Symposium on Foundations of Computer Science.

[69]  François Le Gall,et al.  Powers of tensors and fast matrix multiplication , 2014, ISSAC.

[70]  Ilya P. Razenshteyn High-dimensional similarity search and sketching: algorithms and hardness , 2017 .

[71]  Russell Impagliazzo,et al.  Which Problems Have Strongly Exponential Complexity? , 2001, J. Comput. Syst. Sci..

[72]  Samir Khuller,et al.  A Simple Randomized Sieve Algorithm for the Closest-Pair Problem , 1995, Inf. Comput..

[73]  Ryan Williams,et al.  An Equivalence Class for Orthogonal Vectors , 2018, SODA.

[74]  V. V. Williams ON SOME FINE-GRAINED QUESTIONS IN ALGORITHMS AND COMPLEXITY , 2019, Proceedings of the International Congress of Mathematicians (ICM 2018).