Vector-Matrix-Vector Queries for Solving Linear Algebra, Statistics, and Graph Problems

We consider the general problem of learning about a matrix through vector-matrix-vector queries. These queries provide the value of $\boldsymbol{u}^{\mathrm{T}}\boldsymbol{M}\boldsymbol{v}$ over a fixed field $\mathbb{F}$ for a specified pair of vectors $\boldsymbol{u},\boldsymbol{v} \in \mathbb{F}^n$. To motivate these queries, we observe that they generalize many previously studied models, such as independent set queries, cut queries, and standard graph queries. They also specialize the recently studied matrix-vector query model. Our work is exploratory and broad, and we provide new upper and lower bounds for a wide variety of problems, spanning linear algebra, statistics, and graphs. Many of our results are nearly tight, and we use diverse techniques from linear algebra, randomized algorithms, and communication complexity.

[1]  Clément L. Canonne,et al.  A Survey on Distribution Testing: Your Data is Big. But is it Blue? , 2020, Electron. Colloquium Comput. Complex..

[2]  Noga Alon,et al.  Learning a Hidden Subgraph , 2004, SIAM J. Discret. Math..

[3]  Cyrus Rashtchian,et al.  Equivalence of Systematic Linear Data Structures and Matrix Rigidity , 2019, Electron. Colloquium Comput. Complex..

[4]  Oded Goldreich,et al.  Introduction to Property Testing , 2017 .

[5]  Max Simchowitz,et al.  Tight query complexity lower bounds for PCA via finite sample deformed wigner law , 2018, STOC.

[6]  Ziv Bar-Yossef,et al.  An information statistics approach to data stream and communication complexity , 2002, The 43rd Annual IEEE Symposium on Foundations of Computer Science, 2002. Proceedings..

[7]  Alexander A. Razborov,et al.  On the Distributional Complexity of Disjointness , 1992, Theor. Comput. Sci..

[8]  T. Blumensath,et al.  Theory and Applications , 2011 .

[9]  Andrew Chi-Chih Yao,et al.  Probabilistic computations: Toward a unified measure of complexity , 1977, 18th Annual Symposium on Foundations of Computer Science (sfcs 1977).

[10]  Uriel Feige,et al.  On sums of independent random variables with unbounded variance, and estimating the average degree in a graph , 2004, STOC '04.

[11]  S. Matthew Weinberg,et al.  Computing Exact Minimum Cuts Without Knowing the Graph , 2017, ITCS.

[12]  Kasper Green Larsen,et al.  Tight cell probe bounds for succinct Boolean matrix-vector multiplication , 2017, STOC.

[13]  Noga Alon,et al.  Learning a Hidden Matching , 2004, SIAM J. Comput..

[14]  David P. Woodruff,et al.  Beating the Direct Sum Theorem in Communication Complexity with Implications for Sketching , 2013, SODA.

[15]  Kitty Meeks,et al.  Approximately counting and sampling small witnesses using a colourful decision oracle , 2019, SODA.

[16]  Max Simchowitz,et al.  The gradient complexity of linear regression , 2020, COLT.

[17]  Cyrus Rashtchian,et al.  Edge Estimation with Independent Set Oracles , 2017, ITCS.

[18]  Ziv Bar-Yossef,et al.  Reductions in streaming algorithms, with an application to counting triangles in graphs , 2002, SODA '02.

[19]  Arijit Ghosh,et al.  Inner Product Oracle can Estimate and Sample , 2019, ArXiv.

[20]  David P. Woodruff.  Sketching as a Tool for Numerical Linear Algebra , 2014, Found. Trends Theor. Comput. Sci..

[21]  Zeev Dvir,et al.  Static data structure lower bounds imply rigidity , 2018, Electron. Colloquium Comput. Complex..

[22]  Dana Ron,et al.  Comparing the strength of query types in property testing: The case of k-colorability , 2012, computational complexity.

[23]  Ronitt Rubinfeld,et al.  Sublinear-Time Algorithms for Counting Star Subgraphs via Edge Sampling , 2017, Algorithmica.

[24]  Bala Kalyanasundaram,et al.  The Probabilistic Communication Complexity of Set Intersection , 1992, SIAM J. Discret. Math..

[25]  Sepehr Assadi,et al.  A Simple Sublinear-Time Algorithm for Counting Arbitrary Subgraphs via Edge Sampling , 2018, ITCS.

[26]  Dana Ron,et al.  Counting stars and other small subgraphs in sublinear time , 2010, SODA '10.

[27]  Arijit Ghosh,et al.  Hyperedge Estimation using Polylogarithmic Subset Queries , 2019, ArXiv.

[28]  Danupon Nanongkai,et al.  Weighted min-cut: sequential, cut-query, and streaming algorithms , 2020, STOC.

[29]  David P. Woodruff,et al.  On Approximating Matrix Norms in Data Streams , 2019, SIAM J. Comput..

[30]  Kasper Green Larsen,et al.  Faster Online Matrix-Vector Multiplication , 2016, SODA.

[31]  Dana Ron,et al.  Approximating average parameters of graphs , 2008, Random Struct. Algorithms.

[32]  David P. Woodruff,et al.  Querying a Matrix through Matrix-Vector Products , 2019, ICALP.

[33]  Yang Li,et al.  On Estimating Maximum Matching Size in Graph Streams , 2017, SODA.

[34]  Xi Chen,et al.  Nearly optimal edge estimation with independent set queries , 2020, SODA.

[35]  Dana Ron,et al.  On approximating the number of k-cliques in sublinear time , 2017, STOC.

[36]  Avi Wigderson,et al.  The Randomized Communication Complexity of Set Disjointness , 2007, Theory Comput..

[37]  C. Seshadhri,et al.  A simpler sublinear algorithm for approximating the triangle count , 2015, ArXiv.

[38]  Dan Wang,et al.  Sublinear Algorithms for Big Data Applications , 2015, SpringerBriefs in Computer Science.

[39]  Dana Ron,et al.  Approximately Counting Triangles in Sublinear Time , 2015, 2015 IEEE 56th Annual Symposium on Foundations of Computer Science.

[40]  Krzysztof Onak,et al.  A near-optimal sublinear-time algorithm for approximating the minimum vertex cover size , 2011, SODA.

[41]  Arkadev Chattopadhyay,et al.  Simulation beats richness: new data-structure lower bounds , 2018, Electron. Colloquium Comput. Complex..

[42]  Arijit Ghosh,et al.  Triangle Estimation Using Tripartite Independent Set Queries , 2019, ISAAC.

[43]  Holger Dell,et al.  Fine-grained reductions from approximate counting to decision , 2017, STOC.

[44]  Nader H. Bshouty,et al.  On Learning Graphs with Edge-Detecting Queries , 2018, ALT.

[45]  David P. Woodruff,et al.  On the Communication Complexity of Linear Algebraic Problems in the Message Passing Model , 2014, DISC.

[46]  Man Lung Yiu,et al.  Identifying the Most Connected Vertices in Hidden Bipartite Graphs Using Group Testing , 2013, IEEE Transactions on Knowledge and Data Engineering.

[47]  David P. Woodruff,et al.  Tight Bounds for Sketching the Operator Norm, Schatten Norms, and Subspace Embeddings , 2022, APPROX-RANDOM.