Minimizing Quadratic Functions in Constant Time

A sampling-based optimization method for quadratic functions is proposed. Our method approximately solves the following $n$-dimensional quadratic minimization problem in constant time, which is independent of $n$: $z^*=\min_{\mathbf{v} \in \mathbb{R}^n}\langle\mathbf{v}, A \mathbf{v}\rangle + n\langle\mathbf{v}, \mathrm{diag}(\mathbf{d})\mathbf{v}\rangle + n\langle\mathbf{b}, \mathbf{v}\rangle$, where $A \in \mathbb{R}^{n \times n}$ is a matrix and $\mathbf{d},\mathbf{b} \in \mathbb{R}^n$ are vectors. Our theoretical analysis specifies the number of samples $k(\delta, \epsilon)$ such that the approximated solution $z$ satisfies $|z - z^*| = O(\epsilon n^2)$ with probability $1-\delta$. The empirical performance (accuracy and runtime) is positively confirmed by numerical experiments.

[1]  Takafumi Kanamori,et al.  Relative Density-Ratio Estimation for Robust Distribution Comparison , 2011, Neural Computation.

[2]  László Lovász,et al.  Non-Deterministic Graph Property Testing , 2012, Combinatorics, Probability and Computing.

[3]  Krzysztof Onak,et al.  Constant-Time Approximation Algorithms via Local Improvements , 2008, 2008 49th Annual IEEE Symposium on Foundations of Computer Science.

[4]  M. Bálek,et al.  Large Networks and Graph Limits , 2022 .

[5]  Noga Alon,et al.  Random sampling and approximation of MAX-CSP problems , 2002, STOC '02.

[6]  Krzysztof Onak,et al.  A near-optimal sublinear-time algorithm for approximating the minimum vertex cover size , 2011, SODA.

[7]  Noga Alon,et al.  A combinatorial characterization of the testable graph properties: it's all about regularity , 2006, STOC '06.

[8]  László Lovász,et al.  Graph limits and parameter testing , 2006, STOC '06.

[9]  Dana Ron,et al.  Property testing and its connection to learning and approximation , 1998, JACM.

[10]  Matthias W. Seeger,et al.  Using the Nyström Method to Speed Up Kernel Machines , 2000, NIPS.

[11]  László Lovász,et al.  Limits of dense graph sequences , 2004, J. Comb. Theory B.

[12]  Takafumi Kanamori,et al.  Density Ratio Estimation in Machine Learning , 2012 .

[13]  Claire Mathieu,et al.  Yet another algorithm for dense max cut: go greedy , 2008, SODA '08.

[14]  Yuichi Yoshida,et al.  Optimal constant-time approximation algorithms and (unconditional) inapproximability results for every bounded-degree CSP , 2010, STOC '11.

[15]  Yuichi Yoshida.  A characterization of locally testable affine-invariant properties via decomposition theorems , 2014, STOC.

[16]  V. Sós,et al.  Convergent Sequences of Dense Graphs I: Subgraph Frequencies, Metric Properties and Testing , 2007, math/0702004.

[17]  Léon Bottou,et al.  Stochastic Learning , 2003, Advanced Lectures on Machine Learning.

[18]  Yuichi Yoshida,et al.  Improved Constant-Time Approximation Algorithms for Maximum Matchings and Other Optimization Problems , 2012, SIAM J. Comput..

[19]  Kevin P. Murphy,et al.  Machine learning - a probabilistic perspective , 2012, Adaptive computation and machine learning series.

[20]  GoldreichOded,et al.  Property testing and its connection to learning and approximation , 1998 .

[21]  Peter Hertling,et al.  Feasible Real Random Access Machines , 1998, J. Complex..

[22]  Ronitt Rubinfeld,et al.  Robust Characterizations of Polynomials with Applications to Program Testing , 1996, SIAM J. Comput..

[23]  Masashi Sugiyama,et al.  Least-Squares Independent Component Analysis , 2011, Neural Computation.

[24]  Yuichi Yoshida,et al.  Gowers Norm, Function Limits, and Parameter Estimation , 2014, SODA.

[25]  Alan M. Frieze,et al.  The regularity lemma and approximation schemes for dense problems , 1996, Proceedings of 37th Conference on Foundations of Computer Science.

[26]  David P. Woodruff,et al.  Sublinear Optimization for Machine Learning , 2010, 2010 IEEE 51st Annual Symposium on Foundations of Computer Science.