Quantum algorithms for Second-Order Cone Programming and Support Vector Machines

Second order cone programs (SOCPs) are a class of structured convex optimization problems that generalize linear programs. We present a quantum algorithm for SOCPs based on a quantum variant of the interior point method. Our algorithm outputs a classical solution to the SOCP with objective value $\epsilon$ close to the optimal in time $\widetilde{O} \left( n\sqrt{r} \frac{\zeta \kappa}{\delta^2} \log \left(1/\epsilon\right) \right)$ where $r$ is the rank and $n$ the dimension of the SOCP, $\delta$ bounds the distance from strict feasibility for the intermediate solutions, $\zeta$ is a parameter bounded by $\sqrt{n}$, and $\kappa$ is an upper bound on the condition number of matrices arising in the classical interior point method for SOCPs. We present applications to the support vector machine (SVM) problem in machine learning that reduces to SOCPs. We provide experimental evidence that the quantum algorithm achieves an asymptotic speedup over classical SVM algorithms with a running time $\widetilde{O}(n^{2.557})$ for random SVM instances. The best known classical algorithms for such instances have complexity $\widetilde{O} \left( n^{\omega+0.5}\log(1/\epsilon) \right)$, where $\omega$ is the matrix multiplication exponent that has a theoretical value of around $2.373$, but is closer to $3$ in practice.

[1]  Masoud Mohseni,et al.  Quantum support vector machine for big feature and big data classification , 2013, Physical review letters.

[2]  L. Faybusovich Linear systems in Jordan algebras and primal-dual interior-point algorithms , 1997 .

[3]  András Gilyén,et al.  Improvements in Quantum SDP-Solving with Applications , 2018, ICALP.

[4]  Shinji Mizuno,et al.  An O(√nL)-Iteration Homogeneous and Self-Dual Linear Programming Algorithm , 1994, Math. Oper. Res..

[5]  Takashi Tsuchiya,et al.  Polynomial convergence of primal-dual algorithms for the second-order cone program based on the MZ-family of directions , 2000, Math. Program..

[6]  Seth Lloyd,et al.  Quantum computational finance: quantum algorithm for portfolio optimization , 2018, 1811.03975.

[7]  Stephen P. Boyd,et al.  ECOS: An SOCP solver for embedded systems , 2013, 2013 European Control Conference (ECC).

[8]  S. Lloyd,et al.  Quantum algorithms for supervised and unsupervised machine learning , 2013, 1307.0411.

[9]  Aharon Ben-Tal,et al.  Lectures on modern convex optimization , 1987 .

[10]  Stacey Jeffery,et al.  The power of block-encoded matrix powers: improved regression techniques via faster Hamiltonian simulation , 2018, ICALP.

[11]  Johan A. K. Suykens,et al.  Weighted least squares support vector machines: robustness and sparse approximation , 2002, Neurocomputing.

[12]  Tomasz Arodz,et al.  Quantum Sparse Support Vector Machines , 2019, ArXiv.

[13]  Sanjeev Arora,et al.  The Multiplicative Weights Update Method: a Meta-Algorithm and Applications , 2012, Theory Comput..

[14]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[15]  Johan A. K. Suykens,et al.  Least Squares Support Vector Machine Classifiers , 1999, Neural Processing Letters.

[16]  Iordanis Kerenidis,et al.  A Quantum Interior Point Method for LPs and SDPs , 2018, ACM Transactions on Quantum Computing.

[17]  Narendra Karmarkar,et al.  A new polynomial-time algorithm for linear programming , 1984, STOC '84.

[18]  Iordanis Kerenidis,et al.  Quantum Recommendation Systems , 2016, ITCS.

[19]  A. Prakash,et al.  Quantum gradient descent for linear systems and least squares , 2017, Physical Review A.

[20]  Chih-Jen Lin,et al.  LIBLINEAR: A Library for Large Linear Classification , 2008, J. Mach. Learn. Res..

[21]  Xiaodi Wu,et al.  Quantum SDP Solvers: Large Speed-Ups, Optimality, and Applications to Quantum Learning , 2017, ICALP.

[22]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[23]  Farid Alizadeh,et al.  Associative and Jordan Algebras, and Polynomial Time Interior-Point Algorithms for Symmetric Cones , 2001, Math. Oper. Res..

[24]  Shouvanik Chakrabarti,et al.  Sublinear quantum algorithms for training linear and kernel-based classifiers , 2019, ICML.

[25]  Iordanis Kerenidis,et al.  Quantum classification of the MNIST dataset via Slow Feature Analysis , 2018, ArXiv.

[26]  Kim-Chuan Toh,et al.  Solving semidefinite-quadratic-linear programs using SDPT3 , 2003, Math. Program..

[27]  Yin Tat Lee,et al.  Solving Empirical Risk Minimization in the Current Matrix Multiplication Time , 2019, COLT.

[28]  Nathan Wiebe,et al.  Quantum singular value transformation and beyond: exponential improvements for quantum matrix arithmetics , 2018, STOC.

[29]  Arkadi Nemirovski,et al.  Lectures on modern convex optimization - analysis, algorithms, and engineering applications , 2001, MPS-SIAM series on optimization.

[30]  Richard Kueng,et al.  Faster quantum and classical SDP approximations for quadratic binary optimization , 2019, Quantum.

[31]  Michael J. Todd,et al.  Self-Scaled Barriers and Interior-Point Methods for Convex Programming , 1997, Math. Oper. Res..

[32]  Isaac L. Chuang,et al.  Quantum Computation and Quantum Information: Frontmatter , 2010 .

[33]  Michael J. Todd,et al.  Primal-Dual Interior-Point Methods for Self-Scaled Cones , 1998, SIAM J. Optim..

[34]  Yin Tat Lee,et al.  Solving linear programs in the current matrix multiplication time , 2018, STOC.

[35]  Iordanis Kerenidis,et al.  Quantum Algorithms for Portfolio Optimization , 2019, AFT.

[36]  Sébastien Bubeck,et al.  Convex Optimization: Algorithms and Complexity , 2014, Found. Trends Mach. Learn..

[37]  Iordanis Kerenidis,et al.  q-means: A quantum algorithm for unsupervised machine learning , 2018, NeurIPS.

[38]  Yousef Saad,et al.  Iterative methods for sparse linear systems , 2003 .

[39]  Seth Lloyd,et al.  Quantum-inspired low-rank stochastic regression with logarithmic dependence on the dimension , 2018, ArXiv.

[40]  B. Borchers CSDP, A C library for semidefinite programming , 1999 .

[41]  François Le Gall,et al.  Powers of tensors and fast matrix multiplication , 2014, ISSAC.

[42]  G. Hunanyan,et al.  Portfolio Selection , 2019, Finanzwirtschaft, Banken und Bankmanagement I Finance, Banks and Bank Management.

[43]  A. Adrian Albert On Jordan algebras of linear transformations , 1946 .

[44]  Srinivasan Arunachalam,et al.  Optimizing quantum optimization algorithms via faster quantum gradient computation , 2017, SODA.

[45]  Lov K. Grover A fast quantum mechanical algorithm for database search , 1996, STOC '96.

[46]  Chang-Yu Hsieh,et al.  A quantum extension of SVM-perf for training nonlinear SVMs in almost linear time , 2020, Quantum.

[47]  A. Harrow,et al.  Quantum algorithm for linear systems of equations. , 2008, Physical review letters.

[48]  L. Khachiyan Polynomial algorithms in linear programming , 1980 .

[49]  Ronald de Wolf,et al.  Quantum SDP-Solvers: Better Upper and Lower Bounds , 2017, 2017 IEEE 58th Annual Symposium on Foundations of Computer Science (FOCS).

[50]  Thorsten Joachims,et al.  Training linear SVMs in linear time , 2006, KDD '06.

[51]  Donald Goldfarb,et al.  Second-order cone programming , 2003, Math. Program..

[52]  Peter W. Shor,et al.  Algorithms for quantum computation: discrete logarithms and factoring , 1994, Proceedings 35th Annual Symposium on Foundations of Computer Science.

[53]  V. Barnett,et al.  Applied Linear Statistical Models , 1975 .

[54]  V. Strassen Gaussian elimination is not optimal , 1969 .

[55]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[56]  Knud D. Andersen,et al.  The Mosek Interior Point Optimizer for Linear Programming: An Implementation of the Homogeneous Algorithm , 2000 .

[57]  Krysta Marie Svore,et al.  Quantum Speed-Ups for Solving Semidefinite Programs , 2017, 2017 IEEE 58th Annual Symposium on Foundations of Computer Science (FOCS).