Sublinear quantum algorithms for training linear and kernel-based classifiers

We investigate quantum algorithms for classification, a fundamental problem in machine learning, with provable guarantees. Given $n$ $d$-dimensional data points, the state-of-the-art (and optimal) classical algorithm for training classifiers with constant margin runs in $\tilde{O}(n+d)$ time. We design sublinear quantum algorithms for the same task running in $\tilde{O}(\sqrt{n} +\sqrt{d})$ time, a quadratic improvement in both $n$ and $d$. Moreover, our algorithms use the standard quantization of the classical input and generate the same classical output, suggesting minimal overheads when used as subroutines for end-to-end applications. We also demonstrate a tight lower bound (up to poly-log factors) and discuss the possibility of implementation on near-term quantum machines. As a side result, we also give sublinear quantum algorithms for approximating the equilibria of $n$-dimensional matrix zero-sum games with optimal complexity $\tilde{\Theta}(\sqrt{n})$.

[1]  Albert B Novikoff,et al.  ON CONVERGENCE PROOFS FOR PERCEPTRONS , 1963 .

[2]  Marvin Minsky,et al.  Perceptrons: An Introduction to Computational Geometry , 1969 .

[3]  Leonid Khachiyan,et al.  A sublinear-time randomized approximation algorithm for matrix games , 1995, Oper. Res. Lett..

[4]  Jean-Yves Audibert Optimization for Machine Learning , 1995 .

[5]  Lov K. Grover A fast quantum mechanical algorithm for database search , 1996, STOC '96.

[6]  Christoph Dürr,et al.  A Quantum Algorithm for Finding the Minimum , 1996, ArXiv.

[7]  Gilles Brassard,et al.  Strengths and Weaknesses of Quantum Computing , 1997, SIAM J. Comput..

[8]  Y. Freund,et al.  Adaptive game playing using multiplicative weights , 1999 .

[9]  Lov K. Grover,et al.  Synthesis of quantum superpositions by quantum computation , 2000, Physical review letters.

[10]  G. Brassard,et al.  Quantum Amplitude Amplification and Estimation , 2000, quant-ph/0005055.

[11]  Anthony Widjaja,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2003, IEEE Transactions on Neural Networks.

[12]  Martin Zinkevich,et al.  Online Convex Programming and Generalized Infinitesimal Gradient Ascent , 2003, ICML.

[13]  Alan M. Frieze,et al.  Fast monte-carlo algorithms for finding low-rank approximations , 2004, JACM.

[14]  Johan A. K. Suykens,et al.  Least Squares Support Vector Machine Classifiers , 1999, Neural Processing Letters.

[15]  Steve Mullett,et al.  Read the fine print. , 2009, RN.

[16]  A. Harrow,et al.  Quantum algorithm for linear systems of equations. , 2008, Physical review letters.

[17]  David P. Woodruff,et al.  Sublinear Optimization for Machine Learning , 2010, 2010 IEEE 51st Annual Symposium on Foundations of Computer Science.

[18]  Xinhua Zhang,et al.  New approximation algorithms for minimum enclosing convex shapes , 2009, SODA '11.

[19]  Elad Hazan,et al.  Approximating Semidefinite Programs in Sublinear Time , 2011, NIPS.

[20]  S. Lloyd,et al.  Quantum algorithms for supervised and unsupervised machine learning , 2013, 1307.0411.

[21]  F. Petruccione,et al.  An introduction to quantum machine learning , 2014, Contemporary Physics.

[22]  Ashish Kapoor,et al.  Quantum algorithms for nearest-neighbor methods for supervised and unsupervised learning , 2014, Quantum Inf. Comput..

[23]  Anmer Daskin Quantum Principal Component Analysis , 2015 .

[24]  Andrew M. Childs,et al.  Quantum linear systems algorithm with exponentially improved dependence on precision , 2015 .

[25]  Ashish Kapoor,et al.  Quantum Perceptron Models , 2016, NIPS.

[26]  Ronald de Wolf,et al.  Guest Column: A Survey of Quantum Learning Theory , 2017, SIGA.

[27]  Xiaodi Wu,et al.  Exponential Quantum Speed-ups for Semidefinite Programming with Applications to Quantum Learning , 2017, ArXiv.

[28]  Ronald de Wolf,et al.  A Survey of Quantum Learning Theory , 2017, ArXiv.

[29]  Iordanis Kerenidis,et al.  Quantum Recommendation Systems , 2016, ITCS.

[30]  Jacob biamonte,et al.  Quantum machine learning , 2016, Nature.

[31]  Andrew M. Childs,et al.  Quantum Algorithm for Systems of Linear Equations with Exponentially Improved Dependence on Precision , 2015, SIAM J. Comput..

[32]  Chunhao Wang,et al.  Quantum-inspired sublinear classical algorithms for solving low-rank linear systems , 2018, ArXiv.

[33]  Seth Lloyd,et al.  Quantum-inspired low-rank stochastic regression with logarithmic dependence on the dimension , 2018, ArXiv.

[34]  Iordanis Kerenidis,et al.  Quantum classification of the MNIST dataset via Slow Feature Analysis , 2018, ArXiv.

[35]  Ewin Tang,et al.  Quantum-inspired classical algorithms for principal component analysis and supervised clustering , 2018, ArXiv.

[36]  Hartmut Neven,et al.  Classification with Quantum Neural Networks on Near Term Processors , 2018, 1802.06002.

[37]  Ewin Tang,et al.  A quantum-inspired classical algorithm for recommendation systems , 2018, Electron. Colloquium Comput. Complex..

[38]  John Preskill,et al.  Quantum Computing in the NISQ era and beyond , 2018, Quantum.

[39]  Kristan Temme,et al.  Supervised learning with quantum-enhanced feature spaces , 2018, Nature.

[40]  Xiaodi Wu,et al.  Quantum SDP Solvers: Large Speed-Ups, Optimality, and Applications to Quantum Learning , 2017, ICALP.

[41]  Tongyang Li,et al.  Quantum-inspired classical sublinear-time algorithm for solving low-rank semidefinite programming via sampling approaches , 2019, ArXiv.

[42]  Tongyang Li,et al.  Quantum-Inspired Sublinear Algorithm for Solving Low-Rank Semidefinite Programming , 2019, MFCS.

[43]  Ievgeniia Oshurko Quantum Machine Learning , 2020, Quantum Computing.