On the learnability of quantum neural networks

We consider the learnability of the quantum neural network (QNN) built on the variational hybrid quantum-classical scheme, which remains largely unknown due to the non-convex optimization landscape, the measurement error, and the unavoidable gate errors introduced by noisy intermediate-scale quantum (NISQ) machines. Our contributions in this paper are multi-fold. First, we derive the utility bounds of QNN towards empirical risk minimization, and show that large gate noise, few quantum measurements, and deep circuit depth will lead to the poor utility bounds. This result also applies to the variational quantum circuits with gradient-based classical optimization, and can be of independent interest. We then prove that QNN can be treated as a differentially private (DP) model. Thirdly, we show that if a concept class can be efficiently learned by QNN, then it can also be effectively learned by QNN even with gate noise. This result implies the same learnability of QNN whether it is implemented on noiseless or noisy quantum machines. We last exhibit that the quantum statistical query (QSQ) model can be effectively simulated by noisy QNN. Since the QSQ model can tackle certain tasks with runtime speedup, our result suggests that the modified QNN implemented on NISQ devices will retain the quantum advantage. Numerical simulations support the theoretical results.

[1]  Piotr Indyk,et al.  On the Fine-Grained Complexity of Empirical Risk Minimization: Kernel Methods and Neural Networks , 2017, NIPS.

[2]  Ashish Kapoor,et al.  Quantum deep learning , 2014, Quantum Inf. Comput..

[3]  Kaiming He,et al.  Feature Pyramid Networks for Object Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  M. Schuld,et al.  Circuit-centric quantum classifiers , 2018, Physical Review A.

[5]  Iordanis Kerenidis,et al.  Quantum Algorithms for Deep Convolutional Neural Networks , 2020, ICLR.

[6]  Yurii Nesterov,et al.  Cubic regularization of Newton method and its global performance , 2006, Math. Program..

[7]  Yiming Yang,et al.  XLNet: Generalized Autoregressive Pretraining for Language Understanding , 2019, NeurIPS.

[8]  Ryan Babbush,et al.  Barren plateaus in quantum neural network training landscapes , 2018, Nature Communications.

[9]  Srinivasan Arunachalam,et al.  Quantum statistical query learning , 2020, ArXiv.

[10]  Dacheng Tao,et al.  Implementable Quantum Classifier for Nonlinear Data , 2018, ArXiv.

[11]  A. Montanari,et al.  The landscape of empirical risk for nonconvex losses , 2016, The Annals of Statistics.

[12]  Vladimir Vapnik,et al.  Principles of Risk Minimization for Learning Theory , 1991, NIPS.

[13]  Yuan Feng,et al.  Quantum Privacy-Preserving Perceptron , 2017, ArXiv.

[14]  Kristan Temme,et al.  Supervised learning with quantum-enhanced feature spaces , 2018, Nature.

[15]  Liwei Wang,et al.  Efficient Private ERM for Smooth Objectives , 2017, IJCAI.

[16]  Keisuke Fujii,et al.  Quantum circuit learning , 2018, Physical Review A.

[17]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[18]  Marcello Benedetti,et al.  Parameterized quantum circuits as machine learning models , 2019, Quantum Science and Technology.

[19]  Kunal Sharma,et al.  Noise resilience of variational quantum compiling , 2019, New Journal of Physics.

[20]  K. Birgitta Whaley,et al.  Towards quantum machine learning with tensor networks , 2018, Quantum Science and Technology.

[21]  Ronald de Wolf,et al.  Guest Column: A Survey of Quantum Learning Theory , 2017, SIGA.

[22]  Michael I. Jordan,et al.  How to Escape Saddle Points Efficiently , 2017, ICML.

[23]  Zhang Jiang,et al.  An Exploration of Practical Optimizers for Variational Quantum Algorithms on Superconducting Qubit Processors , 2020, 2005.11011.

[24]  Rocco A. Servedio,et al.  Improved Bounds on Quantum Learning Algorithms , 2004, Quantum Inf. Process..

[25]  J. Gambetta,et al.  Hardware-efficient variational quantum eigensolver for small molecules and quantum magnets , 2017, Nature.

[26]  Li Zhou,et al.  Differential Privacy in Quantum Computation , 2017, 2017 IEEE 30th Computer Security Foundations Symposium (CSF).

[27]  Francesco Petruccione,et al.  Quantum classifier with tailored quantum kernel , 2019 .

[28]  Ohad Shamir,et al.  Size-Independent Sample Complexity of Neural Networks , 2017, COLT.

[29]  Michael I. Jordan,et al.  Convexity, Classification, and Risk Bounds , 2006 .

[30]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[31]  M. Sohaib Alam,et al.  Quantum Kitchen Sinks: An algorithm for machine learning on near-term quantum computers , 2018, 1806.08321.

[32]  Rocco A. Servedio,et al.  Equivalences and Separations Between Quantum and Classical Learnability , 2004, SIAM J. Comput..

[33]  Akira Sone,et al.  Cost-Function-Dependent Barren Plateaus in Shallow Quantum Neural Networks , 2020, ArXiv.

[34]  Pengfei Zhang,et al.  Information Scrambling in Quantum Neural Networks , 2020, Physical review letters.

[35]  Dacheng Tao,et al.  The Expressive Power of Parameterized Quantum Circuits , 2018, ArXiv.

[36]  V. Koltchinskii,et al.  Oracle inequalities in empirical risk minimization and sparse recovery problems , 2011 .

[37]  Maria Schuld,et al.  Implementing a distance-based classifier with a quantum interference circuit , 2017, 1703.10793.

[38]  David Haussler,et al.  Decision Theoretic Generalizations of the PAC Model for Neural Net and Other Learning Applications , 1992, Inf. Comput..

[39]  C. Gogolin,et al.  Evaluating analytic gradients on quantum hardware , 2018, Physical Review A.

[40]  Simone Severini,et al.  Quantum machine learning: a classical perspective , 2017, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[41]  Scott Aaronson,et al.  The computational complexity of linear optics , 2010, STOC '11.

[42]  Maria Schuld,et al.  Quantum Machine Learning in Feature Hilbert Spaces. , 2018, Physical review letters.

[43]  Tobias J. Osborne,et al.  Training deep quantum neural networks , 2020, Nature Communications.

[44]  Cynthia Dwork,et al.  Calibrating Noise to Sensitivity in Private Data Analysis , 2006, TCC.

[45]  Aaron Roth,et al.  The Algorithmic Foundations of Differential Privacy , 2014, Found. Trends Theor. Comput. Sci..

[46]  R. Jozsa,et al.  Classical simulation of commuting quantum computations implies collapse of the polynomial hierarchy , 2010, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[47]  Alán Aspuru-Guzik,et al.  A variational eigenvalue solver on a photonic quantum processor , 2013, Nature Communications.

[48]  Hans-J. Briegel,et al.  Machine learning \& artificial intelligence in the quantum domain , 2017, ArXiv.

[49]  E. Farhi,et al.  A Quantum Approximate Optimization Algorithm , 2014, 1411.4028.

[50]  Marek M. Rams,et al.  Quantum neural networks to simulate many-body quantum systems , 2018, Physical Review B.

[51]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[52]  Yuanzhi Li,et al.  A Convergence Theory for Deep Learning via Over-Parameterization , 2018, ICML.

[53]  Nathan Srebro,et al.  Exploring Generalization in Deep Learning , 2017, NIPS.

[54]  vCaslav Brukner,et al.  Quantum-state preparation with universal gate decompositions , 2010, 1003.5760.

[55]  John Preskill,et al.  Quantum Computing in the NISQ era and beyond , 2018, Quantum.

[56]  Ievgeniia Oshurko Quantum Machine Learning , 2020, Quantum Computing.

[57]  Ashish Kapoor,et al.  Quantum Perceptron Models , 2016, NIPS.

[58]  Logan G. Wright,et al.  The Capacity of Quantum Neural Networks , 2019, 2020 Conference on Lasers and Electro-Optics (CLEO).

[59]  Garnet Kin-Lic Chan,et al.  Quantum Imaginary Time Evolution, Quantum Lanczos, and Quantum Thermal Averaging , 2019 .

[60]  Seth Lloyd,et al.  Continuous-variable quantum neural networks , 2018, Physical Review Research.

[61]  Tobias J. Osborne,et al.  No Free Lunch for Quantum Machine Learning , 2020, 2003.14103.

[62]  Andrzej Opala,et al.  Quantum reservoir processing , 2018, npj Quantum Information.

[63]  Sofya Raskhodnikova,et al.  What Can We Learn Privately? , 2008, 2008 49th Annual IEEE Symposium on Foundations of Computer Science.

[64]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[65]  Tat-Seng Chua,et al.  Neural Collaborative Filtering , 2017, WWW.

[66]  Dacheng Tao,et al.  Efficient Online Quantum Generative Adversarial Learning Algorithms with Applications , 2019, ArXiv.

[67]  Di Wang,et al.  Differentially Private Empirical Risk Minimization with Non-convex Loss Functions , 2019, ICML.

[68]  Thierry Paul,et al.  Quantum computation and quantum information , 2007, Mathematical Structures in Computer Science.

[69]  Raef Bassily,et al.  Differentially Private Empirical Risk Minimization: Efficient Algorithms and Tight Error Bounds , 2014, 1405.7085.

[70]  Maria Schuld,et al.  Stochastic gradient descent for hybrid quantum-classical optimization , 2019, Quantum.

[71]  Yuanzhi Li,et al.  Convergence Analysis of Two-layer Neural Networks with ReLU Activation , 2017, NIPS.

[72]  Aram W. Harrow,et al.  Quantum computational supremacy , 2017, Nature.

[73]  P. Bartlett,et al.  Empirical minimization , 2006 .

[74]  Travis S. Humble,et al.  Quantum supremacy using a programmable superconducting processor , 2019, Nature.

[75]  Hartmut Neven,et al.  Classification with Quantum Neural Networks on Near Term Processors , 2018, 1802.06002.

[76]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[77]  Anand D. Sarwate,et al.  Differentially Private Empirical Risk Minimization , 2009, J. Mach. Learn. Res..

[78]  Shouvanik Chakrabarti,et al.  Quantum Wasserstein Generative Adversarial Networks , 2019, NeurIPS.

[79]  Tuo Zhao,et al.  Towards Understanding the Importance of Noise in Training Neural Networks , 2019, ICML 2019.

[80]  Umesh V. Vazirani,et al.  Quantum complexity theory , 1993, STOC.

[81]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.