Deep quantum neural networks form Gaussian processes

It is well known that artificial neural networks initialized from independent and identically distributed priors converge to Gaussian processes in the limit of large number of neurons per hidden layer. In this work we prove an analogous result for Quantum Neural Networks (QNNs). Namely, we show that the outputs of certain models based on Haar random unitary or orthogonal deep QNNs converge to Gaussian processes in the limit of large Hilbert space dimension $d$. The derivation of this result is more nuanced than in the classical case due to the role played by the input states, the measurement observable, and the fact that the entries of unitary matrices are not independent. An important consequence of our analysis is that the ensuing Gaussian processes cannot be used to efficiently predict the outputs of the QNN via Bayesian statistics. Furthermore, our theorems imply that the concentration of measure phenomenon in Haar random QNNs is worse than previously thought, as we prove that expectation values and gradients concentrate as $\mathcal{O}\left(\frac{1}{e^d \sqrt{d}}\right)$. Finally, we discuss how our results improve our understanding of concentration in $t$-designs.

[1]  Eric R. Anschuetz,et al.  Interpretable Quantum Advantage in Neural Sequence Learning , 2022, PRX Quantum.

[2]  Patrick J. Coles,et al.  Challenges and opportunities in quantum machine learning , 2022, Nature Computational Science.

[3]  M. Cerezo,et al.  Exponential concentration and untrainability in quantum kernel methods , 2022, ArXiv.

[4]  Patrick J. Coles,et al.  Group-Invariant Quantum Machine Learning , 2022, PRX Quantum.

[5]  M. Schuld,et al.  Is Quantum Advantage the Right Goal for Quantum Machine Learning? , 2022, PRX Quantum.

[6]  Jennifer R. Glick,et al.  Representation Learning via Quantum Neural Tangent Kernels , 2021, PRX Quantum.

[7]  Hendrik Poulsen Nautrup,et al.  Quantum machine learning beyond kernel methods , 2021, Nature Communications.

[8]  Patrick J. Coles,et al.  Theory of overparametrization in quantum neural networks , 2021, Nature Computational Science.

[9]  M. Cerezo,et al.  Entangled Datasets for Quantum Machine Learning , 2021, ArXiv.

[10]  Oriol Vinyals,et al.  Highly accurate protein structure prediction with AlphaFold , 2021, Nature.

[11]  Patrick J. Coles,et al.  Equivalence of quantum barren plateaus to cost concentration and narrow gorges , 2021, Quantum Science and Technology.

[12]  Amjad J. Humaidi,et al.  Review of deep learning: concepts, CNN architectures, challenges, applications, future directions , 2021, Journal of Big Data.

[13]  Patrick J. Coles,et al.  Cost function dependent barren plateaus in shallow parametrized quantum circuits , 2021, Nature Communications.

[14]  Maria Schuld,et al.  Quantum machine learning models are kernel methods , 2021, 2101.11020.

[15]  S. Yelin,et al.  Entanglement devised barren plateau mitigation , 2020, Physical Review Research.

[16]  M. Cerezo,et al.  Variational quantum algorithms , 2020, Nature Reviews Physics.

[17]  Stefan Woerner,et al.  The power of quantum neural networks , 2020, Nature Computational Science.

[18]  Nathan Wiebe,et al.  Entanglement Induced Barren Plateaus , 2020, PRX Quantum.

[19]  Patrick J. Coles,et al.  Barren Plateaus Preclude Learning Scramblers. , 2020, Physical review letters.

[20]  Jascha Sohl-Dickstein,et al.  Infinite attention: NNGP and NTK for deep attention networks , 2020, ICML.

[21]  R. Kueng,et al.  Predicting many properties of a quantum system from very few measurements , 2020, Nature Physics.

[22]  Harry Xie,et al.  Preparation of ordered states in ultra-cold gases using Bayesian optimization , 2020, New Journal of Physics.

[23]  S. Lloyd,et al.  Quantum embeddings for machine learning , 2020, 2001.03622.

[24]  Greg Yang,et al.  Tensor Programs I: Wide Feedforward or Recurrent Neural Networks of Any Architecture are Gaussian Processes , 2019, NeurIPS.

[25]  Jos'e I. Latorre,et al.  Data re-uploading for a universal quantum classifier , 2019, Quantum.

[26]  Jaehoon Lee,et al.  Bayesian Deep Convolutional Networks with Many Channels are Gaussian Processes , 2018, ICLR.

[27]  Soonwon Choi,et al.  Quantum convolutional neural networks , 2018, Nature Physics.

[28]  A. Harrow,et al.  Approximate Unitary t-Designs by Short Random Quantum Circuits Using Nearest-Neighbor and Long-Range Gates , 2018, Communications in Mathematical Physics.

[29]  Kristan Temme,et al.  Supervised learning with quantum-enhanced feature spaces , 2018, Nature.

[30]  Ryan Babbush,et al.  Barren plateaus in quantum neural network training landscapes , 2018, Nature Communications.

[31]  Rupak Biswas,et al.  Quantum Machine Learning , 2018 .

[32]  Jeffrey Pennington,et al.  Deep Neural Networks as Gaussian Processes , 2017, ICLR.

[33]  Sukhdev Singh,et al.  Natural language processing: state of the art, current trends and challenges , 2017, Multimedia Tools and Applications.

[34]  Jeongwan Haah,et al.  Operator Spreading in Random Unitary Circuits , 2017, 1705.08975.

[35]  M. Ventra,et al.  Complex dynamics of memristive circuits: Analytical results and universal slow relaxation. , 2016, Physical review. E.

[36]  Omar Fawzi,et al.  Decoupling with Random Quantum Circuits , 2013, Communications in Mathematical Physics.

[37]  Jaroslaw Adam Miszczak,et al.  Symbolic integration with respect to the Haar measure on the unitary groups , 2011, 1109.4244.

[38]  S. Ounpraseuth,et al.  Gaussian Processes for Machine Learning , 2008 .

[39]  A. Harrow,et al.  Random Quantum Circuits are Approximate 2-designs , 2008, 0802.1919.

[40]  P. Hayden,et al.  Black holes as mirrors: Quantum information in random subsystems , 2007, 0708.4025.

[41]  A. J. Short,et al.  Entanglement and the foundations of statistical mechanics , 2005 .

[42]  B. Collins,et al.  Integration with Respect to the Haar Measure on Unitary, Orthogonal and Symplectic Group , 2004, Communications in Mathematical Physics.

[43]  D. Petz,et al.  On asymptotics of large Haar distributed unitary matrices , 2003, Period. Math. Hung..

[44]  B. Collins Moments and cumulants of polynomial random variables on unitarygroups, the Itzykson-Zuber integral, and free probability , 2002, math-ph/0205010.

[45]  Peter Secretan Learning , 1965, Mental Health.

[46]  C. Porter,et al.  Fluctuations of Nuclear Reaction Widths , 1956 .

[47]  L. Isserlis ON A FORMULA FOR THE PRODUCT-MOMENT COEFFICIENT OF ANY ORDER OF A NORMAL FREQUENCY DISTRIBUTION IN ANY NUMBER OF VARIABLES , 1918 .

[48]  Maria Schuld,et al.  Machine Learning with Quantum Computers , 2021, Quantum Science and Technology.

[49]  Infinite attention: NNGP and NTK for deep attention networks , 2020 .

[50]  Christian Kleiber,et al.  Multivariate distributions and the moment problem , 2013, J. Multivar. Anal..

[51]  Radford M. Neal Priors for Infinite Networks , 1996 .