Bayesian deep learning on a quantum computer

Bayesian methods in machine learning, such as Gaussian processes, have great advantages compared to other techniques. In particular, they provide estimates of the uncertainty associated with a prediction. Extending the Bayesian approach to deep architectures has remained a major challenge. Recent results connected deep feedforward neural networks with Gaussian processes, allowing training without backpropagation. This connection enables us to leverage a quantum algorithm designed for Gaussian processes and develop a new algorithm for Bayesian deep learning on quantum computers. The properties of the kernel matrix in the Gaussian process ensure the efficient execution of the core component of the protocol, quantum matrix inversion, providing at least a polynomial speedup over classical algorithms. Furthermore, we demonstrate the execution of the algorithm on contemporary quantum computers and analyze its robustness with respect to realistic noise models.

[1]  Kilian Q. Weinberger,et al.  Proceedings of the 33rd International Conference on International Conference on Machine Learning - Volume 48 , 2016 .

[2]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[3]  Jacob biamonte,et al.  Quantum machine learning , 2016, Nature.

[4]  Yoram Singer,et al.  Toward Deeper Understanding of Neural Networks: The Power of Initialization and a Dual View on Expressivity , 2016, NIPS.

[5]  Masoud Mohseni,et al.  Quantum support vector machine for big feature and big data classification , 2013, Physical review letters.

[6]  Seth Lloyd,et al.  Quantum random access memory. , 2007, Physical review letters.

[7]  E. Miles Stoudenmire,et al.  Learning relevant features of data with multi-scale tensor networks , 2017, ArXiv.

[8]  Maris Ozols,et al.  Hamiltonian simulation with optimal sample complexity , 2016, npj Quantum Information.

[9]  Bei Zeng,et al.  16-qubit IBM universal quantum computer can be fully entangled , 2018, npj Quantum Information.

[10]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[11]  E. Torrontegui,et al.  Universal quantum perceptron as efficient unitary approximators , 2018 .

[12]  Walter Vinci,et al.  Quantum variational autoencoder , 2018, Quantum Science and Technology.

[13]  M. Suzuki,et al.  General theory of higher-order decomposition of exponential operators and symplectic integrators , 1992 .

[14]  Margaret Nichols Trans , 2015, De-centering queer theory.

[15]  Andrew M. Childs,et al.  Hamiltonian Simulation with Nearly Optimal Dependence on all Parameters , 2015, 2015 IEEE 56th Annual Symposium on Foundations of Computer Science.

[16]  Daniel A. Spielman,et al.  Exponential algorithmic speedup by a quantum walk , 2002, STOC '03.

[17]  Maria Schuld,et al.  The quest for a Quantum Neural Network , 2014, Quantum Information Processing.

[18]  Alexei Y. Kitaev,et al.  Quantum measurements and the Abelian Stabilizer Problem , 1995, Electron. Colloquium Comput. Complex..

[19]  Lawrence K. Saul,et al.  Kernel Methods for Deep Learning , 2009, NIPS.

[20]  Seth Lloyd,et al.  Quantum singular-value decomposition of nonsparse low-rank matrices , 2016, 1607.05404.

[21]  M. Lewenstein,et al.  Machine learning by unitary tensor network of hierarchical tree structure , 2017, New Journal of Physics.

[22]  Andrew M. Childs,et al.  Black-box hamiltonian simulation and unitary implementation , 2009, Quantum Inf. Comput..

[23]  Zoubin Ghahramani,et al.  Adversarial Examples, Uncertainty, and Transfer Testing Robustness in Gaussian Process Hybrid Deep Networks , 2017, 1707.02476.

[24]  D. Nychka,et al.  Covariance Tapering for Interpolation of Large Spatial Datasets , 2006 .

[25]  Zoubin Ghahramani,et al.  Probabilistic machine learning and artificial intelligence , 2015, Nature.

[26]  Gunnar Rätsch,et al.  Learning Unitary Operators with Help From u(n) , 2016, AAAI.

[27]  Joseph Fitzsimons,et al.  Quantum assisted Gaussian process regression , 2015, Physical Review A.

[28]  Maria Schuld,et al.  Quantum Machine Learning in Feature Hilbert Spaces. , 2018, Physical review letters.

[29]  L. Wossnig,et al.  Quantum Linear System Algorithm for Dense Matrices. , 2017, Physical review letters.

[30]  S. Lloyd,et al.  Quantum gradient descent and Newton’s method for constrained polynomial optimization , 2016, New Journal of Physics.

[31]  Michael Backes,et al.  How Wrong Am I? - Studying Adversarial Examples and their Impact on Uncertainty in Gaussian Process Machine Learning Models , 2017, ArXiv.

[32]  Peter Wittek,et al.  Compactly Supported Basis Functions as Support Vector Kernels for Classification , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Stephen J. Roberts,et al.  Quantum algorithms for training Gaussian Processes , 2018, Physical Review A.

[34]  William J. Zeng,et al.  A Practical Quantum Instruction Set Architecture , 2016, ArXiv.

[35]  Richard E. Turner,et al.  Gaussian Process Behaviour in Wide Deep Neural Networks , 2018, ICLR.

[36]  Julien Cornebise,et al.  Weight Uncertainty in Neural Networks , 2015, ArXiv.

[37]  Yoshua Bengio,et al.  Unitary Evolution Recurrent Neural Networks , 2015, ICML.

[38]  Scott Pakin,et al.  Quantum Algorithm Implementations for Beginners , 2018, ACM Transactions on Quantum Computing.

[39]  Michael Broughton,et al.  A Universal Training Algorithm for Quantum Deep Learning , 2018, 1806.09729.

[40]  Zoubin Ghahramani,et al.  Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning , 2015, ICML.

[41]  Thierry Paul,et al.  Quantum computation and quantum information , 2007, Mathematical Structures in Computer Science.

[42]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[43]  Michael Broughton,et al.  A quantum algorithm to train neural networks using low-depth circuits , 2017, 1712.05304.

[44]  Andrew W. Cross,et al.  Open Quantum Assembly Language , 2017, 1707.03429.

[45]  I. Chuang,et al.  Quantum Digital Signatures , 2001, quant-ph/0105032.

[46]  W. Marsden I and J , 2012 .

[47]  Sandeep Subramanian,et al.  Deep Complex Networks , 2017, ICLR.

[48]  Julien Cornebise,et al.  Weight Uncertainty in Neural Network , 2015, ICML.

[49]  Seth Lloyd,et al.  Universal Quantum Simulators , 1996, Science.

[50]  Hartmut Neven,et al.  Classification with Quantum Neural Networks on Near Term Processors , 2018, 1802.06002.

[51]  J. Traub,et al.  Quantum algorithm and circuit design solving the Poisson equation , 2012, 1207.2485.

[52]  Radford M. Neal Priors for Infinite Networks , 1996 .

[53]  A. Harrow,et al.  Quantum algorithm for linear systems of equations. , 2008, Physical review letters.

[54]  Danna Zhou,et al.  d. , 1934, Microbial pathogenesis.

[55]  Yudong Cao,et al.  Quantum circuit design for solving linear systems of equations , 2011, 1110.2232.

[56]  Andrew M. Childs On the Relationship Between Continuous- and Discrete-Time Quantum Walk , 2008, 0810.0312.

[57]  Jaehoon Lee,et al.  Deep Neural Networks as Gaussian Processes , 2017, ICLR.

[58]  Marc Toussaint,et al.  Probabilistic inference for solving discrete and continuous state Markov Decision Processes , 2006, ICML.

[59]  P. Høyer,et al.  Higher order decompositions of ordered operator exponentials , 2008, 0812.0562.