Quantum Cross Entropy and Maximum Likelihood Principle

Quantum machine learning is an emerging field at the intersection of machine learning and quantum computing. Classical cross entropy plays a central role in machine learning. We define its quantum generalization, the quantum cross entropy, prove its lower bounds, and investigate its relation to quantum fidelity. In the classical case, minimizing cross entropy is equivalent to maximizing likelihood. In the quantum case, when the quantum cross entropy is constructed from quantum data undisturbed by quantum measurements, this relation holds. Classical cross entropy is equal to negative log-likelihood. When we obtain quantum cross entropy through empirical density matrix based on measurement outcomes, the quantum cross entropy is lower-bounded by negative log-likelihood. These two different scenarios illustrate the information loss when making quantum measurements. We conclude that to achieve the goal of full quantum machine learning, it is crucial to utilize the deferred measurement principle.

[1]  Anmer Daskin.  Quantum Principal Component Analysis , 2015 .

[2]  Robin Blume-Kohout,et al.  Accurate quantum state estimation via "Keeping the experimentalist honest" , 2006, quant-ph/0603116.

[3]  Serge Fehr,et al.  On quantum Rényi entropies: A new generalization and some properties , 2013, 1306.3142.

[4]  Aitor Lewkowycz,et al.  Relative entropy equals bulk relative entropy , 2015, 1512.06431.

[5]  Kevin P. Murphy,et al.  Machine learning - a probabilistic perspective , 2012, Adaptive computation and machine learning series.

[6]  John Preskill,et al.  Information-theoretic bounds on quantum advantage in machine learning , 2021, Physical review letters.

[7]  Mario Berta,et al.  On variational expressions for quantum relative entropies , 2015, ArXiv.

[8]  V. Vedral.  The role of relative entropy in quantum information theory , 2001, quant-ph/0102094.

[9]  Juan Miguel Arrazola,et al.  Machine learning method for state preparation and gate synthesis on photonic quantum computers , 2018, Quantum Science and Technology.

[10]  D. Petz.  Entropy, von Neumann and the von Neumann Entropy , 2001 .

[11]  Demis Hassabis,et al.  Improved protein structure prediction using potentials from deep learning , 2020, Nature.

[12]  Maria Schuld,et al.  Quantum Machine Learning in Feature Hilbert Spaces. , 2018, Physical review letters.

[13]  Seth Lloyd,et al.  Continuous-variable quantum neural networks , 2018, Physical Review Research.

[14]  Danna Zhou,et al.  d. , 1840, Microbial pathogenesis.

[15]  Alán Aspuru-Guzik,et al.  Quantum Neuron: an elementary building block for machine learning on quantum computers , 2017, ArXiv.

[16]  Matthias Troyer,et al.  Solving the quantum many-body problem with artificial neural networks , 2016, Science.

[17]  Lu-Ming Duan,et al.  Efficient representation of quantum many-body states with deep neural networks , 2017, Nature Communications.

[18]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[19]  M. Hastings,et al.  Training A Quantum Optimizer , 2016, 1605.05370.

[20]  S. Huber,et al.  Learning phase transitions by confusion , 2016, Nature Physics.

[21]  Matthias Troyer,et al.  Neural-network quantum state tomography , 2018 .

[22]  Michael D. Westmoreland,et al.  Relative entropy in quantum information theory , 2000, quant-ph/0004045.

[23]  H. Casini.  Relative entropy and the Bekenstein bound , 2008, 0804.2182.

[24]  Kristan Temme,et al.  Supervised learning with quantum-enhanced feature spaces , 2018, Nature.

[25]  R. Jozsa.  Fidelity for Mixed Quantum States , 1994 .

[26]  Z. Hradil.  Quantum-state estimation , 1996, quant-ph/9609012.

[27]  Travis S. Humble,et al.  Quantum supremacy using a programmable superconducting processor , 2019, Nature.

[28]  Dong-Ling Deng,et al.  Machine Learning Topological States , 2016, 1609.09060.

[29]  S. Gu.  Fidelity approach to quantum phase transitions , 2008, 0811.3127.

[30]  Thierry Paul,et al.  Quantum computation and quantum information , 2007, Mathematical Structures in Computer Science.

[31]  Ryan Babbush,et al.  Barren plateaus in quantum neural network training landscapes , 2018, Nature Communications.

[32]  Xizhi Han,et al.  Deep Quantum Geometry of Matrices , 2019, Physical Review X.

[33]  Shangnan Zhou.  Complexity, Entropy, and Markov Chains , 2019, 1902.10538.

[34]  Tobias J. Osborne,et al.  Training deep quantum neural networks , 2020, Nature Communications.

[35]  H. Neven,et al.  Characterizing quantum supremacy in near-term devices , 2016, Nature Physics.

[36]  W. Marsden.  I and J , 2012 .

[37]  Dirk Englund,et al.  Quantum optical neural networks , 2018, npj Quantum Information.

[38]  Roger G. Melko,et al.  Machine learning phases of matter , 2016, Nature Physics.