Entanglement-Based Feature Extraction by Tensor Network Machine Learning

It is a hot topic how entanglement, a quantity from quantum information theory, can assist machine learning. In this work, we implement numerical experiments to classify patterns/images by representing the classifiers as matrix product states (MPS). We show how entanglement can interpret machine learning by characterizing the importance of data and propose a feature extraction algorithm. We show on the MNIST dataset that when reducing the number of the retained pixels to 1/10 of the original number, the decrease of the ten-class testing accuracy is only O (10–3), which significantly improves the efficiency of the MPS machine learning. Our work improves machine learning’s interpretability and efficiency under the MPS representation by using the properties of MPS representing entanglement.

[1]  George Rajna,et al.  Second Quantum Revolution , 2016 .

[2]  F. Verstraete,et al.  Matrix product states, projected entangled pair states, and variational renormalization group methods for quantum spin systems , 2008, 0907.2796.

[3]  Raghavendra Selvan,et al.  Tensor Networks for Medical Image Classification , 2020, MIDL.

[4]  G. Evenbly,et al.  Algorithms for entanglement renormalization , 2007, 0707.1454.

[5]  Motoaki Kawanabe,et al.  How to Explain Individual Classification Decisions , 2009, J. Mach. Learn. Res..

[6]  Jacob biamonte,et al.  Quantum machine learning , 2016, Nature.

[7]  Quanshi Zhang,et al.  Visual interpretability for deep learning: a survey , 2018, Frontiers of Information Technology & Electronic Engineering.

[8]  Maria Schuld,et al.  The quest for a Quantum Neural Network , 2014, Quantum Information Processing.

[9]  Chirag Jain,et al.  A Reliable SVD based Watermarking Schem , 2008, ArXiv.

[10]  Christophe Charrier,et al.  Blind Image Quality Assessment: A Natural Scene Statistics Approach in the DCT Domain , 2012, IEEE Transactions on Image Processing.

[11]  Ors Legeza,et al.  Simulating strongly correlated quantum systems with tree tensor networks , 2010, 1006.3095.

[12]  J. Eisert,et al.  Quantum Games and Quantum Strategies , 1998, quant-ph/9806088.

[13]  White,et al.  Density matrix formulation for quantum renormalization groups. , 1992, Physical review letters.

[14]  Lucas Lamata Manuel.  Basic protocols in quantum reinforcement learning with superconducting circuits , 2017 .

[15]  Gang Su,et al.  Machine learning by unitary tensor network of hierarchical tree structure , 2017, New Journal of Physics.

[16]  J. Cirac,et al.  Resonating valence bond states in the PEPS formalism , 2012, 1203.4816.

[17]  B. Baaquie.  Quantum Finance: Path Integrals and Hamiltonians for Options and Interest Rates , 2004 .

[18]  J. Ignacio Cirac,et al.  From Probabilistic Graphical Models to Generalized Tensor Networks for Supervised Learning , 2018, IEEE Access.

[19]  Lucas Lamata,et al.  Basic protocols in quantum reinforcement learning with superconducting circuits , 2017, Scientific Reports.

[20]  Simone Severini,et al.  Compact Neural Networks based on the Multiscale Entanglement Renormalization Ansatz , 2017, BMVC.

[21]  F. Verstraete,et al.  Renormalization and tensor product states in spin chains and lattices , 2009, 0910.1130.

[22]  Hans-J. Briegel,et al.  Quantum-enhanced machine learning , 2016, Physical review letters.

[23]  J. Rarity,et al.  Photonic quantum technologies , 2009, 1003.3928.

[24]  M. Fannes,et al.  Finitely correlated states on quantum spin chains , 1992 .

[25]  Tieniu Tan,et al.  An SVD-based watermarking scheme for protecting rightful ownership , 2002, IEEE Trans. Multim..

[26]  Kamil Dimililer,et al.  Image compression system with an optimisation of compression ratio , 2019, IET Image Process..

[27]  Román Orús,et al.  Advances on tensor network theory: symmetries, fermions, entanglement, and holography , 2014, 1407.6552.

[28]  F. Verstraete,et al.  Matrix product states represent ground states faithfully , 2005, cond-mat/0505140.

[29]  José Ignacio Latorre,et al.  Image compression and entanglement , 2005, ArXiv.

[30]  Jiangfeng Du,et al.  Experimental realization of quantum games on a quantum computer. , 2001, Physical Review Letters.

[31]  U. Schollwoeck.  The density-matrix renormalization group in the age of matrix product states , 2010, 1008.3477.

[32]  Roger G. Melko,et al.  Kernel methods for interpretable machine learning of order parameters , 2017, 1704.05848.

[33]  G. Vidal,et al.  Classical simulation of quantum many-body systems with a tree tensor network , 2005, quant-ph/0511070.

[34]  F. Verstraete,et al.  Simulations Based on Matrix Product States and Projected Entangled Pair States , 2010 .

[35]  Axel Pinz,et al.  Proceedings of the 9th European conference on Computer Vision - Volume Part I , 2006 .

[36]  N. Ahmed,et al.  Discrete Cosine Transform , 1996 .

[37]  Amnon Shashua,et al.  Deep Learning and Quantum Entanglement: Fundamental Connections with Implications to Network Design , 2017, ICLR.

[38]  Guifré Vidal.  Efficient simulation of one-dimensional quantum many-body systems. , 2004, Physical review letters.

[39]  S. Rommer,et al.  CLASS OF ANSATZ WAVE FUNCTIONS FOR ONE-DIMENSIONAL SPIN SYSTEMS AND THEIR RELATION TO THE DENSITY MATRIX RENORMALIZATION GROUP , 1997 .

[40]  Jinhui Wang,et al.  Anomaly Detection with Tensor Networks , 2020, ArXiv.

[41]  Fei Yan,et al.  A survey of quantum image representations , 2015, Quantum Information Processing.

[42]  Subhash Kak,et al.  Quantum Neural Computing , 1995 .

[43]  Frank Verstraete,et al.  Peps as unique ground states of local hamiltonians , 2007, Quantum Inf. Comput..

[44]  Lei Wang,et al.  Differentiable Learning of Quantum Circuit Born Machine , 2018, Physical Review A.

[45]  G. Vidal.  Class of quantum many-body states that can be efficiently simulated. , 2006, Physical review letters.

[46]  Alessio Gianelle,et al.  Quantum-inspired Machine Learning on high-energy physics data , 2020, ArXiv.

[47]  Lukasz Cincio,et al.  Projected entangled pair states at finite temperature: Imaginary time evolution with ancillas , 2012, 1209.0454.

[48]  Justin Reyes,et al.  A Multi-Scale Tensor Network Architecture for Classification and Regression , 2020, ArXiv.

[49]  Vedran Dunjko,et al.  Exponential improvements for quantum-accessible reinforcement learning , 2017, 1710.11160.

[50]  Qubism: self-similar visualization of many-body wavefunctions , 2011, 1112.3560.

[51]  Jack Hidary,et al.  TensorNetwork for Machine Learning , 2019, ArXiv.

[52]  Gang Su,et al.  Generative Tensor Network Classification Model for Supervised Machine Learning , 2019, Physical Review B.

[53]  S. Lloyd,et al.  Quantum algorithms for supervised and unsupervised machine learning , 2013, 1307.0411.

[54]  A. F. Adams,et al.  The Survey , 2021, Dyslexia in Higher Education.

[55]  David J. Schwab,et al.  Supervised Learning with Tensor Networks , 2016, NIPS.

[56]  John Martyn,et al.  Entanglement and Tensor Networks for Supervised Image Classification , 2020, ArXiv.

[57]  Manuel Scherzer,et al.  Machine Learning of Explicit Order Parameters: From the Ising Model to SU(2) Lattice Gauge Theory , 2017, 1705.05582.

[58]  G. Vidal.  Entanglement renormalization. , 2005, Physical review letters.

[59]  A. Wallraff,et al.  Exploring Interacting Quantum Many-Body Systems by Experimentally Creating Continuous Matrix Product States in Superconducting Circuits , 2015, 1508.06471.

[60]  Ke Liu,et al.  Identification of emergent constraints and hidden order in frustrated magnets using tensorial kernel methods of machine learning , 2019, Physical Review B.

[61]  Mohamed Elhoseny,et al.  Deep learning model for real-time image compression in Internet of Underwater Things (IoUT) , 2020, Journal of Real-Time Image Processing.

[62]  Been Kim,et al.  Towards A Rigorous Science of Interpretable Machine Learning , 2017, 1702.08608.

[63]  Jessica J. Fridrich,et al.  Low-Complexity Features for JPEG Steganalysis Using Undecimated DCT , 2015, IEEE Transactions on Information Forensics and Security.

[64]  Franco Turini,et al.  A Survey of Methods for Explaining Black Box Models , 2018, ACM Comput. Surv..

[65]  B. Hao,et al.  Fractals related to long DNA sequences and complete genomes , 2000 .

[66]  G. Vidal.  Efficient classical simulation of slightly entangled quantum computations. , 2003, Physical review letters.

[67]  C-Y Lu,et al.  Entanglement-based machine learning on a quantum computer. , 2015, Physical review letters.

[68]  Simone Severini,et al.  Hierarchical quantum classifiers , 2018, npj Quantum Information.

[69]  Maciej Lewenstein,et al.  Phase detection with neural networks: interpreting the black box , 2020, New Journal of Physics.

[70]  R. Dennis Cook,et al.  Detection of Influential Observation in Linear Regression , 2000, Technometrics.

[71]  R. A. Leibler,et al.  On Information and Sufficiency , 1951 .

[72]  B. Iooss,et al.  A Review on Global Sensitivity Analysis Methods , 2014, 1404.2405.

[73]  Jonathan P Dowling,et al.  Quantum technology: the second quantum revolution , 2003, Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[74]  D. Pérez-García,et al.  PEPS as ground states: Degeneracy and topology , 2010, 1001.3807.

[75]  Lei Wang,et al.  Variational quantum eigensolver with fewer qubits , 2019, Physical Review Research.

[76]  K. Birgitta Whaley,et al.  Towards quantum machine learning with tensor networks , 2018, Quantum Science and Technology.

[77]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[78]  Carlo Meloni,et al.  Uncertainty Management in Simulation-Optimization of Complex Systems : Algorithms and Applications , 2015 .

[79]  G. Evenbly,et al.  Tensor Network States and Geometry , 2011, 1106.1082.

[80]  Frank Verstraete,et al.  Matrix product state representations , 2006, Quantum Inf. Comput..

[81]  Peter Wittek,et al.  Inductive Supervised Quantum Learning. , 2017, Physical review letters.

[82]  Yiannis Vlassopoulos,et al.  Tensor network language model , 2017, ArXiv.

[83]  Bin Xi,et al.  Theory of network contractor dynamics for exploring thermodynamic properties of two-dimensional quantum lattice models , 2013, 1301.6439.

[84]  Vijay Ganesh,et al.  Discovering Symmetry Invariants and Conserved Quantities by Interpreting Siamese Neural Networks , 2020, Physical Review Research.

[85]  Kaoru Hirota,et al.  A flexible representation of quantum images for polynomial preparation, image compression, and processing operations , 2011, Quantum Inf. Process..

[86]  Abhishek Das,et al.  Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[87]  Jie Su,et al.  A New Trend of Quantum Image Representations , 2020, IEEE Access.

[88]  Jun Wang,et al.  Unsupervised Generative Modeling Using Matrix Product States , 2017, Physical Review X.

[89]  J. Greitemann,et al.  The view of TK-SVM on the phase hierarchy in the classical kagome Heisenberg antiferromagnet , 2020, Journal of physics. Condensed matter : an Institute of Physics journal.

[90]  E. Miles Stoudenmire,et al.  Learning relevant features of data with multi-scale tensor networks , 2017, ArXiv.

[91]  X. Wang,et al.  Spin-qubit noise spectroscopy from randomized benchmarking by supervised learning , 2018, Physical Review A.

[92]  Christopher T. Chubb,et al.  Hand-waving and interpretive dance: an introductory course on tensor networks , 2016, 1603.03039.

[93]  F. Verstraete,et al.  Renormalization algorithms for Quantum-Many Body Systems in two and higher dimensions , 2004, cond-mat/0407066.

[94]  Vineeth N. Balasubramanian,et al.  Grad-CAM++: Generalized Gradient-Based Visual Explanations for Deep Convolutional Networks , 2017, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).

[95]  P. Pyllkkänen,et al.  New Directions in Cognitive Science , 1995 .

[96]  Masoud Mohseni,et al.  Commercialize quantum technologies in five years , 2017, Nature.

[97]  Lei Wang,et al.  Tree Tensor Networks for Generative Modeling , 2019, Physical Review B.

[98]  Bin Xi,et al.  Optimized decimation of tensor networks with super-orthogonalization for two-dimensional quantum lattice models , 2012, 1205.5636.

[99]  D. Perez-Garcia,et al.  Criticality, the area law, and the computational power of PEPS , 2006, quant-ph/0601075.

[100]  Roman Orus,et al.  A Practical Introduction to Tensor Networks: Matrix Product States and Projected Entangled Pair States , 2013, 1306.2164.