Optimizing Quantum Error Correction Codes with Reinforcement Learning

Quantum error correction is widely thought to be the key to fault-tolerant quantum computation. However, determining the most suited encoding for unknown error channels or specific laboratory setups is highly challenging. Here, we present a reinforcement learning framework for optimizing and fault-tolerantly adapting quantum error correction codes. We consider a reinforcement learning agent tasked with modifying a quantum memory until a desired logical error rate is reached. Using efficient simulations of a surface code quantum memory with about 70 physical qubits, we demonstrate that such a reinforcement learning agent can determine near-optimal solutions, in terms of the number of physical qubits, for various error models of interest. Moreover, we show that agents trained on one task are able to transfer their experience to similar tasks. This ability for transfer learning showcases the inherent strengths of reinforcement learning and the applicability of our approach for optimization both in off-line simulations and on-line under laboratory conditions.

[1]  R. M. Kretchmar Parallel Reinforcement Learning , 2002 .

[2]  P. Baireuther,et al.  Neural network decoder for topological color codes with circuit level noise , 2018, New Journal of Physics.

[3]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[4]  M. Lukin,et al.  Probing many-body dynamics on a 51-atom quantum simulator , 2017, Nature.

[5]  B. Lanyon,et al.  Observation of entangled states of a fully-controlled 20 qubit system , 2017, 1711.11092.

[6]  Jean-Daniel Zucker,et al.  Abstraction in Artificial Intelligence and Complex Systems , 2013, Springer New York.

[7]  DiVincenzo,et al.  Fault-Tolerant Error Correction with Efficient Quantum Codes. , 1996, Physical review letters.

[8]  Justus Piater,et al.  Improving Exploration of Deep Reinforcement Learning using Planning for Policy Search , 2019 .

[9]  Jens Eisert,et al.  Reinforcement learning decoders for fault-tolerant quantum computation , 2018, Mach. Learn. Sci. Technol..

[10]  Hans-J. Briegel,et al.  Meta-learning within Projective Simulation , 2016, IEEE Access.

[11]  Scott Aaronson,et al.  Improved Simulation of Stabilizer Circuits , 2004, ArXiv.

[12]  Pooya Ronagh,et al.  Deep neural decoders for near term fault-tolerant experiments , 2018, Quantum Science and Technology.

[13]  Sergey Levine,et al.  Guided Policy Search , 2013, ICML.

[14]  Peter D. Johnson,et al.  QVECTOR: an algorithm for device-tailored quantum error correction , 2017, 1711.02249.

[15]  J. Preskill Fault-tolerant quantum computation , 1997, quant-ph/9712048.

[16]  Sebastian Thrun,et al.  Is Learning The n-th Thing Any Easier Than Learning The First? , 1995, NIPS.

[17]  C. Monroe,et al.  Observation of a many-body dynamical phase transition with a 53-qubit quantum simulator , 2017, Nature.

[18]  Alexander Vardy,et al.  The intractability of computing the minimum distance of a code , 1997, IEEE Trans. Inf. Theory.

[19]  M. A. Martin-Delgado,et al.  Quantum measurements and gates by code deformation , 2007, 0704.2540.

[20]  J. Britton,et al.  Toward scalable ion traps for quantum information processing , 2009, 0909.2464.

[21]  John M. Martinis,et al.  Logic gates at the surface code threshold: Superconducting qubits poised for fault-tolerant quantum computing , 2014 .

[22]  Daniel Nigg,et al.  A quantum information processor with trapped ions , 2013, 1308.3096.

[23]  T. Monz,et al.  14-Qubit entanglement: creation and coherence. , 2010, Physical review letters.

[24]  B. Terhal Quantum error correction for quantum memories , 2013, 1302.3428.

[25]  T. Beth,et al.  Codes for the quantum erasure channel , 1996, quant-ph/9610042.

[26]  Thierry Paul,et al.  Quantum computation and quantum information , 2007, Mathematical Structures in Computer Science.

[27]  Hermann Kampermann,et al.  Symmetries for a high-level neural decoder on the toric code , 2019, 1910.01662.

[28]  R. Barends,et al.  Superconducting quantum circuits at the surface code threshold for fault tolerance , 2014, Nature.

[29]  Daniel Gottesman,et al.  Stabilizer Codes and Quantum Error Correction , 1997, quant-ph/9705052.

[30]  Barbara Caputo,et al.  Safety in numbers: Learning categories from few examples with multi model knowledge transfer , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[31]  Benjamin J. Brown,et al.  Analysing correlated noise on the surface code using adaptive decoding algorithms , 2017 .

[32]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[33]  John Preskill,et al.  Fault-tolerant quantum computation versus Gaussian noise , 2008, 0810.4953.

[34]  Hans-J. Briegel,et al.  Projective Simulation for Classical Learning Agents: A Comprehensive Investigation , 2015, New Generation Computing.

[35]  Nicolai Friis,et al.  Fault-tolerant interface between quantum memories and quantum processors , 2016, Nature Communications.

[36]  Fernando Pastawski,et al.  Fault-tolerant logical gates in quantum error-correcting codes , 2014, 1408.1720.

[37]  Koen Bertels,et al.  Decoding surface code with a distributed neural network–based decoder , 2019, Quantum Machine Intelligence.

[38]  Sergey Bravyi,et al.  Classification of topologically protected gates for local stabilizer codes. , 2012, Physical review letters.

[39]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[40]  Hans J. Briegel,et al.  Benchmarking Projective Simulation in Navigation Problems , 2018, IEEE Access.

[41]  Andrew Zisserman,et al.  Tabula rasa: Model transfer for object category detection , 2011, 2011 International Conference on Computer Vision.

[42]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[43]  Demis Hassabis,et al.  Mastering the game of Go without human knowledge , 2017, Nature.

[44]  Robert L. Kosut,et al.  Quantum error correction via convex optimization , 2009, Quantum Inf. Process..

[45]  Taghi M. Khoshgoftaar,et al.  A survey of transfer learning , 2016, Journal of Big Data.

[46]  Barbara Caputo,et al.  Learning to Learn, from Transfer Learning to Domain Adaptation: A Unifying Perspective , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[47]  Michalis Skotiniotis,et al.  Reinforcement learning for optimal error correction of toric codes , 2019 .

[48]  Xiaotong Ni,et al.  Neural Network Decoders for Large-Distance 2D Toric Codes , 2018, Quantum.

[49]  A. Yacoby,et al.  Demonstration of Entanglement of Electrostatically Coupled Singlet-Triplet Qubits , 2012, Science.

[50]  Koen Bertels,et al.  Comparing Neural Network Based Decoders for the Surface Code , 2018, IEEE Transactions on Computers.

[51]  Keisuke Fujii,et al.  General framework for constructing fast and near-optimal machine-learning-based decoder of the topological stabilizer codes , 2018, 1801.04377.

[52]  Nicolas Delfosse,et al.  Almost-linear time decoding algorithm for topological codes , 2017, Quantum.

[53]  Florian Marquardt,et al.  Reinforcement Learning with Neural Networks for Quantum Feedback , 2018, Physical Review X.

[54]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[55]  M. Freedman,et al.  Scalable designs for quasiparticle-poisoning-protected topological quantum computation with Majorana zero modes , 2016, 1610.05289.

[56]  P. Baireuther,et al.  Machine-learning-assisted correction of correlated qubit errors in a topological code , 2017, 1705.07855.

[57]  David Poulin,et al.  Neural Belief-Propagation Decoders for Quantum Error-Correcting Codes. , 2018, Physical review letters.

[58]  D. Yost,et al.  3D integrated superconducting qubits , 2017, 1706.04116.

[59]  Gilles Zémor,et al.  Linear-Time Maximum Likelihood Decoding of Surface Codes over the Quantum Erasure Channel , 2017, Physical Review Research.

[60]  Ashley Montanaro,et al.  Applying quantum algorithms to constraint satisfaction problems , 2018, Quantum.

[61]  Stephen D Bartlett,et al.  Ultrahigh Error Threshold for Surface Codes with Biased Noise. , 2017, Physical review letters.

[62]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.

[63]  Austin G. Fowler,et al.  Quantifying the effects of local many-qubit errors and nonlocal two-qubit errors on the surface code , 2014 .

[64]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 2005, IEEE Transactions on Neural Networks.

[65]  Barbara Caputo,et al.  The More You Know, the Less You Learn: From Knowledge Transfer to One-shot Learning of Object Categories , 2009, BMVC.

[66]  R F Werner,et al.  Iterative optimization of quantum error correcting codes. , 2005, Physical review letters.

[67]  M. Mariantoni,et al.  Surface codes: Towards practical large-scale quantum computation , 2012, 1208.0928.

[68]  Keisuke Fujii,et al.  Error and loss tolerances of surface codes with general lattice structures , 2012, 1202.2743.

[69]  N. Maskara,et al.  Advantages of versatile neural-network decoding for topological codes , 2018, Physical Review A.

[70]  Milap Sheth,et al.  Neural ensemble decoding for topological quantum error-correcting codes , 2019, Physical Review A.

[71]  C. H. Bennett,et al.  Capacities of Quantum Erasure Channels , 1997, quant-ph/9701015.

[72]  John Preskill,et al.  Fault-tolerant quantum computation with long-range correlated noise. , 2006, Physical review letters.

[73]  A. Chatterjee,et al.  Introduction to Quantum Computation , 2003 .

[74]  Mario Krenn,et al.  Active learning machine learns to create new quantum experiments , 2017, Proceedings of the National Academy of Sciences.

[75]  José Miguel Hernández-Lobato,et al.  Taking gradients through experiments: LSTMs and memory proximal policy optimization for black-box quantum control , 2018, ISC Workshops.

[76]  Renato Renner,et al.  Discovering physical concepts with neural networks , 2018, Physical review letters.

[77]  Justus H. Piater,et al.  Robotic playing for hierarchical complex skill learning , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[78]  John Preskill,et al.  Fault-tolerant computing with biased-noise superconducting qubits: a case study , 2008, 0806.0383.

[79]  J. Preskill,et al.  Topological quantum memory , 2001, quant-ph/0110143.

[80]  Hans-J. Briegel,et al.  Machine learning \& artificial intelligence in the quantum domain , 2017, ArXiv.

[81]  Hans-J. Briegel,et al.  Projective simulation with generalization , 2015, Scientific Reports.

[82]  Liang Jiang,et al.  Deep Neural Network Probabilistic Decoder for Stabilizer Codes , 2017, Scientific Reports.

[83]  Joel Johansson,et al.  Quantum error correction for the toric code using deep reinforcement learning , 2018, Quantum.

[84]  Xiaotong Ni,et al.  Scalable Neural Network Decoders for Higher Dimensional Quantum Codes , 2017, 1710.09489.

[85]  Thomas G. Dietterich,et al.  In Advances in Neural Information Processing Systems 12 , 1991, NIPS 1991.

[86]  Lenka Zdeborová,et al.  New tool in the box , 2017, Nature Physics.

[87]  J. Ignacio Cirac,et al.  Computational speedups using small quantum devices , 2018, Physical review letters.

[88]  Giacomo Torlai,et al.  Neural Decoder for Topological Codes. , 2016, Physical review letters.

[89]  T. R. Tan,et al.  Coherent diabatic ion transport and separation in a multizone trap array. , 2012, Physical review letters.

[90]  A. Kitaev,et al.  Quantum codes on a lattice with boundary , 1998, quant-ph/9811052.

[91]  W. Marsden I and J , 2012 .

[92]  Ben Reichardt,et al.  Fault-Tolerant Quantum Computation , 2016, Encyclopedia of Algorithms.

[93]  David Poulin,et al.  A linear-time benchmarking tool for generalized surface codes , 2016, ArXiv.

[94]  Enda Barrett,et al.  A parallel framework for Bayesian reinforcement learning , 2014, Connect. Sci..

[95]  H. Briegel,et al.  Fast simulation of stabilizer circuits using a graph-state representation , 2005, quant-ph/0504117.

[96]  Pradeep Kiran Sarvepalli,et al.  Neural Decoder for Topological Codes using Pseudo-Inverse of Parity Check Matrix , 2019, 2019 IEEE Information Theory Workshop (ITW).

[97]  Hans J. Briegel,et al.  Projective simulation for artificial intelligence , 2011, Scientific Reports.

[98]  Koen Bertels,et al.  Decoding small surface codes with feedforward neural networks , 2017, 1705.00857.

[99]  R. Cleve,et al.  Efficient computations of encodings for quantum error correction , 1996, quant-ph/9607030.