Variational quantum compiling with double Q-learning

Quantum compiling aims to construct a quantum circuit V by quantum gates drawn from a native gate alphabet, which is functionally equivalent to the target unitary U. It is a crucial stage for the running of quantum algorithms on noisy intermediate-scale quantum (NISQ) devices. However, the space for structure exploration of quantum circuit is enormous, resulting in the requirement of human expertise, hundreds of experimentations or modifications from existing quantum circuits. In this paper, we propose a variational quantum compiling (VQC) algorithm based on reinforcement learning, in order to automatically design the structure of quantum circuit for VQC with no human intervention. An agent is trained to sequentially select quantum gates from the native gate alphabet and the qubits they act on by double Q-learning with ϵ-greedy exploration strategy and experience replay. At first, the agent randomly explores a number of quantum circuits with different structures, and then iteratively discovers structures with higher performance on the learning task. Simulation results show that the proposed method can make exact compilations with less quantum gates compared to previous VQC algorithms. It can reduce the errors of quantum algorithms due to decoherence process and gate noise in NISQ devices, and enable quantum algorithms especially for complex algorithms to be executed within coherence time.

[1]  R. Barends,et al.  Superconducting quantum circuits at the surface code threshold for fault tolerance , 2014, Nature.

[2]  Ramesh Raskar,et al.  Designing Neural Network Architectures using Reinforcement Learning , 2016, ICLR.

[3]  K. Fujii,et al.  Variational Quantum Gate Optimization. , 2018, 1810.12745.

[4]  Austin G. Fowler.  Constructing arbitrary Steane code single logical qubit fault-tolerant gates , 2011, Quantum Inf. Comput..

[5]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[6]  Igor L. Markov,et al.  Smaller two-qubit circuits for quantum communication and computation , 2004, Proceedings Design, Automation and Test in Europe Conference and Exhibition.

[7]  Ryan Babbush,et al.  Barren plateaus in quantum neural network training landscapes , 2018, Nature Communications.

[8]  Alejandro Perdomo-Ortiz,et al.  A generative modeling approach for benchmarking and training shallow quantum circuits , 2018, npj Quantum Information.

[9]  Dong-Ling Deng,et al.  Topological Quantum Compiling with Reinforcement Learning , 2020, Physical review letters.

[10]  Y. Gurevich,et al.  Efficient decomposition of single-qubit gates intoVbasis circuits , 2013, 1303.1411.

[11]  Shenggen Zheng,et al.  Quantum generative adversarial network for generating discrete data , 2018 .

[12]  C. Gogolin,et al.  Evaluating analytic gradients on quantum hardware , 2018, Physical Review A.

[13]  Tyson Jones,et al.  Quantum compilation and circuit optimisation via energy dissipation , 2018 .

[14]  David Silver,et al.  Deep Reinforcement Learning with Double Q-Learning , 2015, AAAI.

[15]  V.V. Shende,et al.  Synthesis of quantum-logic circuits , 2006, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[16]  Mehryar Mohri,et al.  Multi-armed Bandit Algorithms and Empirical Evaluation , 2005, ECML.

[17]  A. Mandilara,et al.  Quantum compiling on locally adjusted circuits of designated architecture , 2019, 1908.03994.

[18]  Wei Wu,et al.  BlockQNN: Efficient Block-Wise Neural Network Architecture Generation , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  A. Mandilara,et al.  Quantum compiling with diffusive sets of gates , 2017, Physical Review A.

[20]  Andrew Y. Ng,et al.  Policy Invariance Under Reward Transformations: Theory and Application to Reward Shaping , 1999, ICML.

[21]  Andrew W. Cross,et al.  Quantum optimization using variational algorithms on near-term quantum devices , 2017, Quantum Science and Technology.

[22]  P. Coveney,et al.  Scalable Quantum Simulation of Molecular Energies , 2015, 1512.06860.

[23]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[24]  Long-Ji Lin,et al.  Reinforcement learning for robots using neural networks , 1992 .

[25]  Blake R. Johnson,et al.  Unsupervised Machine Learning on a Hybrid Quantum Computer , 2017, 1712.05771.

[26]  Dmitri Maslov,et al.  Asymptotically optimal approximation of single qubit unitaries by Clifford and T circuits using a constant number of ancillary qubits , 2012, Physical review letters.

[27]  Kunal Sharma,et al.  Noise resilience of variational quantum compiling , 2019, New Journal of Physics.

[28]  J. Gambetta,et al.  Hardware-efficient variational quantum eigensolver for small molecules and quantum magnets , 2017, Nature.

[29]  T. R. Tan,et al.  High-Fidelity Universal Gate Set for ^{9}Be^{+} Ion Qubits. , 2016, Physical review letters.

[30]  Robert Babuska,et al.  Experience Replay for Real-Time Reinforcement Learning Control , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[31]  Akira Sone,et al.  Cost-Function-Dependent Barren Plateaus in Shallow Quantum Neural Networks , 2020, ArXiv.

[32]  John Preskill,et al.  Quantum Computing in the NISQ era and beyond , 2018, Quantum.

[33]  Markus Brink,et al.  Demonstration of quantum volume 64 on a superconducting quantum computing system , 2020, Quantum Science and Technology.

[34]  Simone Severini,et al.  Hierarchical quantum classifiers , 2018, npj Quantum Information.

[35]  Patrick J. Coles,et al.  Machine Learning of Noise-Resilient Quantum Circuits , 2020, PRX Quantum.

[36]  Prasanna Balaprakash,et al.  Reinforcement-Learning-Based Variational Quantum Circuits Optimization for Combinatorial Problems , 2019, ArXiv.

[37]  Margaret Martonosi,et al.  Programming languages and compiler design for realistic quantum hardware , 2017, Nature.

[38]  Kristan Temme,et al.  Supervised learning with quantum-enhanced feature spaces , 2018, Nature.

[39]  Patrick J. Coles,et al.  Learning the quantum algorithm for state overlap , 2018, New Journal of Physics.

[40]  Matthias Troyer,et al.  A software methodology for compiling quantum programs , 2016, ArXiv.

[41]  Ryan LaRose,et al.  Quantum-assisted quantum compiling , 2018, Quantum.