Application-Motivated, Holistic Benchmarking of a Full Quantum Computing Stack

Quantum computing systems need to be benchmarked in terms of practical tasks they would be expected to do. Here, we propose 3 "application-motivated" circuit classes for benchmarking: deep (relevant for state preparation in the variational quantum eigensolver algorithm), shallow (inspired by IQP-type circuits that might be useful for near-term quantum machine learning), and square (inspired by the quantum volume benchmark). We quantify the performance of a quantum computing system in running circuits from these classes using several figures of merit, all of which require exponential classical computing resources and a polynomial number of classical samples (bitstrings) from the system. We study how performance varies with the compilation strategy used and the device on which the circuit is run. Using systems made available by IBM Quantum, we examine their performance, showing that noise-aware compilation strategies may be beneficial, and that device connectivity and noise levels play a crucial role in the performance of the system according to our benchmarks.

[1]  R. Cleve,et al.  Efficient Quantum Algorithms for Simulating Sparse Hamiltonians , 2005, quant-ph/0508139.

[2]  Arnaud Carignan-Dugas,et al.  From randomized benchmarking experiments to gate-set circuit fidelity: how to interpret randomized benchmarking decay parameters , 2018, New Journal of Physics.

[3]  Ross Duncan,et al.  Optimising Clifford Circuits with Quantomatic , 2019, QPL.

[4]  J. McClean,et al.  Strategies for quantum computing molecular energies using the unitary coupled cluster ansatz , 2017, Quantum Science and Technology.

[5]  Kathleen E. Hamilton,et al.  Generative model benchmarks for superconducting qubits , 2018, Physical Review A.

[6]  Ramis Movassagh,et al.  Efficient unitary paths and quantum computational supremacy: A proof of average-case hardness of Random Circuit Sampling , 2018, 1810.04681.

[7]  Hans De Raedt,et al.  Benchmarking the quantum approximate optimization algorithm , 2020, Quantum Inf. Process..

[8]  Andrew W. Cross,et al.  Validating quantum computers using randomized model circuits , 2018, Physical Review A.

[9]  Travis S. Humble,et al.  Establishing the quantum supremacy frontier with a 281 Pflop/s simulation , 2019, Quantum Science and Technology.

[10]  C. Figgatt,et al.  Demonstration of the QCCD trapped-ion quantum computer architecture , 2020, 2003.01293.

[11]  Ryan LaRose,et al.  Overview and Comparison of Gate Level Quantum Software Platforms , 2018, Quantum.

[12]  Jay M. Gambetta,et al.  Process verification of two-qubit quantum gates by randomized benchmarking , 2012, 1210.7011.

[13]  Robin Blume-Kohout,et al.  Direct Randomized Benchmarking for Multiqubit Devices. , 2018, Physical review letters.

[14]  Margaret Martonosi,et al.  Noise-Adaptive Compiler Mappings for Noisy Intermediate-Scale Quantum Computers , 2019, ASPLOS.

[15]  Kevin J. Sung,et al.  Hartree-Fock on a superconducting qubit quantum computer , 2020, Science.

[16]  Margaret Martonosi,et al.  ScaffCC: a framework for compilation and analysis of quantum computing programs , 2014, Conf. Computing Frontiers.

[17]  John Preskill,et al.  Quantum computing and the entanglement frontier , 2012, 1203.5813.

[18]  Ross Duncan,et al.  Phase Gadget Synthesis for Shallow Circuits , 2019, QPL.

[19]  F. Brandão,et al.  Local random quantum circuits are approximate polynomial-designs: numerical results , 2012, 1208.0692.

[20]  Ashley Montanaro,et al.  Average-case complexity versus approximate simulation of commuting quantum computations , 2015, Physical review letters.

[21]  Seth Lloyd,et al.  Convergence conditions for random quantum circuits , 2005, quant-ph/0503210.

[22]  Ivano Tavernelli,et al.  Quantum algorithms for electronic structure calculations: Particle-hole Hamiltonian and optimized wave-function expansions , 2018, Physical Review A.

[23]  R. Pooser,et al.  Cloud Quantum Computing of an Atomic Nucleus. , 2018, Physical Review Letters.

[24]  Elham Kashefi,et al.  Verification of Quantum Computation: An Overview of Existing Approaches , 2017, Theory of Computing Systems.

[25]  Adam Bouland,et al.  Quantum Supremacy and the Complexity of Random Circuit Sampling , 2018, ITCS.

[26]  E. Knill,et al.  Randomized Benchmarking of Quantum Gates , 2007, 0707.0963.

[27]  M Steffen,et al.  Characterization of addressability by simultaneous randomized benchmarking. , 2012, Physical review letters.

[28]  Peter D. Johnson,et al.  Expressibility and Entangling Capability of Parameterized Quantum Circuits for Hybrid Quantum‐Classical Algorithms , 2019, Advanced Quantum Technologies.

[29]  H. Neven,et al.  Characterizing quantum supremacy in near-term devices , 2016, Nature Physics.

[30]  Sarah Sheldon,et al.  Three-Qubit Randomized Benchmarking. , 2017, Physical review letters.

[31]  Alejandro Perdomo-Ortiz,et al.  A generative modeling approach for benchmarking and training shallow quantum circuits , 2018, npj Quantum Information.

[32]  John A. Gunnels,et al.  Leveraging Secondary Storage to Simulate Deep 54-qubit Sycamore Circuits , 2019, 1910.09534.

[33]  Elham Kashefi,et al.  The Born supremacy: quantum advantage and training of an Ising Born machine , 2019, npj Quantum Information.

[34]  W. L. Tan,et al.  Quantum Approximate Optimization with a Trapped-Ion Quantum Simulator , 2019, 1906.02700.

[35]  Elham Kashefi,et al.  Information Theoretically Secure Hypothesis Test for Temporally Unstructured Quantum Computation (Extended Abstract) , 2017, 1704.01998.

[36]  Anton Frisk Kockum,et al.  Quantum approximate optimization of the exact-cover problem on a superconducting quantum processor , 2019, 1912.10495.

[37]  Ross Duncan,et al.  t|ket⟩: a retargetable compiler for NISQ devices , 2020, Quantum Science and Technology.

[38]  Kenneth Rudinger,et al.  What Randomized Benchmarking Actually Measures. , 2017, Physical review letters.

[39]  Richard J. Lipton,et al.  Some connections between nonuniform and uniform complexity classes , 1980, STOC '80.

[40]  Stephanie Wehner,et al.  Towards Large-Scale Quantum Networks , 2019, NANOCOM.

[41]  Andrew W. Cross,et al.  Correlated randomized benchmarking , 2018, 2003.02354.

[42]  J. Eisert,et al.  Architectures for quantum simulation showing a quantum speedup , 2017, 1703.00466.

[43]  S. Debnath,et al.  Demonstration of a small programmable quantum computer with atomic qubits , 2016, Nature.

[44]  Rupak Biswas,et al.  A flexible high-performance simulator for verifying and benchmarking quantum circuits implemented on real hardware , 2018, npj Quantum Information.

[45]  R. Jozsa,et al.  Classical simulation of commuting quantum computations implies collapse of the polynomial hierarchy , 2010, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[46]  M. Bremner,et al.  Temporally unstructured quantum computation , 2009, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[47]  Aram W. Harrow,et al.  Quantum computational supremacy , 2017, Nature.

[48]  Travis S. Humble,et al.  Quantum supremacy using a programmable superconducting processor , 2019, Nature.

[49]  J. Gambetta,et al.  Hardware-efficient variational quantum eigensolver for small molecules and quantum magnets , 2017, Nature.

[50]  William J. Zeng,et al.  A Practical Quantum Instruction Set Architecture , 2016, ArXiv.

[51]  Ross Duncan,et al.  On the qubit routing problem , 2019, TQC.

[52]  Robert S. Smith,et al.  A quantum-classical cloud platform optimized for variational hybrid algorithms , 2020, ArXiv.

[53]  Robin Blume-Kohout,et al.  A volumetric framework for quantum computer benchmarks , 2019, Quantum.

[54]  Jay M. Gambetta,et al.  Effective Hamiltonian models of the cross-resonance gate , 2018, Physical Review A.

[55]  Alán Aspuru-Guzik,et al.  A variational eigenvalue solver on a photonic quantum processor , 2013, Nature Communications.

[56]  Simon J. Devitt,et al.  Performing Quantum Computing Experiments in the Cloud , 2016, 1605.05709.

[57]  Hans-J. Briegel,et al.  Machine learning \& artificial intelligence in the quantum domain , 2017, ArXiv.

[58]  Dmitri Maslov,et al.  Experimental comparison of two quantum computing architectures , 2017, Proceedings of the National Academy of Sciences.

[59]  Elham Kashefi,et al.  Methods for classically simulating noisy networked quantum architectures , 2018, Quantum Science and Technology.

[60]  H Neven,et al.  A blueprint for demonstrating quantum supremacy with superconducting qubits , 2017, Science.

[61]  Travis S. Humble,et al.  XACC: a system-level software infrastructure for heterogeneous quantum–classical computing , 2019, Quantum Science and Technology.

[62]  Jayadev Misra,et al.  A Constructive Proof of Vizing's Theorem , 1992, Inf. Process. Lett..

[63]  Scott Aaronson,et al.  Complexity-Theoretic Foundations of Quantum Supremacy Experiments , 2016, CCC.

[64]  I. Chuang,et al.  Quantum Computation and Quantum Information: Introduction to the Tenth Anniversary Edition , 2010 .

[65]  Thierry Paul,et al.  Quantum computation and quantum information , 2007, Mathematical Structures in Computer Science.

[66]  Ashley Montanaro,et al.  Achieving quantum supremacy with sparse and noisy commuting quantum computations , 2016, 1610.01808.

[67]  Alán Aspuru-Guzik,et al.  Quantum computational chemistry , 2018, Reviews of Modern Physics.

[68]  Joseph Emerson,et al.  Scalable and robust randomized benchmarking of quantum processes. , 2010, Physical review letters.

[69]  D. Cory,et al.  Quantum bootstrapping via compressed quantum Hamiltonian learning , 2014, 1409.1524.

[70]  M. Blaauboer,et al.  An analytical decomposition protocol for optimal implementation of two-qubit entangling gates , 2006, cond-mat/0609750.

[71]  Urmila Mahadev,et al.  Classical Verification of Quantum Computations , 2018, 2018 IEEE 59th Annual Symposium on Foundations of Computer Science (FOCS).

[72]  Joel J. Wallman,et al.  Noise tailoring for scalable quantum computation via randomized compiling , 2015, 1512.01098.

[73]  B. Recht,et al.  Efficient discrete approximations of quantum gates , 2001, quant-ph/0111031.

[74]  Scott Aaronson,et al.  The computational complexity of linear optics , 2010, STOC '11.

[75]  Jack J. Dongarra,et al.  The LINPACK Benchmark: past, present and future , 2003, Concurr. Comput. Pract. Exp..

[76]  Hartmut Neven,et al.  Fourier analysis of sampling from noisy chaotic quantum circuits , 2017, 1708.01875.

[77]  Masoud Mohseni,et al.  Quantum approximate optimization of non-planar graph problems on a planar superconducting processor , 2020, 2004.04197.

[78]  Dacheng Tao,et al.  The Expressive Power of Parameterized Quantum Circuits , 2018, ArXiv.

[79]  Seth Lloyd,et al.  Pseudo-Random Unitary Operators for Quantum Information Processing , 2003, Science.

[80]  C. Porter,et al.  Fluctuations of Nuclear Reaction Widths , 1956 .

[81]  Andrew W. Cross,et al.  Topological and Subsystem Codes on Low-Degree Graphs with Flag Qubits , 2019, Physical Review X.

[82]  Jérôme F Gonthier,et al.  An application benchmark for fermionic quantum simulations. , 2020, 2003.01862.

[83]  J. Eisert,et al.  Direct certification of a class of quantum simulations , 2016, 1602.00703.

[84]  Kathleen E. Hamilton,et al.  Error-mitigated data-driven circuit learning on noisy quantum hardware , 2019, Quantum Mach. Intell..

[85]  Chen-Fu Chiang,et al.  Scaffold: Quantum Programming Language , 2012 .

[86]  Travis S. Humble,et al.  Quantum chemistry as a benchmark for near-term quantum computers , 2019, npj Quantum Information.

[87]  Jay M. Gambetta,et al.  Characterizing Quantum Gates via Randomized Benchmarking , 2011, 1109.6887.

[88]  Margaret Martonosi,et al.  Full-Stack, Real-System Quantum Computer Studies: Architectural Comparisons and Design Insights , 2019, 2019 ACM/IEEE 46th Annual International Symposium on Computer Architecture (ISCA).

[89]  Margaret Martonosi,et al.  Compiler Management of Communication and Parallelism for Quantum Computation , 2015, ASPLOS.

[90]  Margaret Martonosi,et al.  Software Mitigation of Crosstalk on Noisy Intermediate-Scale Quantum Computers , 2019, ASPLOS.

[91]  Koen Bertels,et al.  Evaluation of Parameterized Quantum Circuits: on the design, and the relation between classification accuracy, expressibility and entangling capability , 2020, ArXiv.