Optimization Applications as Quantum Performance Benchmarks

Combinatorial optimization is anticipated to be one of the primary use cases for quantum computation in the coming years. The Quantum Approximate Optimization Algorithm (QAOA) and Quantum Annealing (QA) have the potential to demonstrate significant run-time performance benefits over current state-of-the-art solutions. Using existing methods for characterizing classical optimization algorithms, we analyze solution quality obtained by solving Max-Cut problems using a quantum annealing device and gate-model quantum simulators and devices. This is used to guide the development of an advanced benchmarking framework for quantum computers designed to evaluate the trade-off between run-time execution performance and the solution quality for iterative hybrid quantum-classical applications. The framework generates performance profiles through effective visualizations that show performance progression as a function of time for various problem sizes and illustrates algorithm limitations uncovered by the benchmarking approach. The framework is an enhancement to the existing open-source QED-C Application-Oriented Benchmark suite and can connect to the open-source analysis libraries. The suite can be executed on various quantum simulators and quantum hardware systems.

[1]  Luning Zhao,et al.  Application-Oriented Performance Benchmarks for Quantum Computing , 2021, IEEE Transactions on Quantum Engineering.

[2]  Carleton Coffrin,et al.  On the Emerging Potential of Quantum Annealing Hardware for Combinatorial Optimization , 2022, 2210.04291.

[3]  D. Leermakers,et al.  Evaluating the Q-score of Quantum Annealers , 2022, 2022 IEEE International Conference on Quantum Software (QSW).

[4]  Antika Sinha.  Development of research network on Quantum Annealing Computation and Information using Google Scholar data , 2022, Philosophical Transactions of the Royal Society A.

[5]  Z. Al-Ars,et al.  QPack Scores: Quantitative performance metrics for application-oriented quantum computer benchmarking , 2022, ArXiv.

[6]  Stefan Seritan,et al.  Establishing trust in quantum computations , 2022, 2204.07568.

[7]  B. Chakrabarti,et al.  Quantum Annealing and Computation , 2022, 2203.15839.

[8]  S. Eidenbenz,et al.  Quantum Volume in Practice: What Users Can Expect From NISQ Devices , 2022, IEEE Transactions on Quantum Engineering.

[9]  Kaitlin N. Smith,et al.  SupermarQ: A Scalable Quantum Benchmark Suite , 2022, 2022 IEEE International Symposium on High-Performance Computer Architecture (HPCA).

[10]  J. Klepsch,et al.  QUARK: A Framework for Quantum Computing Application Benchmarking , 2022, 2022 IEEE International Conference on Quantum Computing and Engineering (QCE).

[11]  C. Baldwin,et al.  Re-examining the quantum volume test: Ideal distributions, compiler optimizations, confidence intervals, and scalable resource estimations , 2021, Quantum.

[12]  D. Alonso,et al.  Benchmarking quantum annealing dynamics: The spin-vector Langevin model , 2021, Physical Review Research.

[13]  Daniel A. Lidar,et al.  3-regular three-XORSAT planted solutions benchmark of classical and quantum heuristic optimizers , 2021, Quantum Science and Technology.

[14]  Carleton Coffrin,et al.  Programmable Quantum Annealers as Noisy Gibbs Samplers , 2020, PRX Quantum.

[15]  Gian Giacomo Guerreschi,et al.  Evaluation of QAOA based on the approximation ratio of individual samples , 2020, Quantum Science and Technology.

[16]  Samuel A. Stein,et al.  QASMBench: A Low-Level Quantum Benchmark Suite for NISQ Evaluation and Simulation , 2020, ACM Transactions on Quantum Computing.

[17]  M. Lukin,et al.  Quantum optimization of maximum independent set using Rydberg atom arrays , 2018, Science.

[18]  A. Pham,et al.  Evaluating performance of hybrid quantum optimization algorithms for MAXCUT Clustering using IBM runtime environment , 2021, 2112.03199.

[19]  J. Wurtz,et al.  Fixed-angle conjectures for the quantum approximate optimization algorithm on regular MaxCut graphs , 2021, Physical Review A.

[20]  Blake R. Johnson,et al.  Quality, Speed, and Scale: three key attributes to measure the performance of near-term quantum computers , 2021, 2110.14108.

[21]  Alicia B. Magann,et al.  Progress toward favorable landscapes in quantum combinatorial optimization , 2021, Physical Review A.

[22]  P. Love,et al.  MaxCut quantum approximate optimization algorithm performance guarantees for p>1 , 2021 .

[23]  Carleton Coffrin,et al.  Single-Qubit Fidelity Assessment of Quantum Annealing Hardware , 2021, IEEE Transactions on Quantum Engineering.

[24]  J. Biamonte,et al.  Parameter concentrations in quantum approximate optimization , 2021, Physical Review A.

[25]  Patrick J. Coles,et al.  Cost function dependent barren plateaus in shallow parametrized quantum circuits , 2021, Nature Communications.

[26]  Travis S. Humble,et al.  Empirical performance bounds for quantum approximate optimization , 2021, Quantum Information Processing.

[27]  M. Kliesch,et al.  Training Variational Quantum Algorithms Is NP-Hard. , 2021, Physical review letters.

[28]  M. Cerezo,et al.  Variational quantum algorithms , 2020, Nature Reviews Physics.

[29]  Nathan Wiebe,et al.  Entanglement Induced Barren Plateaus , 2020, PRX Quantum.

[30]  Patrick J. Coles,et al.  Noise-induced barren plateaus in variational quantum algorithms , 2020, Nature Communications.

[31]  R. de Sousa,et al.  Benchmarking Hamiltonian Noise in the D-Wave Quantum Annealer , 2020, IEEE Transactions on Quantum Engineering.

[32]  Jian-Wei Pan,et al.  Quantum computational advantage using photons , 2020, Science.

[33]  Carleton Coffrin,et al.  The potential of quantum annealing for rapid solution structure identification , 2019, Constraints.

[34]  R. Blume-Kohout,et al.  Measuring the capabilities of quantum computers , 2020, Nature Physics.

[35]  Jakub Marecek,et al.  Quantum Computing for Finance: State-of-the-Art and Future Prospects , 2020, IEEE Transactions on Quantum Engineering.

[36]  Firas Hamze,et al.  Chook - A comprehensive suite for generating binary optimization problems with planted solutions , 2020, ArXiv.

[37]  Marc Coram,et al.  Quantum optimization with a novel Gibbs objective function and ansatz architecture search , 2019, Physical Review Research.

[38]  Giacomo Nannicini,et al.  Improving Variational Quantum Optimization using CVaR , 2019, Quantum.

[39]  F. Jin,et al.  Benchmarking the quantum approximate optimization algorithm , 2019, Quantum Inf. Process..

[40]  Robin Blume-Kohout,et al.  A volumetric framework for quantum computer benchmarks , 2019, Quantum.

[41]  M. Serrao,et al.  Quantum Computer Architecture: Towards Full-Stack Quantum Accelerators , 2019, 2020 Design, Automation & Test in Europe Conference & Exhibition (DATE).

[42]  Leo Zhou,et al.  Quantum Approximate Optimization Algorithm: Performance, Mechanism, and Implementation on Near-Term Devices , 2018, Physical Review X.

[43]  John C. Platt,et al.  Quantum supremacy using a programmable superconducting processor , 2019, Nature.

[44]  Andrew W. Cross,et al.  Validating quantum computers using randomized model circuits , 2018, Physical Review A.

[45]  Fred W. Glover,et al.  Quantum Bridge Analytics I: a tutorial on formulating and using QUBO models , 2018, 4OR.

[46]  K. Brown,et al.  Controlling error orientation to improve quantum algorithm success rates , 2018, Physical Review A.

[47]  Rupak Biswas,et al.  Readiness of Quantum Optimization Machines for Industrial Applications , 2017, Physical Review Applied.

[48]  Russell Bent,et al.  Evaluating Ising Processing Units with Integer Programming , 2017, CPAIOR.

[49]  Gavin E. Crooks,et al.  Performance of the Quantum Approximate Optimization Algorithm on the Maximum Cut Problem , 2018, 1811.08419.

[50]  Ryan Babbush,et al.  Barren plateaus in quantum neural network training landscapes , 2018, Nature Communications.

[51]  Nicholas Rubin,et al.  QAOA Performance Benchmarks using Max-Cut , 2018 .

[52]  Helmut G. Katzgraber,et al.  A deceptive step towards quantum speedup detection , 2017, Quantum Science and Technology.

[53]  Daniel A. Lidar,et al.  Demonstration of a Scaling Advantage for a Quantum Annealer over Simulated Annealing , 2017, Physical Review X.

[54]  C. Vuik,et al.  On the impact of quantum computing technology on future developments in high-performance scientific computing , 2017, Ethics and Information Technology.

[55]  Ehsan Zahedinejad,et al.  Combinatorial Optimization on Gate Model Quantum Computers: A Survey , 2017, ArXiv.

[56]  Peter Maunz,et al.  Demonstration of qubit operations below a rigorous fault tolerance threshold with gate set tomography , 2016, Nature Communications.

[57]  Guillaume Chapuis,et al.  Efficient Combinatorial Optimization Using Quantum Annealing , 2016, 1801.08653.

[58]  H. Neven,et al.  Characterizing quantum supremacy in near-term devices , 2016, Nature Physics.

[59]  Christoph Koch,et al.  Multiple Query Optimization on the D-Wave 2X Adiabatic Quantum Computer , 2015, Proc. VLDB Endow..

[60]  Catherine C. McGeoch,et al.  Benchmarking a quantum annealing processor with the time-to-target metric , 2015, 1508.05087.

[61]  Daniel A. Lidar,et al.  Probing for quantum speedup in spin-glass problems with planted solutions , 2015, 1502.01663.

[62]  Andrew J. Ochoa,et al.  Efficient Cluster Algorithm for Spin Glasses in Any Space Dimension. , 2015, Physical review letters.

[63]  E. Farhi,et al.  A Quantum Approximate Optimization Algorithm , 2014, 1411.4028.

[64]  Fred W. Glover,et al.  The unconstrained binary quadratic programming problem: a survey , 2014, Journal of Combinatorial Optimization.

[65]  M. W. Johnson,et al.  Entanglement in a Quantum Annealing Processor , 2014, 1401.3500.

[66]  Daniel A. Lidar,et al.  Defining and detecting quantum speedup , 2014, Science.

[67]  Andrew Lucas,et al.  Ising formulations of many NP problems , 2013, Front. Physics.

[68]  M. W. Johnson,et al.  Thermally assisted quantum annealing of a 16-qubit problem , 2013, Nature Communications.

[69]  Cong Wang,et al.  Experimental evaluation of an adiabiatic quantum system for combinatorial optimization , 2013, CF '13.

[70]  M. W. Johnson,et al.  Quantum annealing with manufactured spins , 2011, Nature.

[71]  Joseph Emerson,et al.  Scalable and robust randomized benchmarking of quantum processes. , 2010, Physical review letters.

[72]  Vicky Choi,et al.  Minor-embedding in adiabatic quantum computation: I. The parameter setting problem , 2008, Quantum Inf. Process..

[73]  Endre Boros,et al.  Local search heuristics for Quadratic Unconstrained Binary Optimization (QUBO) , 2007, J. Heuristics.

[74]  E. Knill,et al.  Randomized Benchmarking of Quantum Gates , 2007, 0707.0963.

[75]  M. Ruskai,et al.  Bounds for the adiabatic approximation with applications to quantum computation , 2006, quant-ph/0603175.

[76]  M. Powell.  A View of Algorithms for Optimization without Derivatives 1 , 2007 .

[77]  Johan Håstad,et al.  Some optimal inapproximability results , 2001, JACM.

[78]  Alexander Elgart,et al.  The Adiabatic Theorem of Quantum Mechanics , 1998 .

[79]  H. Nishimori,et al.  Quantum annealing in the transverse Ising model , 1998, cond-mat/9804280.

[80]  J. D. Doll,et al.  Quantum annealing: A new method for minimizing multidimensional functions , 1994, chem-ph/9404003.

[81]  G. Kochenberger,et al.  0-1 Quadratic programming approach for optimum solutions of two scheduling problems , 1994 .

[82]  Mihalis Yannakakis,et al.  Optimization, approximation, and complexity classes , 1991, STOC '88.

[83]  ReineltGerhard,et al.  An Application of Combinatorial Optimization to Statistical Physics and Circuit Layout Design , 1988 .

[84]  Martin Grötschel,et al.  An Application of Combinatorial Optimization to Statistical Physics and Circuit Layout Design , 1988, Oper. Res..

[85]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[86]  David S. Johnson,et al.  Some Simplified NP-Complete Graph Problems , 1976, Theor. Comput. Sci..

[87]  V. Fock,et al.  Beweis des Adiabatensatzes , 1928 .

[88]  E. Hellinger,et al.  Neue Begründung der Theorie quadratischer Formen von unendlichvielen Veränderlichen. , 1909 .