Benchmarking a quantum annealing processor with the time-to-target metric

In the evaluation of quantum annealers, metrics based on ground state success rates have two major drawbacks. First, evaluation requires computation time for both quantum and classical processors that grows exponentially with problem size. This makes evaluation itself computationally prohibitive. Second, results are heavily dependent on the eects of analog noise on the quantum processors, which is an engineering issue that complicates the study of the underlying quantum annealing algorithm. We introduce a novel \time-to-target" metric which avoids these two issues by challenging software solvers to match the results obtained by a quantum annealer in a short amount of time. We evaluate D-Wave’s latest quantum annealer, the D-Wave 2X system, on an array of problem classes and nd that it performs well on several input classes relative to state of the art software solvers running single-threaded on a CPU.

[1]  Scott Kirkpatrick,et al.  Optimization by Simmulated Annealing , 1983, Sci..

[2]  Ray Jain,et al.  The art of computer systems performance analysis - techniques for experimental design, measurement, simulation, and modeling , 1991, Wiley professional computing.

[3]  Mauricio G. C. Resende,et al.  Designing and reporting on computational experiments with heuristic methods , 1995, J. Heuristics.

[4]  David S. Johnson,et al.  A theoretician's guide to the experimental analysis of algorithms , 1999, Data Structures, Near Neighbor Searches, and Methodology.

[5]  Russell Impagliazzo,et al.  Complexity of k-SAT , 1999, Proceedings. Fourteenth Annual IEEE Conference on Computational Complexity (Formerly: Structure in Complexity Theory Conference) (Cat.No.99CB36317).

[6]  Philip H. Ramsey.  Nonparametric Statistical Methods , 1974, Technometrics.

[7]  E. Farhi,et al.  A Quantum Adiabatic Evolution Algorithm Applied to Random Instances of an NP-Complete Problem , 2001, Science.

[8]  Nando de Freitas,et al.  From Fields to Trees , 2004, UAI.

[9]  Thomas Bartz-Beielstein,et al.  Experimental Methods for the Analysis of Optimization Algorithms , 2010 .

[10]  M. W. Johnson,et al.  Experimental investigation of an eight-qubit unit cell in a superconducting optimization processor , 2010, 1004.1628.

[11]  M. W. Johnson,et al.  Quantum annealing with manufactured spins , 2011, Nature.

[12]  Catherine C. McGeoch.  A Guide to Experimental Algorithmics , 2012 .

[13]  Daniel A. Lidar,et al.  Experimental signature of programmable quantum annealing , 2012, Nature Communications.

[14]  Daniel A. Lidar,et al.  Adiabatic quantum optimization with the wrong Hamiltonian , 2013, 1310.0529.

[15]  Cong Wang,et al.  Experimental evaluation of an adiabiatic quantum system for combinatorial optimization , 2013, CF '13.

[16]  M. W. Johnson,et al.  Thermally assisted quantum annealing of a 16-qubit problem , 2013, Nature Communications.

[17]  Rishi Saket,et al.  A PTAS for the Classical Ising Spin Glass Problem on the Chimera Graph Structure , 2013, ArXiv.

[18]  S. Knysh,et al.  Quantum Optimization of Fully-Connected Spin Glasses , 2014, 1406.7553.

[19]  Alex Selby.  Efficient subgraph-based sampling of Ising-type models with frustration , 2014, 1409.3934.

[20]  Daniel A. Lidar,et al.  Error-corrected quantum annealing with hundreds of qubits , 2013, Nature Communications.

[21]  Masoud Mohseni,et al.  Computational Role of Collective Tunneling in a Quantum Annealer , 2014 .

[22]  M. W. Johnson,et al.  Entanglement in a Quantum Annealing Processor , 2014, 1401.3500.

[23]  Mark W. Johnson,et al.  Architectural Considerations in the Design of a Superconducting Quantum Annealing Processor , 2014, IEEE Transactions on Applied Superconductivity.

[24]  Daniel A. Lidar,et al.  Defining and detecting quantum speedup , 2014, Science.

[25]  Distinguishing Classical and Quantum Models for the D-Wave Device , 2014 .

[26]  Damian S. Steiger,et al.  Heavy Tails in the Distribution of Time to Solution for Classical and Quantum Annealing. , 2015, Physical review letters.

[27]  Matthias Troyer,et al.  Optimised simulated annealing for Ising spin glasses , 2014, Comput. Phys. Commun..

[28]  Daniel A. Lidar,et al.  Consistency tests of classical and quantum models for a quantum annealer , 2014, 1403.4228.

[29]  Jeremy D. Wendt,et al.  Benchmarking Adiabatic Quantum Optimization for Complex Network Analysis , 2015, 1604.00319.

[30]  Daniel A. Lidar,et al.  Probing for quantum speedup in spin-glass problems with planted solutions , 2015, 1502.01663.

[31]  Andrew D. King,et al.  Performance of a quantum annealer on range-limited constraint satisfaction problems , 2015, ArXiv.