Coherent transport of quantum states by deep reinforcement learning

Some problems in physics can be handled only after a suitable ansatz solution has been guessed, proving to be resilient to generalization. The coherent transport of a quantum state by adiabatic passage through an array of semiconductor quantum dots is an excellent example of such a problem, where it is necessary to introduce a so-called counterintuitive control sequence. Instead, the deep reinforcement learning (DRL) technique has proven to be able to solve very complex sequential decision-making problems, despite a lack of prior knowledge. We show that DRL discovers a control sequence that outperforms the counterintuitive control sequence. DRL can even discover novel strategies when realistic disturbances affect an ideal system, such as detuning or when dephasing or losses are added to the master equation. DRL is effective in controlling the dynamics of quantum states and, more generally, whenever an ansatz solution is unknown or insufficient to effectively treat the problem.Many problems in physics do not have an exact solution method, so their resolution has been sometimes possible only by guessing test functions. The authors apply Deep Reinforcement Learning (DRL) to control coherent transport of quantum states in arrays of quantum dots and demonstrate that DRL can solve the control problem in the absence of a known analytical solution even under disturbance conditions.

[1]  Elena Ferraro,et al.  Effective Hamiltonian for the hybrid double quantum dot qubit , 2013, Quantum Inf. Process..

[2]  Tzyh Jong Tarn,et al.  Fidelity-Based Probabilistic Q-Learning for Control of Quantum Systems , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[3]  Andrew D. Greentree,et al.  Dark-state adiabatic passage with spin-one particles , 2014, 1402.4552.

[4]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[5]  N Clément,et al.  One-by-one trap activation in silicon nanowire transistors. , 2010, Nature communications.

[6]  Enrico Prati,et al.  The finite quantum grand canonical ensemble and temperature from single-electron statistics for a mesoscopic device , 2010, 1001.2342.

[7]  Moritz August,et al.  Using Recurrent Neural Networks to Optimize Dynamical Decoupling for Quantum Memory , 2016, ArXiv.

[8]  Hartmut Neven,et al.  Universal quantum control through deep reinforcement learning , 2018, npj Quantum Information.

[9]  Florian Marquardt,et al.  Reinforcement Learning with Neural Networks for Quantum Feedback , 2018, Physical Review X.

[10]  Edoardo Charbon,et al.  Quantum information density scaling and qubit operation time constraints of CMOS silicon-based quantum computer architectures , 2017, npj Quantum Information.

[11]  M. Lukin,et al.  Relaxation, dephasing, and quantum control of electron spins in double quantum dots , 2006, cond-mat/0602470.

[12]  Antonio Celani,et al.  Flow Navigation by Smart Microswimmers via Reinforcement Learning , 2017, Physical review letters.

[13]  A. Wieck,et al.  Charge noise and spin noise in a semiconductor quantum device , 2013, Nature Physics.

[14]  Elena Ferraro,et al.  Maximum density of quantum information in a scalable CMOS implementation of the hybrid qubit architecture , 2014, Quantum Inf. Process..

[15]  Andrea Castelletti,et al.  Tree-based variable selection for dimensionality reduction of large-scale control systems , 2011, 2011 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL).

[16]  Christiane P Koch,et al.  Controlling open quantum systems: tools, achievements, and limitations , 2015, Journal of physics. Condensed matter : an Institute of Physics journal.

[17]  A. G. Fowler,et al.  Two-dimensional architectures for donor-based quantum computing , 2006 .

[18]  Irene Burghardt,et al.  Effective-mode representation of non-Markovian dynamics: a hierarchical approximation of the spectral density. II. Application to environment-induced nonadiabatic dynamics. , 2009, The Journal of chemical physics.

[19]  J. Eckel,et al.  Quantum coherent biomolecular energy transfer with spatially correlated fluctuations , 2010, 1003.3857.

[20]  Pieter Abbeel,et al.  Benchmarking Deep Reinforcement Learning for Continuous Control , 2016, ICML.

[21]  Sergio M. Savaresi,et al.  Batch Reinforcement Learning for semi-active suspension control , 2009, 2009 IEEE Control Applications, (CCA) & Intelligent Control, (ISIC).

[22]  Tzyh Jong Tarn,et al.  Incoherent Control of Quantum Systems With Wavefunction-Controllable Subspaces via Quantum Reinforcement Learning , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[23]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[24]  Gloria Platero,et al.  Fast long-range charge transfer in quantum dot arrays , 2018, Nanotechnology.

[25]  R Maurand,et al.  A CMOS silicon spin qubit , 2016, Nature Communications.

[26]  Giorgio Ferrari,et al.  Giant random telegraph signal generated by single charge trapping in submicron n-metal-oxide-semiconductor field-effect transistors , 2008 .

[27]  David J. Tannor,et al.  Simple and robust extension of the stimulated Raman adiabatic passage technique toN-level systems , 1997 .

[28]  Pankaj Mehta,et al.  Reinforcement Learning in Different Phases of Quantum Control , 2017, Physical Review X.

[29]  R. Xu,et al.  Theory of open quantum systems , 2002 .

[30]  Franco Nori,et al.  QuTiP: An open-source Python framework for the dynamics of open quantum systems , 2011, Comput. Phys. Commun..

[31]  Jaemin Lim,et al.  Efficient Simulation of Finite-Temperature Open Quantum Systems. , 2018, Physical review letters.

[32]  Giovanni Onida,et al.  GeVn complexes for silicon-based room-temperature single-atom nanoelectronics , 2018, Scientific Reports.

[33]  Martin B. Plenio,et al.  Exact mapping between system-reservoir quantum models and semi-infinite discrete chains using orthogonal polynomials , 2010, 1006.4507.

[34]  Al'an Aspuru-Guzik,et al.  Relaxation and dephasing in open quantum systems time-dependent density functional theory: Properties of exact functionals from an exactly-solvable model system , 2010, 1101.0141.

[35]  Takahiro Shinada,et al.  Band transport across a chain of dopant sites in silicon over micron distances and high temperatures , 2016, Scientific Reports.

[36]  Nicolai Friis,et al.  Optimizing Quantum Error Correction Codes with Reinforcement Learning , 2018, Quantum.

[37]  N. Vitanov,et al.  Laser-induced population transfer by adiabatic passage techniques. , 2001, Annual review of physical chemistry.

[38]  Xin Wang,et al.  When does reinforcement learning stand out in quantum control? A comparative study on state preparation , 2019, npj Quantum Information.

[39]  Edoardo Charbon,et al.  From the Quantum Moore's Law toward Silicon Based Universal Quantum Computing , 2017, 2017 IEEE International Conference on Rebooting Computing (ICRC).

[40]  Andrea Morello,et al.  Quantum Information in Silicon Devices Based on Individual Dopants , 2016 .

[41]  Sergey Levine,et al.  Trust Region Policy Optimization , 2015, ICML.

[42]  Andrew D. Greentree,et al.  Spatial adiabatic passage in a realistic triple well structure , 2008, 0802.2398.

[43]  R. G. Barnes,et al.  Unusual doublet structure in proton magnetic-resonance spectra of yttrium and lutetium trihydrides , 2004 .

[44]  A. Greentree,et al.  Spatial adiabatic passage: a review of recent progress , 2016, Reports on progress in physics. Physical Society.

[45]  Enrico Prati,et al.  Quantum neuromorphic hardware for quantum artificial intelligence , 2017 .

[46]  Jian-Shun Tang,et al.  Reconstruction of a Photonic Qubit State with Reinforcement Learning , 2018, Advanced Quantum Technologies.

[47]  Andrew D. Greentree,et al.  Coherent electronic transfer in quantum dot systems using adiabatic passage , 2004 .

[48]  Mario Krenn,et al.  Active learning machine learns to create new quantum experiments , 2017, Proceedings of the National Academy of Sciences.

[49]  José Miguel Hernández-Lobato,et al.  Taking gradients through experiments: LSTMs and memory proximal policy optimization for black-box quantum control , 2018, ISC Workshops.

[50]  M. Fanciulli,et al.  Coherent Tunneling by Adiabatic Passage of an exchange-only spin qubit in a double quantum dot chain , 2014, 1409.1785.

[51]  Andrea Bonarini,et al.  Batch Reinforcement Learning for Controlling a Mobile Wheeled Pendulum Robot , 2008, IFIP AI.

[52]  Lucas Lamata,et al.  Basic protocols in quantum reinforcement learning with superconducting circuits , 2017, Scientific Reports.

[53]  W. Heisenberg,et al.  Psoriasis - a review of recent progress, characteristics, diagnostic management , 2022, Journal of Education, Health and Sport.

[54]  Romain Wacquez,et al.  Mass Production of Silicon MOS-SETs: Can We Live with Nano-Devices' Variability? , 2011, FET.

[55]  Edoardo Charbon,et al.  A reconfigurable cryogenic platform for the classical control of quantum processors. , 2016, The Review of scientific instruments.

[56]  Marcello Restelli,et al.  A multiobjective reinforcement learning approach to water resources systems operation: Pareto frontier approximation in a single run , 2013 .

[57]  Giorgio Ferrari,et al.  Measuring the Temperature of a Mesoscopic Quantum Electron System by means of Single Electron Statistics , 2010 .

[58]  Elena Ferraro,et al.  Universal set of quantum gates for double-dot exchange-only spin qubits with intradot coupling , 2014, 1404.2423.

[59]  M. Vinet,et al.  Background charges and quantum effects in quantum dots transport spectroscopy , 2008, 0810.0672.

[60]  Yasemin Altun,et al.  Relative Entropy Policy Search , 2010 .

[61]  Jens Eisert,et al.  Reinforcement learning decoders for fault-tolerant quantum computation , 2018, Mach. Learn. Sci. Technol..

[62]  Giorgio Ferrari,et al.  Anderson-Mott transition in arrays of a few dopant atoms in a silicon transistor. , 2012, Nature nanotechnology.

[63]  Enrique Solano,et al.  Measurement-based adaptation protocol with quantum reinforcement learning , 2018, Quantum Reports.

[64]  Amir Yacoby,et al.  Dephasing time of GaAs electron-spin qubits coupled to a nuclear bath exceeding 200 μs , 2011 .

[65]  Ole Tange,et al.  GNU Parallel: The Command-Line Power Tool , 2011, login Usenix Mag..

[66]  J. G. Muga,et al.  Shortcuts to Adiabaticity , 2012, 1212.6343.

[67]  Stefan Schaal,et al.  Natural Actor-Critic , 2003, Neurocomputing.