Quantum Ridgelet Transform: Winning Lottery Ticket of Neural Networks with Quantum Computation

A significant challenge in the field of quantum machine learning (QML) is to establish applications of quantum computation to accelerate common tasks in machine learning such as those for neural networks. Ridgelet transform has been a fundamental mathematical tool in the theoretical studies of neural networks, but the practical applicability of ridgelet transform to conducting learning tasks was limited since its numerical implementation by conventional classical computation requires an exponential runtime $\exp(O(D))$ as data dimension $D$ increases. To address this problem, we develop a quantum ridgelet transform (QRT), which implements the ridgelet transform of a quantum state within a linear runtime $O(D)$ of quantum computation. As an application, we also show that one can use QRT as a fundamental subroutine for QML to efficiently find a sparse trainable subnetwork of large shallow wide neural networks without conducting large-scale optimization of the original network. This application discovers an efficient way in this regime to demonstrate the lottery ticket hypothesis on finding such a sparse trainable neural network. These results open an avenue of QML for accelerating learning tasks with commonly used classical neural networks.

[1]  Sho Sonoda,et al.  Universality of group convolutional neural networks based on ridgelet analysis on groups , 2022, NeurIPS.

[2]  Takashi Yamakawa,et al.  Verifiable Quantum Advantage without Structure , 2022, 2022 IEEE 63rd Annual Symposium on Foundations of Computer Science (FOCS).

[3]  Sho Sonoda,et al.  Fully-Connected Network on Noncompact Symmetric Space and Ridgelet Transform based on Helgason-Fourier Analysis , 2022, ICML.

[4]  Jordan S. Cotler,et al.  Quantum advantage in learning from experiments , 2021, Science.

[5]  Hsin-Yuan Huang,et al.  Exponential Separations Between Learning With and Without Quantum Memory , 2021, 2021 IEEE 62nd Annual Symposium on Foundations of Computer Science (FOCS).

[6]  Jiman Zhao,et al.  Quantum Radon Transforms and Their Applications , 2021, IEEE Transactions on Quantum Engineering.

[7]  Sho Sonoda,et al.  Exponential Error Convergence in Data Classification with Optimized Random Features: Acceleration by Quantum Machine Learning , 2021, ArXiv.

[8]  G. Long,et al.  Multilevel 2-D Quantum Wavelet Transforms , 2021, IEEE Transactions on Cybernetics.

[9]  John Preskill,et al.  Information-theoretic bounds on quantum advantage in machine learning , 2021, Physical review letters.

[10]  Connor T. Hann,et al.  Resilience of Quantum Random Access Memory to Generic Noise , 2020, PRX Quantum.

[11]  K. Temme,et al.  A rigorous and robust quantum speed-up in supervised machine learning , 2020, Nature Physics.

[12]  Daniel M. Roy,et al.  Pruning Neural Networks at Initialization: Why are We Missing the Mark? , 2020, International Conference on Learning Representations.

[13]  Jens Eisert,et al.  On the Quantum versus Classical Learnability of Discrete Distributions , 2020, Quantum.

[14]  Sho Sonoda,et al.  Ridge Regression with Over-parametrized Two-Layer Networks Converge to Ridgelet Spectrum , 2020, AISTATS.

[15]  Ali Farhadi,et al.  Supermasks in Superposition , 2020, NeurIPS.

[16]  Ankit Pensia,et al.  Optimal Lottery Tickets via SubsetSum: Logarithmic Over-Parameterization is Sufficient , 2020, NeurIPS.

[17]  Daniel L. K. Yamins,et al.  Pruning neural networks without any data by iteratively conserving synaptic flow , 2020, NeurIPS.

[18]  Marcus Hutter,et al.  Logarithmic Pruning is All You Need , 2020, Neural Information Processing Systems.

[19]  Sho Sonoda,et al.  Learning with Optimized Random Features: Exponential Speedup by Quantum Machine Learning without Sparsity and Low-Rank Assumptions , 2020, NeurIPS.

[20]  Guodong Zhang,et al.  Picking Winning Tickets Before Training by Preserving Gradient Flow , 2020, ICLR.

[21]  Gilad Yehudai,et al.  Proving the Lottery Ticket Hypothesis: Pruning is All You Need , 2020, ICML.

[22]  Haiying Xia,et al.  Quantum multi-level wavelet transforms , 2019, Inf. Sci..

[23]  Ali Farhadi,et al.  What’s Hidden in a Randomly Weighted Neural Network? , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Hang Su,et al.  Pruning from Scratch , 2019, AAAI.

[25]  Taiji Suzuki,et al.  On the minimax optimality and superiority of deep neural network learning over sparse parameter spaces , 2019, Neural Networks.

[26]  Jason Yosinski,et al.  Deconstructing Lottery Tickets: Zeros, Signs, and the Supermask , 2019, NeurIPS.

[27]  Michele Mosca,et al.  Fault-Tolerant Resource Estimation of Quantum Random-Access Memories , 2019, IEEE Transactions on Quantum Engineering.

[28]  Nicholas J. A. Harvey,et al.  Tight Analyses for Non-Smooth Stochastic Gradient Descent , 2018, COLT.

[29]  Philip H. S. Torr,et al.  SNIP: Single-shot Network Pruning based on Connection Sensitivity , 2018, ICLR.

[30]  Ewin Tang,et al.  A quantum-inspired classical algorithm for recommendation systems , 2018, Electron. Colloquium Comput. Complex..

[31]  Nathan Wiebe,et al.  Quantum singular value transformation and beyond: exponential improvements for quantum matrix arithmetics , 2018, STOC.

[32]  Stacey Jeffery,et al.  The power of block-encoded matrix powers: improved regression techniques via faster Hamiltonian simulation , 2018, ICALP.

[33]  Michael Carbin,et al.  The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks , 2018, ICLR.

[34]  Rupak Biswas,et al.  Quantum Machine Learning , 2018 .

[35]  Simone Severini,et al.  Quantum machine learning: a classical perspective , 2017, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[36]  A. Prakash,et al.  Quantum Recommendation Systems , 2016, ITCS.

[37]  Andrew M. Childs,et al.  Quantum Algorithm for Systems of Linear Equations with Exponentially Improved Dependence on Precision , 2015, SIAM J. Comput..

[38]  Noboru Murata,et al.  Neural Network with Unbounded Activation Functions is Universal Approximator , 2015, 1505.03654.

[39]  Francis R. Bach,et al.  On the Equivalence between Kernel Quadrature Rules and Random Feature Expansions , 2015, J. Mach. Learn. Res..

[40]  Ronald de Wolf,et al.  Quantum Computing: Lecture Notes , 2015, ArXiv.

[41]  Noboru Murata,et al.  Sampling Hidden Parameters from Oracle Distribution , 2014, ICANN.

[42]  Andris Ambainis,et al.  Variable time amplitude amplification and quantum algorithms for linear algebra problems , 2012, STACS.

[43]  Isaac L. Chuang,et al.  Quantum Computation and Quantum Information (10th Anniversary edition) , 2011 .

[44]  Francisco Argüello,et al.  Quantum wavelet transforms of any order , 2009, Quantum Inf. Comput..

[45]  Steve Mullett,et al.  Read the fine print. , 2009, RN.

[46]  Yi-Kai Liu,et al.  Quantum algorithms using the curvelet transform , 2008, STOC '09.

[47]  S. Lloyd,et al.  Architectures for a quantum random access memory , 2008, 0807.4994.

[48]  Benjamin Recht,et al.  Random Features for Large-Scale Kernel Machines , 2007, NIPS.

[49]  S. Lloyd,et al.  Quantum random access memory. , 2007, Physical review letters.

[50]  William H Press,et al.  Discrete Radon transform has an exact, fast inverse and generalizes to operations other than sums along lines , 2006, Proceedings of the National Academy of Sciences.

[51]  Eric Andres,et al.  3-D Discrete Analytical Ridgelet Transform , 2006, IEEE Transactions on Image Processing.

[52]  Eric Andres,et al.  Discrete analytical Ridgelet transform , 2004, Signal Process..

[53]  Chien-Cheng Tseng,et al.  Quantum circuit design of 8 /spl times/ 8 discrete Hartley transform , 2004, 2004 IEEE International Symposium on Circuits and Systems (IEEE Cat. No.04CH37512).

[54]  Fritz Keinert,et al.  Wavelets and Multiwavelets , 2003 .

[55]  Christof Zalka,et al.  Exact quantum Fourier transforms and discrete logarithm algorithms , 2003, quant-ph/0301093.

[56]  Lov K. Grover,et al.  Creating superpositions that correspond to efficiently integrable probability distributions , 2002, quant-ph/0208112.

[57]  D. Coppersmith An approximate Fourier transform useful in quantum factoring , 2002, quant-ph/0201067.

[58]  J. Astola,et al.  Fast classical and quantum fractional Walsh transforms , 2001, ISPA 2001. Proceedings of the 2nd International Symposium on Image and Signal Processing and Analysis. In conjunction with 23rd International Conference on Information Technology Interfaces (IEEE Cat..

[59]  J. Astola,et al.  Fast classical and quantum fractional Haar wavelet transforms , 2001, ISPA 2001. Proceedings of the 2nd International Symposium on Image and Signal Processing and Analysis. In conjunction with 23rd International Conference on Information Technology Interfaces (IEEE Cat..

[60]  Sean Hallgren,et al.  An improved quantum Fourier transform algorithm and applications , 2000, Proceedings 41st Annual Symposium on Foundations of Computer Science.

[61]  Yoram Bresler,et al.  A multilevel domain decomposition algorithm for fast O(N/sup 2/logN) reprojection of tomographic images , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[62]  Richard Cleve,et al.  Fast parallel circuits for the quantum Fourier transform , 2000, Proceedings 41st Annual Symposium on Foundations of Computer Science.

[63]  Achi Brandt,et al.  A Fast and Accurate Multilevel Inversion of the Radon Transform , 1999, SIAM J. Appl. Math..

[64]  Colin P. Williams,et al.  Quantum Wavelet Transforms: Fast Algorithms and Complete Circuits , 1998, QCQC.

[65]  Martin L. Brady,et al.  A Fast Discrete Approximation Algorithm for the Radon Transform , 1998, SIAM J. Comput..

[66]  P. Høyer Efficient Quantum Transforms , 1997, quant-ph/9702028.

[67]  Noboru Murata,et al.  An Integral Representation of Functions Using Three-layered Networks and Their Approximation Bounds , 1996, Neural Networks.

[68]  W. A. Götz,et al.  A fast digital Radon transform--An efficient means for evaluating the Hough transform , 1995, Pattern Recognit..

[69]  Alexei Y. Kitaev,et al.  Quantum measurements and the Abelian Stabilizer Problem , 1995, Electron. Colloquium Comput. Complex..

[70]  Peter W. Shor,et al.  Polynomial-Time Algorithms for Prime Factorization and Discrete Logarithms on a Quantum Computer , 1995, SIAM Rev..

[71]  Daniel R. Simon,et al.  On the power of quantum computation , 1994, Proceedings 35th Annual Symposium on Foundations of Computer Science.

[72]  D. Donoho Unconditional Bases Are Optimal Bases for Data Compression and for Statistical Estimation , 1993 .

[73]  Vijay K. Madisetti,et al.  The fast discrete Radon transform. I. Theory , 1993, IEEE Trans. Image Process..

[74]  Umesh V. Vazirani,et al.  Quantum complexity theory , 1993, STOC.

[75]  Andrew R. Barron,et al.  Universal approximation bounds for superpositions of a sigmoidal function , 1993, IEEE Trans. Inf. Theory.

[76]  Gregory Beylkin,et al.  Discrete radon transform , 1987, IEEE Trans. Acoust. Speech Signal Process..

[77]  Zhangyang Wang,et al.  Peek-a-Boo: What (More) is Disguised in a Randomly Weighted Neural Network, and How to Find It Efficiently , 2022, ICLR.

[78]  Maria Schuld,et al.  Machine Learning with Quantum Computers , 2021, Quantum Science and Technology.

[79]  Saleem M. R. Taha,et al.  Wavelets and Multiwavelets Implementation Using Quantum Computing , 2016 .

[80]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[81]  Mohamed-Jalal Fadili,et al.  Curvelets and Ridgelets , 2009, Encyclopedia of Complexity and Systems Science.

[82]  AI Koan,et al.  Weighted Sums of Random Kitchen Sinks: Replacing minimization with randomization in learning , 2008, NIPS.

[83]  Johan Efberg,et al.  YALMIP : A toolbox for modeling and optimization in MATLAB , 2004 .

[84]  J. Lofberg,et al.  YALMIP : a toolbox for modeling and optimization in MATLAB , 2004, 2004 IEEE International Conference on Robotics and Automation (IEEE Cat. No.04CH37508).

[85]  M. Vetterli,et al.  The finite ridgelet transform for image representation , 2003, IEEE Trans. Image Process..