Hyperparameter Tuning using Quantum Genetic Algorithms

Correctly tuning the hyperparameters of a machine learning model can improve classification results. Typically hyperparameter tuning is made by humans and experience is needed to fine tune them. Algorithmic approaches have been extensively studied in the literature and can find better results. In our work we employ a quantum genetic algorithm to address the hyperparameter optimization problem. The algorithm is based on qudits instead of qubits, allowing more available states. Experiments were performed on two datasets MNIST and CIFAR10 and results were compared against classic genetic algorithms.

[1]  Huaixiao Wang,et al.  The Improvement of Quantum Genetic Algorithm and Its Application on Function Optimization , 2013 .

[2]  Jong-Hwan Kim,et al.  Parallel quantum-inspired genetic algorithm for combinatorial optimization problem , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[3]  Steven R. Young,et al.  Optimizing deep learning hyper-parameters through an evolutionary algorithm , 2015, MLHPC@SC.

[4]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[5]  Antoine Geissbühler,et al.  Model Selection for Support Vector Classifiers via Genetic Algorithms. An Application to Medical Decision Support , 2004, ISBMDA.

[6]  Valerii Tkachuk.  Quantum Genetic Algorithm Based on Qutrits and Its Application , 2018 .

[7]  A. Kuhn,et al.  Photonic qubits, qutrits and ququads accurately prepared and delivered on demand , 2012, 1203.5614.

[8]  Gexiang Zhang,et al.  Quantum-inspired evolutionary algorithms: a survey and empirical study , 2011, J. Heuristics.

[9]  Ajit Narayanan,et al.  Quantum-inspired genetic algorithms , 1996, Proceedings of IEEE International Conference on Evolutionary Computation.

[10]  Gang Luo,et al.  A review of automatic selection methods for machine learning algorithms and hyper-parameter values , 2016, Network Modeling Analysis in Health Informatics and Bioinformatics.

[11]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[12]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[13]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[14]  Christian Igel,et al.  Evolutionary tuning of multiple SVM parameters , 2005, ESANN.

[15]  Jong-Hwan Kim,et al.  Genetic quantum algorithm and its application to combinatorial optimization problem , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[16]  Zhixiang Deng,et al.  Quantum Genetic Algorithm and its Application in Power System Reactive Power Optimization , 2009, 2009 International Conference on Computational Intelligence and Security.

[17]  Charles E. Taylor.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence. Complex Adaptive Systems.John H. Holland , 1994 .

[18]  Risto Miikkulainen,et al.  Efficient Reinforcement Learning Through Evolving Neural Network Topologies , 2002, GECCO.

[19]  Alex Krizhevsky,et al.  Learning Multiple Layers of Features from Tiny Images , 2009 .

[20]  Masanori Suganuma,et al.  A genetic programming approach to designing convolutional neural network architectures , 2017, GECCO.