A Reinforcement Learning Model for Solving the Folding Problem

In this paper we aim at proposing a reinforcement learning based model for solving combinatorial optimization problems. Combinatorial optimization problems are hard to solve optimally, that is why any attempt to improve their solutions is beneficent. We are particularly focusing on the bidimensional protein folding problem, a well known NP-hard optimizaton problem important within many fields including bioinformatics, biochemistry, molecular biology and medicine. A reinforcement learning model is introduced for solving the problem of predicting the bidimensional structure of proteins in the hydrophobic-polar model. The model proposed in this paper can be easily extended to solve other optimization problems. We also give a mathematical validation of the proposed reinforcement learning based model, indicating this way the potential of our proposal.

[1]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[2]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[3]  Xin Yao,et al.  Drift analysis and average time complexity of evolutionary algorithms , 2001, Artif. Intell..

[4]  D. Sofge.  THE ROLE OF EXPLORATION IN LEARNING CONTROL , 1992 .

[5]  Daniel Merkle,et al.  Protein Folding in the HP-Model Solved With a Hybrid Population Based ACO Algorithm , 2008 .

[6]  Long Ji Lin,et al.  Self-improving reactive agents based on reinforcement learning, planning and teaching , 1992, Machine Learning.

[7]  Leslie Pack Kaelbling,et al.  Input Generalization in Delayed Reinforcement Learning: An Algorithm and Performance Comparisons , 1991, IJCAI.

[8]  R Unger,et al.  Genetic algorithms for protein folding simulations. , 1992, Journal of molecular biology.

[9]  Chris H. Q. Ding,et al.  Multi-class protein fold recognition using support vector machines and neural networks , 2001, Bioinform..

[10]  Dirk Sudholt,et al.  Computational Complexity of Ant Colony Optimization and Its Hybridization with Local Search , 2009, Innovations in Swarm Intelligence.

[11]  Manuel López-Ibáñez,et al.  Ant colony optimization , 2010, GECCO '10.

[12]  Sebastian Thrun,et al.  The role of exploration in learning control , 1992 .

[13]  C. Anfinsen.  Principles that govern the folding of protein chains. , 1973, Science.

[14]  Eduardo F. Morales,et al.  An Introduction to Reinforcement Learning , 2011 .

[15]  K. Dill,et al.  A fast conformational search strategy for finding low energy structures of model proteins , 1996, Protein science : a publication of the Protein Society.

[16]  William E. Hart,et al.  Optimizing an Arbitrary Function is Hard for the Genetic Algorithm , 1991 .

[17]  K. Dill,et al.  A lattice statistical mechanics model of the conformational and sequence spaces of proteins , 1989 .

[18]  Holger H. Hoos,et al.  An ant colony optimisation algorithm for the 2D and 3D hydrophobic polar protein folding problem , 2005, BMC Bioinformatics.

[19]  Camelia Chira.  HILL-CLIMBING SEARCH IN EVOLUTIONARY MODELS FOR PROTEIN FOLDING SIMULATIONS , 2010 .

[20]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[21]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[22]  Ting Wang,et al.  3D Protein structure prediction with genetic tabu search algorithm , 2009, 2009 Second International Symposium on Knowledge Acquisition and Modeling.