EA-based hyperparameter optimization of hybrid deep learning models for effective drug-target interactions prediction

Abstract The identification of drug-target interactions (DTIs) is an important process in drug repositioning and drug discovery. However, it is very expensive and time-consuming to determine all possible DTIs with experimental approaches. Most existing machine learning-based methods formulate the DTIs prediction problem as a binary classification problem. Nevertheless, the lack of experimentally validated negative samples results in imbalanced class distribution within the datasets, which may have a negative influence on the DTI prediction performance. Casting DTI prediction task as a regression problem seems an interesting alternative to avoid this issue especially with the recent increase in protein structural data and DTI datasets. Within this context, a twofold contribution is described in this paper. First, we propose a novel deep learning model for predicting drug-target binding affinities called “Convolution Neural Network with Attention-based bidirectional Long Short-Term Memory network” (CNN-AbiLSTM), which combines a CNN with an attention-based biLSTM. Second, building a powerful hybrid CNN-AbiLSTM model can be highly complicated and requires a suitable setting of the model’s hyperparameters. To handle this problem, we propose an evolutionary algorithm-based framework more specifically a Differential Evolution (DE) algorithm to find the optimal configuration of the proposed model. Experimental results show that the proposed DE-based CNN-AbiLSTM model achieves better performance compared with baseline methods.

[1]  Bin Chen,et al.  Predicting drug target interactions using meta-path-based semantic network analysis , 2016, BMC Bioinformatics.

[2]  Yoshihiro Yamanishi,et al.  Supervised prediction of drug–target interactions using bipartite local models , 2009, Bioinform..

[3]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[4]  Junwei Zhou,et al.  Arabic Sentiment Classification Using Convolutional Neural Network and Differential Evolution Algorithm , 2019, Comput. Intell. Neurosci..

[5]  BengioYoshua,et al.  Random search for hyper-parameter optimization , 2012 .

[6]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[7]  Benjamin Piwowarski,et al.  Evolutionary optimization of convolutional neural networks for cancer miRNA biomarkers classification , 2018, Appl. Soft Comput..

[8]  Shuigeng Zhou,et al.  Boosting compound-protein interaction prediction by deep learning , 2015, 2015 IEEE International Conference on Bioinformatics and Biomedicine (BIBM).

[9]  Marc Parizeau,et al.  DEAP: a python framework for evolutionary algorithms , 2012, GECCO '12.

[10]  Paul D Lyne,et al.  Structure-based virtual screening: an overview. , 2002, Drug discovery today.

[11]  Yanli Wang,et al.  PubChem: Integrated Platform of Small Molecules and Biological Activities , 2008 .

[12]  Artem Cherkasov,et al.  SimBoost: a read-across approach for predicting drug–target binding affinities using gradient boosting machines , 2017, Journal of Cheminformatics.

[13]  Godfrey C. Onwubolu,et al.  Forward Backward Transformation , 2009 .

[14]  Roland Vollgraf,et al.  Contextual String Embeddings for Sequence Labeling , 2018, COLING.

[15]  Xin Yao,et al.  Many-Objective Evolutionary Algorithms , 2015, ACM Comput. Surv..

[16]  Alex S. Fukunaga,et al.  Success-history based parameter adaptation for Differential Evolution , 2013, 2013 IEEE Congress on Evolutionary Computation.

[17]  Chee-Keong Kwoh,et al.  Computational prediction of drug-target interactions using chemogenomic approaches: an empirical survey , 2019, Briefings Bioinform..

[18]  Mehmet Gönen,et al.  Predicting drug-target interactions from chemical and genomic kernels using Bayesian matrix factorization , 2012, Bioinform..

[19]  Jian-Yu Shi,et al.  A unified solution for different scenarios of predicting drug-target interactions via triple matrix factorization , 2018, BMC Systems Biology.

[20]  Elliot Meyerson,et al.  Evolving Deep Neural Networks , 2017, Artificial Intelligence in the Age of Neural Networks and Brain Computing.

[21]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[22]  Yutaka Akiyama,et al.  NRLMFβ: Beta-distribution-rescored neighborhood regularized logistic matrix factorization for improving the performance of drug–target interaction prediction , 2019, Biochemistry and biophysics reports.

[23]  Yasuo Tabei,et al.  Identification of chemogenomic features from drug–target interaction networks using interpretable classifiers , 2012, Bioinform..

[24]  Christopher Potts,et al.  Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank , 2013, EMNLP.

[25]  Kyunghyun Cho,et al.  Efficient Character-level Document Classification by Combining Convolution and Recurrent Layers , 2016, ArXiv.

[26]  Michael J. Keiser,et al.  Predicting new molecular targets for known drugs , 2009, Nature.

[27]  T. Ashburn,et al.  Drug repositioning: identifying and developing new uses for existing drugs , 2004, Nature Reviews Drug Discovery.

[28]  Gang Liu,et al.  Bidirectional LSTM with attention mechanism and convolutional layer for text classification , 2019, Neurocomputing.

[29]  Sean Ekins,et al.  In silico repositioning of approved drugs for rare and neglected diseases. , 2011, Drug discovery today.

[30]  Jasper Snoek,et al.  Practical Bayesian Optimization of Machine Learning Algorithms , 2012, NIPS.

[31]  Mehmet Fatih Tasgetiren,et al.  Metaheuristic algorithms for the quadratic assignment problem , 2013, 2013 IEEE Symposium on Computational Intelligence in Production and Logistics Systems (CIPLS).

[32]  Hao Ding,et al.  Similarity-based machine learning methods for predicting drug-target interactions: a brief review , 2014, Briefings Bioinform..

[33]  Ola Spjuth,et al.  Large-scale virtual screening on public cloud resources with Apache Spark , 2017, Journal of Cheminformatics.

[34]  Quan-Ke Pan,et al.  Discrete/Binary Approach , 2009 .

[35]  José Ranilla,et al.  Particle swarm optimization for hyper-parameter selection in deep neural networks , 2017, GECCO.

[36]  Chee Keong Kwoh,et al.  Drug-target interaction prediction by learning from local information and neighbors , 2013, Bioinform..

[37]  Tobias Senst,et al.  Hyper-parameter optimization for convolutional neural network committees based on evolutionary algorithms , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[38]  Xiaobo Zhou,et al.  Semi-supervised drug-protein interaction prediction from heterogeneous biological spaces , 2010, BMC Systems Biology.

[39]  Ping Zhang,et al.  Interpretable Drug Target Prediction Using Deep Neural Representation , 2018, IJCAI.

[40]  Kunal Roy,et al.  Some case studies on application of “rm2” metrics for judging quality of quantitative structure–activity relationship predictions: Emphasis on scaling of response data , 2013, J. Comput. Chem..

[41]  Yanli Wang,et al.  Predicting drug-target interactions by dual-network integrated logistic matrix factorization , 2017, Scientific Reports.

[42]  Christopher D. Manning,et al.  Effective Approaches to Attention-based Neural Machine Translation , 2015, EMNLP.

[43]  Ming Wen,et al.  Deep-Learning-Based Drug-Target Interaction Prediction. , 2017, Journal of proteome research.

[44]  Nikos Pelekis,et al.  DataStories at SemEval-2017 Task 4: Deep LSTM with Attention for Message-level and Topic-based Sentiment Analysis , 2017, *SEMEVAL.

[45]  Nurshazlyn Mohd Aszemi,et al.  Hyperparameter Optimization in Convolutional Neural Network using Genetic Algorithms , 2019, International Journal of Advanced Computer Science and Applications.

[46]  Phil Blunsom,et al.  A Convolutional Neural Network for Modelling Sentences , 2014, ACL.

[47]  Alán Aspuru-Guzik,et al.  Automatic Chemical Design Using a Data-Driven Continuous Representation of Molecules , 2016, ACS central science.

[48]  Thierry Kogej,et al.  Generating Focused Molecule Libraries for Drug Discovery with Recurrent Neural Networks , 2017, ACS central science.

[49]  Jiancheng Lv,et al.  Automatically Designing CNN Architectures Using Genetic Algorithm for Image Classification , 2018, ArXiv.

[50]  Artem Cherkasov,et al.  PADME: A Deep Learning-based Framework for Drug-Target Interaction Prediction , 2018, ArXiv.

[51]  Hua Yu,et al.  A Systematic Prediction of Multiple Drug-Target Interactions from Chemical, Genomic, and Pharmacological Data , 2012, PloS one.

[52]  Hao Ding,et al.  Collaborative matrix factorization with multiple similarities for predicting drug-target interactions , 2013, KDD.

[53]  Hualiang Jiang,et al.  Structure of Mpro from SARS-CoV-2 and discovery of its inhibitors , 2020, Nature.

[54]  Milan Tuba,et al.  Optimizing Convolutional Neural Network Hyperparameters by Enhanced Swarm Intelligence Metaheuristics , 2020, Algorithms.

[55]  Diab M. Diab,et al.  Using differential evolution for fine tuning naïve Bayesian classifiers and its application for text classification , 2017, Appl. Soft Comput..

[56]  Kevin Leyton-Brown,et al.  Auto-WEKA: combined selection and hyperparameter optimization of classification algorithms , 2012, KDD.

[57]  Kun Zhang,et al.  A Multi-Label Learning Framework for Drug Repurposing , 2019, Pharmaceutics.

[58]  Xing Chen,et al.  Drug-target interaction prediction by random walk on the heterogeneous network. , 2012, Molecular bioSystems.

[59]  Cathy H. Wu,et al.  UniProt: the Universal Protein knowledgebase , 2004, Nucleic Acids Res..

[60]  P. N. Suganthan,et al.  Differential Evolution: A Survey of the State-of-the-Art , 2011, IEEE Transactions on Evolutionary Computation.

[61]  Peter Antal,et al.  VB-MK-LMF: fusion of drugs, targets and interactions using variational Bayesian multiple kernel logistic matrix factorization , 2017, BMC Bioinformatics.

[62]  Xin Wen,et al.  BindingDB: a web-accessible database of experimentally determined protein–ligand binding affinities , 2006, Nucleic Acids Res..

[63]  Rawaa Dawoud Al-Dabbagh,et al.  Algorithmic design issues in adaptive differential evolution schemes: Review and taxonomy , 2018, Swarm Evol. Comput..

[64]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[65]  Tapio Pahikkala,et al.  Toward more realistic drug^target interaction predictions , 2014 .

[66]  Yuhao Wang,et al.  Predicting drug-target interactions using restricted Boltzmann machines , 2013, Bioinform..

[67]  Yasubumi Sakakibara,et al.  Statistical prediction of protein-chemical interactions based on chemical structure and mass spectrometry data , 2007, Bioinform..

[68]  Min Wu,et al.  Drug-target interaction prediction using ensemble learning and dimensionality reduction. , 2017, Methods.

[69]  Ivan Zelinka,et al.  Discrete Set Handling , 2009 .

[70]  Krzysztof Marasek,et al.  Deep Belief Neural Networks and Bidirectional Long-Short Term Memory Hybrid for Speech Recognition , 2015 .

[71]  Yong Zhou,et al.  A Computational-Based Method for Predicting Drug-Target Interactions by Using Stacked Autoencoder Deep Neural Network , 2017, J. Comput. Biol..

[72]  Xiangrong Liu,et al.  Machine Learning for Drug-Target Interaction Prediction , 2018, Molecules.

[73]  Lin He,et al.  Exploring Off-Targets and Off-Systems for Adverse Drug Reactions via Chemical-Protein Interactome — Clozapine-Induced Agranulocytosis as a Case Study , 2011, PLoS Comput. Biol..

[74]  Chang Liu,et al.  Predicting Drug–Target Interactions Using Probabilistic Matrix Factorization , 2013, J. Chem. Inf. Model..

[75]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[76]  Keith C. C. Chan,et al.  Large-scale prediction of drug-target interactions from deep representations , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[77]  Xiaofeng Wang,et al.  Drug–target affinity prediction using graph neural network and contact maps , 2020, RSC advances.

[78]  Derek C. Rose,et al.  Deep Machine Learning - A New Frontier in Artificial Intelligence Research [Research Frontier] , 2010, IEEE Computational Intelligence Magazine.

[79]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[80]  Tao Wang,et al.  Automatic Gait Optimization with Gaussian Process Regression , 2007, IJCAI.

[81]  Chee Keong Kwoh,et al.  Drug-Target Interaction Prediction with Graph Regularized Matrix Factorization , 2017, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[82]  Yang Liu,et al.  Collaborative Security , 2015, ACM Comput. Surv..

[83]  Steven R. Young,et al.  Optimizing deep learning hyper-parameters through an evolutionary algorithm , 2015, MLHPC@SC.

[84]  Diyi Yang,et al.  Hierarchical Attention Networks for Document Classification , 2016, NAACL.

[85]  Hui Yan,et al.  Generalized Mechanism of Field Emission from Nanostructured Semiconductor Film Cathodes , 2017, Scientific Reports.

[86]  Alexander D. MacKerell,et al.  Recent advances in ligand-based drug design: relevance and utility of the conformationally sampled pharmacophore approach. , 2011, Current computer-aided drug design.

[87]  Godfrey C. Onwubolu,et al.  Differential Evolution: A Handbook for Global Permutation-Based Combinatorial Optimization , 2009 .

[88]  Francisco Herrera,et al.  Deep Learning Hyper-parameter Tuning for Sentiment Analysis in Twitter based on Evolutionary Algorithms , 2019, 2019 Federated Conference on Computer Science and Information Systems (FedCSIS).

[89]  M. Gonen,et al.  Concordance probability and discriminatory power in proportional hazards regression , 2005 .

[90]  Ting Liu,et al.  Recent advances in convolutional neural networks , 2015, Pattern Recognit..

[91]  Sung-Bae Cho,et al.  Particle Swarm Optimization-based CNN-LSTM Networks for Forecasting Energy Consumption , 2019, 2019 IEEE Congress on Evolutionary Computation (CEC).

[92]  Jing Li,et al.  Drug Target Predictions Based on Heterogeneous Graph Inference , 2012, Pacific Symposium on Biocomputing.

[93]  Yoshua Bengio,et al.  Algorithms for Hyper-Parameter Optimization , 2011, NIPS.

[94]  Ali Masoudi-Nejad,et al.  Drug–target interaction prediction via chemogenomic space: learning-based methods , 2014, Expert opinion on drug metabolism & toxicology.

[95]  Di Wu,et al.  DeepAffinity: Interpretable Deep Learning of Compound-Protein Affinity through Unified Recurrent and Convolutional Neural Networks , 2018, bioRxiv.

[96]  Yutaka Akiyama,et al.  Efficient hyperparameter optimization by using Bayesian optimization for drug-target interaction prediction , 2017, 2017 IEEE 7th International Conference on Computational Advances in Bio and Medical Sciences (ICCABS).

[97]  Arzucan Özgür,et al.  DeepDTA: deep drug–target binding affinity prediction , 2018, Bioinform..

[98]  Yoshihiro Yamanishi,et al.  Prediction of drug–target interaction networks from the integration of chemical and genomic spaces , 2008, ISMB.

[99]  Arzucan Özgür,et al.  A novel methodology on distributed representations of proteins using their interacting ligands , 2018, Bioinform..

[100]  Chunyan Miao,et al.  Neighborhood Regularized Logistic Matrix Factorization for Drug-Target Interaction Prediction , 2016, PLoS Comput. Biol..

[101]  Elif Ozkirimli,et al.  WideDTA: prediction of drug-target binding affinity , 2019, ArXiv.

[102]  Jun Zhao,et al.  Recurrent Convolutional Neural Networks for Text Classification , 2015, AAAI.

[103]  Jürgen Schmidhuber,et al.  Learning to forget: continual prediction with LSTM , 1999 .

[104]  Chuang Liu,et al.  Prediction of Drug-Target Interactions and Drug Repositioning via Network-Based Inference , 2012, PLoS Comput. Biol..

[105]  Lihong Li,et al.  An Empirical Evaluation of Thompson Sampling , 2011, NIPS.

[106]  Jens Sadowski,et al.  Comparison of Support Vector Machine and Artificial Neural Network Systems for Drug/Nondrug Classification , 2003, J. Chem. Inf. Comput. Sci..

[107]  Yoshihiro Yamanishi,et al.  Chemogenomic approaches to infer drug-target interaction networks. , 2013, Methods in molecular biology.

[108]  Wu Zhong,et al.  Remdesivir and chloroquine effectively inhibit the recently emerged novel coronavirus (2019-nCoV) in vitro , 2020, Cell Research.

[109]  Cícero Nogueira dos Santos,et al.  Deep Convolutional Neural Networks for Sentiment Analysis of Short Texts , 2014, COLING.

[110]  Alan L. Yuille,et al.  Genetic CNN , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[111]  Kiyoharu Aizawa,et al.  Efficient Optimization of Convolutional Neural Networks Using Particle Swarm Optimization , 2017, 2017 IEEE Third International Conference on Multimedia Big Data (BigMM).

[112]  Salvatore Alaimo,et al.  Drug–target interaction prediction through domain-tuned network-based inference , 2013, Bioinform..

[113]  Yoshua Bengio,et al.  Random Search for Hyper-Parameter Optimization , 2012, J. Mach. Learn. Res..

[114]  Geoffrey E. Hinton,et al.  Speech recognition with deep recurrent neural networks , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[115]  Mindy I. Davis,et al.  Comprehensive analysis of kinase inhibitor selectivity , 2011, Nature Biotechnology.

[116]  Janez Brest,et al.  Multi-Objective Differential Evolution for feature selection in Facial Expression Recognition systems , 2017, Expert Syst. Appl..

[117]  Tao Xu,et al.  Making Sense of Large-Scale Kinase Inhibitor Bioactivity Data Sets: A Comparative and Integrative Analysis , 2014, J. Chem. Inf. Model..

[118]  Josh Harguess,et al.  Image Classification Using Generative Neuro Evolution for Deep Learning , 2015, 2015 IEEE Winter Conference on Applications of Computer Vision.

[119]  Shi-Hua Zhang,et al.  DrugE-Rank: improving drug–target interaction prediction of new candidate drugs or targets by ensemble learning to rank , 2016, Bioinform..

[120]  Kevin Leyton-Brown,et al.  Sequential Model-Based Optimization for General Algorithm Configuration , 2011, LION.

[121]  Iddo Greental,et al.  Genetic algorithms for evolving deep neural networks , 2014, GECCO.

[122]  Manoj Kumar Gupta,et al.  A comprehensive review of feature based methods for drug target interaction prediction , 2019, J. Biomed. Informatics.

[123]  Houfeng Wang,et al.  Bidirectional Recurrent Convolutional Neural Network for Relation Classification , 2016, ACL.

[124]  Nando de Freitas,et al.  Taking the Human Out of the Loop: A Review of Bayesian Optimization , 2016, Proceedings of the IEEE.