Employing decomposable partially observable Markov decision processes to control gene regulatory networks

OBJECTIVE Formulate the induction and control of gene regulatory networks (GRNs) from gene expression data using Partially Observable Markov Decision Processes (POMDPs). METHODS AND MATERIAL Different approaches exist to model GRNs; they are mostly simulated as mathematical models that represent relationships between genes. Actually, it has been realized that biological functions at the cellular level are controlled by genes; thus, by controlling the behavior of genes, it is possible to regulate these biological functions. The GRN control problem has been studied mostly with the aid of probabilistic Boolean networks, and corresponding control policies have been devised. Though turns into a more challenging problem, we argue that partial observability would be a more natural and realistic method for handling the control of GRNs. Partial observability is a fundamental aspect of the problem; it is mostly ignored and substituted by assumption that states of GRN are known precisely, prescribed as full observability. We propose a method for the construction of POMDP model of GRN from only raw gene expression data which is original and novel. Then, we introduce a novel approach to decompose/factor the POMDP model into sub-POMDP's in order to solve it efficiently with the help of divide-and-conquer strategy. RESULTS In order to demonstrate the effectiveness of the proposed solution we experimented with two synthetic network and one real network data from the literature. We also conducted two sets of separate experiments used to explore the impact of network connectivity and data order to our approach CONCLUSIONS: The reported test results using both synthetic and real GRNs are promising in demonstrating the applicability, effectiveness and efficiency of the proposed approach. This is due to the fact that partial observability fits well to the problem of noisy acquisition of gene expression data as there are technological limitations to measure precisely exact expression levels of genes.

[1]  J. Raser,et al.  Noise in Gene Expression: Origins, Consequences, and Control , 2005, Science.

[2]  L. Aravind,et al.  Interplay between gene expression noise and regulatory network architecture. , 2012, Trends in genetics : TIG.

[3]  Edward R. Dougherty,et al.  CAN MARKOV CHAIN MODELS MIMIC BIOLOGICAL REGULATION , 2002 .

[4]  M. West,et al.  Sparse graphical models for exploring gene expression data , 2004 .

[5]  P. Poupart.  Exploiting structure to efficiently solve large scale partially observable Markov decision processes , 2005 .

[6]  Nikos A. Vlassis,et al.  Perseus: Randomized Point-based Value Iteration for POMDPs , 2005, J. Artif. Intell. Res..

[7]  Reda Alhajj,et al.  Large-scale approximate intervention strategies for Probabilistic Boolean Networks as models of gene regulation , 2008, 2008 8th IEEE International Conference on BioInformatics and BioEngineering.

[8]  Reda Alhajj,et al.  Markov decision processes based optimal control policies for probabilistic boolean networks , 2004, Proceedings. Fourth IEEE Symposium on Bioinformatics and Bioengineering.

[9]  Anton Crombach,et al.  Efficient Reverse-Engineering of a Developmental Gene Regulatory Network , 2012, PLoS Comput. Biol..

[10]  Richard Bellman,et al.  Adaptive Control Processes: A Guided Tour , 1961, The Mathematical Gazette.

[11]  Stein Aerts,et al.  iRegulon: From a Gene List to a Gene Regulatory Network Using Large Motif and Track Collections , 2014, PLoS Comput. Biol..

[12]  Daniel J. Kvitek,et al.  Whole Genome, Whole Population Sequencing Reveals That Loss of Signaling Networks Is the Major Adaptive Strategy in a Constant Environment , 2013, PLoS genetics.

[13]  Reda Alhajj,et al.  Scalable approach for effective control of gene regulatory networks , 2010, Artif. Intell. Medicine.

[14]  Daniel Bryce,et al.  Planning for Gene Regulatory Network Intervention , 2006, 2006 IEEE/NLM Life Science Systems and Applications Workshop.

[15]  Reda Alhajj,et al.  Optimal Multi-Objective Control Method for Discrete Genetic Regulatory Networks , 2006, Sixth IEEE Symposium on BioInformatics and BioEngineering (BIBE'06).

[16]  Peter J. Bickel,et al.  Comparative analysis of regulatory information and circuits across distant species , 2014, Nature.

[17]  Aniruddha Datta,et al.  Robust Intervention in Probabilistic Boolean Networks , 2007, 2007 American Control Conference.

[18]  Tommi S. Jaakkola,et al.  Using Graphical Models and Genomic Expression Data to Statistically Validate Models of Genetic Regulatory Networks , 2000, Pacific Symposium on Biocomputing.

[19]  V. Anne Smith,et al.  Using Bayesian Network Inference Algorithms to Recover Molecular Genetic Regulatory Networks , 2002 .

[20]  Claude Desplan,et al.  Opposite Feedbacks in the Hippo Pathway for Growth Control and Neural Fate , 2013, Science.

[21]  N. Sampas,et al.  Molecular classification of cutaneous malignant melanoma by gene expression profiling , 2000, Nature.

[22]  Stuart A. Kauffman,et al.  The origins of order , 1993 .

[23]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[24]  Aidong Zhang,et al.  Protein Interaction Networks: Computational Analysis , 2009 .

[25]  David R. Thompson,et al.  Generating Exponentially Smaller POMDP Models Using Conditionally Irrelevant Variable Abstraction , 2007, ICAPS.

[26]  E. Dougherty,et al.  Gene perturbation and intervention in probabilistic Boolean networks. , 2002, Bioinformatics.

[27]  Aniruddha Datta,et al.  Intervention in context-sensitive probabilistic Boolean networks , 2005, Bioinform..

[28]  Yurong Xu,et al.  Episodic task learning in Markov decision processes , 2011, Artificial Intelligence Review.

[29]  Helen M. Byrne,et al.  Boolean Modelling Reveals New Regulatory Connections between Transcription Factors Orchestrating the Development of the Ventral Spinal Cord , 2014, PloS one.

[30]  Aniruddha Datta,et al.  Optimal infinite horizon control for probabilistic Boolean networks , 2006, 2006 American Control Conference.

[31]  H Matsuno,et al.  Hybrid Petri net representation of gene regulatory network. , 1999, Pacific Symposium on Biocomputing. Pacific Symposium on Biocomputing.

[32]  Mark R. Green,et al.  Targeting targeted therapy. , 2004, The New England journal of medicine.

[33]  Reda Alhajj,et al.  Partially Observable Gene Regulatory Network Control without a Boundary on Horizon , 2012, 2012 IEEE 24th International Conference on Tools with Artificial Intelligence.

[34]  Reda Alhajj,et al.  Automated Large-Scale Control of Gene Regulatory Networks , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[35]  Daniel E. Runcie,et al.  The Impact of Gene Expression Variation on the Robustness and Evolvability of a Developmental Gene Regulatory Network , 2013, PLoS biology.

[36]  C. Guestrin,et al.  Solving Factored MDPs with Hybrid State and Action Variables , 2006, J. Artif. Intell. Res..

[37]  Aniruddha Datta,et al.  External Control in Markovian Genetic Regulatory Networks , 2004, Machine Learning.

[38]  Yu Lasheng,et al.  Research on task decomposition and state abstraction in reinforcement learning , 2012, Artificial Intelligence Review.

[39]  Leslie Pack Kaelbling,et al.  Planning and Acting in Partially Observable Stochastic Domains , 1998, Artif. Intell..

[40]  Craig Boutilier,et al.  Exploiting Structure in Policy Construction , 1995, IJCAI.

[41]  Eva Onaindia,et al.  New prioritized value iteration for Markov decision processes , 2012, Artificial Intelligence Review.

[42]  Anthony R. Cassandra,et al.  Optimal Policies for Partially Observable Markov Decision Processes , 1994 .