Bayesian Regression With Network Prior: Optimal Bayesian Filtering Perspective

The recently introduced intrinsically Bayesian robust filter (IBRF) provides fully optimal filtering relative to a prior distribution over an uncertainty class of joint random process models, whereas formerly the theory was limited to model-constrained Bayesian robust filters, for which optimization was limited to the filters that are optimal for models in the uncertainty class. This paper extends the IBRF theory to the situation where there are both a prior on the uncertainty class and sample data. The result is optimal Bayesian filtering (OBF), where optimality is relative to the posterior distribution derived from the prior and the data. The IBRF theories for effective characteristics and canonical expansions extend to the OBF setting. A salient focus of the present study is to demonstrate the advantages of Bayesian regression within the OBF setting over the classical Bayesian approach in the context of linear Gaussian models.

[1]  Radford M. Neal Pattern Recognition and Machine Learning , 2007, Technometrics.

[2]  L. Tierney Markov Chains for Exploring Posterior Distributions , 1994 .

[3]  Simon Rogers,et al.  A Bayesian regression approach to the inference of regulatory networks from gene expression data , 2005, Bioinform..

[4]  Patrik D'haeseleer,et al.  Linear Modeling of mRNA Expression Levels During CNS Development and Injury , 1998, Pacific Symposium on Biocomputing.

[5]  C. Holmes,et al.  Approximate Models and Robust Decisions , 2014, 1402.6118.

[6]  Aniruddha Datta,et al.  Robust Intervention in Probabilistic Boolean Networks , 2007, 2007 American Control Conference.

[7]  O. Hino,et al.  Tsc tumour suppressor proteins antagonize amino-acid–TOR signalling , 2002, Nature Cell Biology.

[8]  Saleem A. Kassam,et al.  Robust Wiener filters , 1977 .

[9]  H. Vincent Poor,et al.  Robust matched filters , 1983, IEEE Trans. Inf. Theory.

[10]  John K Kruschke,et al.  Bayesian data analysis. , 2010, Wiley interdisciplinary reviews. Cognitive science.

[11]  Kevin P. Murphy,et al.  Machine learning - a probabilistic perspective , 2012, Adaptive computation and machine learning series.

[12]  Edward R. Dougherty,et al.  Optimal classifiers with minimum expected error within a Bayesian framework - Part I: Discrete and Gaussian models , 2013, Pattern Recognit..

[13]  Edward R. Dougherty,et al.  Bayesian robust optimal linear filters , 2001, Signal Process..

[14]  Pradeep Ravikumar,et al.  On Poisson Graphical Models , 2013, NIPS.

[15]  F. Mazzocchi Complexity in biology , 2008, EMBO Reports.

[16]  M. Yuan,et al.  Model selection and estimation in regression with grouped variables , 2006 .

[17]  Masao Nagasaki,et al.  Inference of Gene Regulatory Networks Incorporating Multi-Source Biological Knowledge via a State Space Model with L1 Regularization , 2014, PloS one.

[18]  G. Casella,et al.  The Bayesian Lasso , 2008 .

[19]  Korbinian Strimmer,et al.  An empirical Bayes approach to inferring large-scale gene association networks , 2005, Bioinform..

[20]  D. Bernardo,et al.  A Yeast Synthetic Network for In Vivo Assessment of Reverse-Engineering and Modeling Approaches , 2009, Cell.

[21]  Edward R. Dougherty,et al.  Quantifying the Objective Cost of Uncertainty in Complex Dynamical Systems , 2013, IEEE Transactions on Signal Processing.

[22]  Donald E. Ingber,et al.  The structural and mechanical complexity of cell-growth control , 1999, Nature Cell Biology.

[23]  Aniruddha Datta,et al.  Optimal infinite horizon control for probabilistic Boolean networks , 2006, 2006 American Control Conference.

[24]  Michael I. Jordan,et al.  Graphical Models, Exponential Families, and Variational Inference , 2008, Found. Trends Mach. Learn..

[25]  Dapeng Wu,et al.  Joint Association Graph Screening and Decomposition for Large-Scale Linear Dynamical Systems , 2014, IEEE Transactions on Signal Processing.

[26]  Jianping Hua,et al.  Tracking transcriptional activities with high-content epifluorescent imaging. , 2012, Journal of biomedical optics.

[27]  Stuart A. Kauffman,et al.  ORIGINS OF ORDER , 2019, Origins of Order.

[28]  Alexander J. Hartemink,et al.  Informative Structure Priors: Joint Learning of Dynamic Regulatory Networks from Multiple Types of Data , 2004, Pacific Symposium on Biocomputing.

[29]  Jeanne M O Eloundou-Mbebi,et al.  Gene regulatory network inference using fused LASSO on multiple data sets , 2016, Scientific Reports.

[30]  Aniruddha Datta,et al.  Optimal Intervention in Markovian Gene Regulatory Networks With Random-Length Therapeutic Response to Antitumor Drug , 2013, IEEE Transactions on Biomedical Engineering.

[31]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .

[32]  Edward R. Dougherty,et al.  Effect of Function Perturbation on the Steady-State Distribution of Genetic Regulatory Networks: Optimal Structural Intervention , 2008, IEEE Transactions on Signal Processing.

[33]  Diego di Bernardo,et al.  Inference of gene regulatory networks and compound mode of action from time course gene expression profiles , 2006, Bioinform..

[34]  Pravin Varaiya,et al.  Stochastic Systems: Estimation, Identification, and Adaptive Control , 1986 .

[35]  Florian Steinke,et al.  Bayesian Inference and Optimal Design in the Sparse Linear Model , 2007, AISTATS.

[36]  T. J. Mitchell,et al.  Bayesian Variable Selection in Linear Regression , 1988 .

[37]  Koeng-Mo Sung,et al.  On robust Kalman filtering with forgetting factor for sequential speech analysis , 1997, Signal Process..

[38]  Anne-Laure Boulesteix,et al.  Over-optimism in bioinformatics research , 2010, Bioinform..

[39]  Jesper Tegnér,et al.  Reverse engineering gene networks using singular value decomposition and robust regression , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[40]  Edward R. Dougherty,et al.  Design and Analysis of Robust Binary Filters in the Context of a Prior Distribution for the States of Nature , 2004, Journal of Mathematical Imaging and Vision.

[41]  V. S. Pugachev,et al.  Theory of Random Functions: And Its Application to Control Problems , 2013 .

[42]  Edward R. Dougherty,et al.  Incorporation of Biological Pathway Knowledge in the Construction of Priors for Optimal Bayesian Classification , 2014, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[43]  Edwin T. Jaynes Prior Probabilities , 2010, Encyclopedia of Machine Learning.

[44]  M. Yuan,et al.  Model selection and estimation in the Gaussian graphical model , 2007 .

[45]  Edward R. Dougherty,et al.  Random processes for image and signal processing , 1998, SPIE / IEEE series on imaging science and engineering.

[46]  Edward R. Dougherty,et al.  Intrinsically Optimal Bayesian Robust Filtering , 2014, IEEE Transactions on Signal Processing.

[47]  Arnold Zellner,et al.  Models, prior information, and Bayesian analysis☆ , 1996 .

[48]  H. Jeffreys An invariant form for the prior probability in estimation problems , 1946, Proceedings of the Royal Society of London. Series A. Mathematical and Physical Sciences.

[49]  H. Vincent Poor,et al.  On minimax robustness: A general approach and applications , 1984, IEEE Trans. Inf. Theory.

[50]  B. Mallick VARIABLE SELECTION FOR REGRESSION MODELS , 2016 .

[51]  Edward R. Dougherty,et al.  Optimal classifiers with minimum expected error within a Bayesian framework - Part II: Properties and performance analysis , 2013, Pattern Recognit..

[52]  Aniruddha Datta,et al.  Bayesian Robustness in the Control of Gene Regulatory Networks , 2007, IEEE Transactions on Signal Processing.

[53]  Diogo M. Camacho,et al.  Wisdom of crowds for robust gene network inference , 2012, Nature Methods.

[54]  K. Vastola,et al.  Robust Wiener-Kolmogorov theory , 1984, IEEE Trans. Inf. Theory.

[55]  Edward R. Dougherty,et al.  MCMC implementation of the optimal Bayesian classifier for non-Gaussian models: model-based RNA-Seq classification , 2014, BMC Bioinformatics.

[56]  H. Poor On robust wiener filtering , 1980 .

[57]  Ashutosh Kumar Singh,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2010 .