Information-theoretic sensitivity analysis: a general method for credit assignment in complex networks

Most systems can be represented as networks that couple a series of nodes to each other via one or more edges, with typically unknown equations governing their quantitative behaviour. A major question then pertains to the importance of each of the elements that act as system inputs in determining the output(s). We show that any such system can be treated as a ‘communication channel’ for which the associations between inputs and outputs can be quantified via a decomposition of their mutual information into different components characterizing the main effect of individual inputs and their interactions. Unlike variance-based approaches, our novel methodology can easily accommodate correlated inputs.

[1]  William Bialek,et al.  Entropy and information in neural spike trains: progress on the sampling problem. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[2]  Jörg Stelling,et al.  Systems interface biology , 2006, Journal of The Royal Society Interface.

[3]  Michael J. Berry,et al.  Weak pairwise correlations imply strongly correlated network states in a neural population , 2005, Nature.

[4]  E. Davidson.  The Regulatory Genome: Gene Regulatory Networks In Development And Evolution , 2006 .

[5]  Marcelo A. Montemurro,et al.  Tight Data-Robust Bounds to Mutual Information Combining Shuffling and Model Selection Techniques , 2007, Neural Computation.

[6]  Hong Wang,et al.  Insights into the behaviour of systems biology models from dynamic sensitivity and identifiability analysis: a case study of an NF-kappaB signalling pathway. , 2006, Molecular bioSystems.

[7]  Stefano Tarantola,et al.  Sensitivity Analysis in Practice: A Guide to Assessing Scientific Models , 2004 .

[8]  A. Saltelli,et al.  Sensitivity Anaysis as an Ingredient of Modeling , 2000 .

[9]  Stefano Panzeri,et al.  Analytical estimates of limited sampling biases in different information measures. , 1996, Network.

[10]  Amiel Feinstein,et al.  Transmission of Information. , 1962 .

[11]  J. Doyle,et al.  Bow Ties, Metabolism and Disease , 2022 .

[12]  D. Kell.  Systems biology, metabolic modelling and metabolomics in drug discovery and development. , 2006, Drug discovery today.

[13]  Douglas J. Morrice,et al.  An information theoretic approach to computer simulation sensitivity analysis , 1993, WSC '93.

[14]  William Bialek,et al.  The Information Content of Receptive Fields , 2003, Neuron.

[15]  William J. McGill.  Multivariate information transmission , 1954, Trans. IRE Prof. Group Inf. Theory.

[16]  James R. Johnson,et al.  Oscillations in NF-κB Signaling Control the Dynamics of Gene Expression , 2004, Science.

[17]  A. Barabasi,et al.  Network biology: understanding the cell's functional organization , 2004, Nature Reviews Genetics.

[18]  A. Hoffmann,et al.  The I (cid:1) B –NF-(cid:1) B Signaling Module: Temporal Control and Selective Gene Activation , 2022 .

[19]  B. Palsson.  Systems Biology: Properties of Reconstructed Networks , 2006 .

[20]  Eric H. Davidson,et al.  Gene Regulatory Networks for Development: What They Are, How They Work, and What They Mean , 2006 .

[21]  A. Saltelli,et al.  Sensitivity analysis for chemical models. , 2005, Chemical reviews.

[22]  E. Jaynes.  Information Theory and Statistical Mechanics , 1957 .

[23]  D. Broomhead,et al.  Sensitivity analysis of parameters controlling oscillatory signalling in the NFk B pathway : the roles of IKK and I k B a , 2004 .

[24]  Axel Kowald,et al.  Systems Biology in Practice: Concepts, Implementation and Application , 2005 .

[25]  D. Fell,et al.  The small world inside large metabolic networks , 2000, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[26]  D. Kell.  Metabolomics, modelling and machine learning in systems biology – towards an understanding of the languages of cells , 2006, The FEBS journal.

[27]  Sidney Addelman,et al.  trans-Dimethanolbis(1,1,1-trifluoro-5,5-dimethylhexane-2,4-dionato)zinc(II) , 2008, Acta crystallographica. Section E, Structure reports online.

[28]  K E Willard,et al.  Probabilistic sensitivity analysis methods for general decision models. , 1986, Computers and biomedical research, an international journal.

[29]  Shun-ichi Amari,et al.  Information geometry on hierarchy of probability distributions , 2001, IEEE Trans. Inf. Theory.

[30]  Michael Satosi Watanabe,et al.  Information Theoretical Analysis of Multivariate Correlation , 1960, IBM J. Res. Dev..

[31]  Mill Johannes G.A. Van,et al.  Transmission Of Information , 1961 .

[32]  I. Sobola,et al.  Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates , 2001 .

[33]  Yukio Nakamura,et al.  Genetic approaches in mice to understand Rel/NF-κB and IκB function: transgenics and knockouts , 1999, Oncogene.

[34]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[35]  A. Wagner.  Robustness and Evolvability in Living Systems , 2005 .

[36]  Darren J. Wilkinson.  Stochastic Modelling for Systems Biology , 2006 .

[37]  Stefano Tarantola,et al.  Sensitivity Analysis as an Ingredient of Modeling , 2000 .

[38]  D. S. Broomhead,et al.  Synergistic control of oscillations in the NFk B signalling pathway , 2000 .

[39]  Ga Miller,et al.  Note on the bias of information estimates , 1955 .

[40]  Thomas M. Cover,et al.  Elements of Information Theory: Cover/Elements of Information Theory, Second Edition , 2005 .

[41]  Yoichiro Takada.  ON THE MATHEMATICAL THEORY OF COMMUNICATION , 1954 .

[42]  Yukiko Matsuoka,et al.  Using process diagrams for the graphical representation of biological networks , 2005, Nature Biotechnology.

[43]  Hamid Bolouri,et al.  A data integration methodology for systems biology. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[44]  H. Rabitz,et al.  General foundations of high‐dimensional model representations , 1999 .

[45]  E T Rolls,et al.  Correlations and the encoding of information in the nervous system , 1999, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[46]  David S. Broomhead,et al.  Sensitivity analysis of parameters controlling oscillatory signalling in the NF-/sub K/Bpathway: the roles of IKK and I/sub K/B/sub alpha/ , 2004 .

[47]  D S Broomhead,et al.  Synergistic control of oscillations in the NF-kappaB signalling pathway. , 2005, Systems biology.

[48]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .