On Phase Transitions in Learning Sparse Networks

In this paper we study the identification of sparse interaction networks as a machine learning problem. Sparsity mean that we are provided with a small data set and a high number of unknown components of the system, most of which are zero. Under these circumstances, a model needs to be learned that fits the underlying system, capable of generalization. This corresponds to the student-teacher setting in machine learning. In the first part of this paper we introduce a learning algorithm, based on L 1 -minimization, to identify interaction networks from poor data and analyze its dynamics with respect to phase transitions. The efficiency of the algorithm is measured by the generalization error, which represents the probability that the student is a good fit to the teacher. In the second part of this paper we show that from a system with a specific system size value the generalization error of other system sizes can be estimated. A comparison with a set of simulation experiments show a very good fit.

[1]  J. Tyson,et al.  Modeling the control of DNA replication in fission yeast. , 1997, Proceedings of the National Academy of Sciences of the United States of America.

[2]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[3]  Hidde de Jong,et al.  Modeling and Simulation of Genetic Regulatory Systems: A Literature Review , 2002, J. Comput. Biol..

[4]  H. D. Jong,et al.  Qualitative simulation of genetic regulatory networks using piecewise-linear models , 2004, Bulletin of mathematical biology.

[5]  Marc Gyssens,et al.  The Identification of Dynamic Gene-Protein Networks , 2006, KDECB.

[6]  L. Glass,et al.  The logical analysis of continuous, non-linear biochemical control networks. , 1973, Journal of theoretical biology.

[7]  Yvan Saeys,et al.  Knowledge Discovery and Emergent Complexity in Bioinformatics , 2006, KDECB.

[8]  J. Fuchs.  More on sparse representations in arbitrary bases , 2003 .

[9]  Jesper Tegnér,et al.  Reverse engineering gene networks using singular value decomposition and robust regression , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[10]  Jean-Jacques Fuchs,et al.  On sparse representations in arbitrary redundant bases , 2004, IEEE Transactions on Information Theory.

[11]  Ralf Peeters,et al.  On the identification of sparse gene regulatory networks , 2004 .