Sparse solution of nonnegative least squares problems with applications in the construction of probabilistic Boolean networks

In this paper, we consider finding a sparse solution of nonnegative least squares problems with a linear equality constraint. We propose a projection-based gradient descent method for solving huge size constrained least squares problems. Traditional Newton-based methods require solving a linear system. However, when the matrix is huge, it is neither practical to store it nor possible to solve it in a reasonable time. We therefore propose a matrix-free iterative scheme for solving the minimizer of the captured problem. This iterative scheme can be explained as a projection-based gradient descent method. In each iteration, a projection operation is performed to ensure the solution is feasible. The projection operation is equivalent to a shrinkage operator, which can guarantee the sparseness of the solution obtained. We show that the objective function is decreasing. We then apply the proposed method to the inverse problem of constructing a probabilistic Boolean network. Numerical examples are then given to illustrate both the efficiency and effectiveness of our proposed method. Copyright © 2015 John Wiley & Sons, Ltd.

[1]  Y. Ye,et al.  Sparse Portfolio Selection via Quasi-Norm Regularization , 2013, 1312.6350.

[2]  Tak Kuen Siu,et al.  On modeling credit defaults: A probabilistic Boolean network approach , 2013, Risk Decis. Anal..

[3]  I. Daubechies,et al.  Sparse and stable Markowitz portfolios , 2007, Proceedings of the National Academy of Sciences.

[4]  Jack Xin,et al.  A Method for Finding Structured Sparse Solutions to Nonnegative Least Squares Problems with Applications , 2013, SIAM J. Imaging Sci..

[5]  D. A. Baxter,et al.  Mathematical Modeling of Gene Networks , 2000, Neuron.

[6]  Stefania Bellavia,et al.  An interior point Newton‐like method for non‐negative least‐squares problems with degenerate solution , 2006, Numer. Linear Algebra Appl..

[7]  José M. Bioucas-Dias,et al.  Fast Image Recovery Using Variable Splitting and Constrained Optimization , 2009, IEEE Transactions on Image Processing.

[8]  W. Ching,et al.  Construction of Probabilistic Boolean Networks from a Prescribed Transition Probability Matrix: A Maximum Entropy Rate Approach , 2011 .

[9]  Stuart A. Kauffman,et al.  The origins of order , 1993 .

[10]  Edward R. Dougherty,et al.  From Boolean to probabilistic Boolean networks as models of genetic regulatory networks , 2002, Proc. IEEE.

[11]  E.J. Candes,et al.  An Introduction To Compressive Sampling , 2008, IEEE Signal Processing Magazine.

[12]  S. Kauffman.  Metabolic stability and epigenesis in randomly constructed genetic nets. , 1969, Journal of theoretical biology.

[13]  Yin Zhang,et al.  Interior-Point Gradient Method for Large-Scale Totally Nonnegative Least Squares Problems , 2005 .

[14]  Xiaoming Yuan,et al.  Linearized Alternating Direction Method of Multipliers for Constrained Linear Least-Squares Problem , 2012 .

[15]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[16]  Raymond H. Chan,et al.  Journal of Computational and Applied Mathematics a Reduced Newton Method for Constrained Linear Least-squares Problems , 2022 .

[17]  Edward R. Dougherty,et al.  Probabilistic Boolean Networks - The Modeling and Control of Gene Regulatory Networks , 2010 .

[18]  Patrick L. Combettes,et al.  Signal Recovery by Proximal Forward-Backward Splitting , 2005, Multiscale Model. Simul..

[19]  I. Konnov.  Application of the Proximal Point Method to Nonmonotone Equilibrium Problems , 2003 .

[20]  Hidde de Jong,et al.  Modeling and Simulation of Genetic Regulatory Systems: A Literature Review , 2002, J. Comput. Biol..

[21]  Michael K. Ng,et al.  An approximation method for solving the steady-state probability distribution of probabilistic Boolean networks , 2007, Bioinform..

[22]  Inderjit S. Dhillon,et al.  A non-monotonic method for large-scale non-negative least squares , 2013, Optim. Methods Softw..

[23]  Martin Jaggi,et al.  Revisiting Frank-Wolfe: Projection-Free Sparse Convex Optimization , 2013, ICML.

[24]  Philip Wolfe,et al.  An algorithm for quadratic programming , 1956 .

[25]  Michael Elad,et al.  On the Uniqueness of Nonnegative Sparse Solutions to Underdetermined Systems of Equations , 2008, IEEE Transactions on Information Theory.

[26]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[27]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[28]  M. Ng,et al.  EFFICIENT BOX-CONSTRAINED TV-TYPE-l^1 ALGORITHMS FOR RESTORING IMAGES WITH IMPULSE NOISE , 2013 .

[29]  W. Ching,et al.  Generating probabilistic Boolean networks from a prescribed transition probability matrix. , 2009, IET systems biology.

[30]  J. Navarro-Pedreño.  Numerical Methods for Least Squares Problems , 1996 .

[31]  Matthias Hein,et al.  Non-negative least squares for high-dimensional linear models: consistency and sparse recovery without regularization , 2012, 1205.0953.

[32]  Xiaojun Chen,et al.  Smoothing Methods and Semismooth Methods for Nondifferentiable Operator Equations , 2000, SIAM J. Numer. Anal..

[33]  T. Chan,et al.  WAVELET INPAINTING BY NONLOCAL TOTAL VARIATION , 2010 .

[34]  W. Ching,et al.  On Construction of Sparse Probabilistic Boolean Networks , 2012 .

[35]  Robert H. Halstead,et al.  Matrix Computations , 2011, Encyclopedia of Parallel Computing.

[36]  Michael Elad,et al.  Stable recovery of sparse overcomplete representations in the presence of noise , 2006, IEEE Transactions on Information Theory.