A novel one-layer recurrent neural network for the l1-regularized least square problem

The l1-regularized least square problem has been considered in diverse fields. However, finding its solution is exacting as its objective function is not differentiable. In this paper, we propose a new one-layer neural network to find the optimal solution of the l1-regularized least squares problem. To solve the problem, we first convert it into a smooth quadratic minimization by splitting the desired variable into its positive and negative parts. Accordingly, a novel neural network is proposed to solve the resulting problem, which is guaranteed to converge to the solution of the problem. Furthermore, the rate of the convergence is dependent on a scaling parameter, not to the size of datasets. The proposed neural network is further adjusted to encompass the total variation regularization. Extensive experiments on the l1 and total variation regularized problems illustrate the reasonable performance of the proposed neural network. © 2018 Elsevier B.V.

[1]  Jiming Liu,et al.  Piecewise-constant and low-rank approximation for identification of recurrent copy number variations , 2014, Bioinform..

[2]  Laurent Condat,et al.  A Primal–Dual Splitting Method for Convex Optimization Involving Lipschitzian, Proximable and Linear Composite Terms , 2012, Journal of Optimization Theory and Applications.

[3]  Shuai Li,et al.  Neural Dynamics for Cooperative Control of Redundant Robot Manipulators , 2018, IEEE Transactions on Industrial Informatics.

[4]  José R. Dorronsoro,et al.  Group Fused Lasso , 2013, ICANN.

[5]  Ajay N. Jain,et al.  Genomic and transcriptional aberrations linked to breast cancer pathophysiologies. , 2006, Cancer cell.

[6]  Yunhai Xiao,et al.  Non-smooth equations based method for l 1 -norm problems with applications to compressed sensing , 2011 .

[7]  Yong Yu,et al.  Robust Subspace Segmentation by Low-Rank Representation , 2010, ICML.

[8]  O. Mangasarian.  Equivalence of the Complementarity Problem to a System of Nonlinear Equations , 1976 .

[9]  Majid Mohammadi,et al.  A robust Correntropy-based method for analyzing multisample aCGH data. , 2015, Genomics.

[10]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[11]  Jean-Jacques Fuchs,et al.  On sparse representations in arbitrary redundant bases , 2004, IEEE Transactions on Information Theory.

[12]  D. Pinkel,et al.  Array comparative genomic hybridization and its applications in cancer , 2005, Nature Genetics.

[13]  L. Feuk,et al.  Structural variation in the human genome , 2006, Nature Reviews Genetics.

[14]  Haesun Park,et al.  Fast Active-set-type Algorithms for L1-regularized Linear Regression , 2010, AISTATS.

[15]  René Vidal,et al.  Sparse Subspace Clustering: Algorithm, Theory, and Applications , 2012, IEEE transactions on pattern analysis and machine intelligence.

[16]  Guillermo Sapiro,et al.  Online dictionary learning for sparse coding , 2009, ICML '09.

[17]  R. Bellman.  The stability of solutions of linear differential equations , 1943 .

[18]  George E. Liu,et al.  A Genome-Wide Analysis of Array-Based Comparative Genomic Hybridization (CGH) Data to Detect Intra-Species Variations and Evolutionary Relationships , 2009, PloS one.

[19]  Stephen P. Boyd,et al.  An Interior-Point Method for Large-Scale $\ell_1$-Regularized Least Squares , 2007, IEEE Journal of Selected Topics in Signal Processing.

[20]  L. Zanni,et al.  Accelerating gradient projection methods for ℓ1-constrained signal recovery by steplength selection rules , 2009 .

[21]  Rajat Raina,et al.  Efficient sparse coding algorithms , 2006, NIPS.

[22]  René Vidal,et al.  Sparse subspace clustering , 2009, CVPR.

[23]  Yinhe Cao,et al.  Exploiting noise in array CGH data to improve detection of DNA copy number change , 2007, Nucleic acids research.

[24]  Christian A. Rees,et al.  Microarray analysis reveals a major direct role of DNA copy number alteration in the transcriptional program of human breast tumors , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[25]  Mário A. T. Figueiredo,et al.  Gradient Projection for Sparse Reconstruction: Application to Compressed Sensing and Other Inverse Problems , 2007, IEEE Journal of Selected Topics in Signal Processing.

[26]  Hongyu Zhao,et al.  Multisample aCGH Data Analysis via Total Variation and Spectral Regularization , 2013, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[27]  Zaïd Harchaoui,et al.  Catching Change-points with Lasso , 2007, NIPS.

[28]  Stephen J. Wright,et al.  Sparse reconstruction by separable approximation , 2009, IEEE Trans. Signal Process..

[29]  Paul Tseng,et al.  A coordinate gradient descent method for nonsmooth separable minimization , 2008, Math. Program..

[30]  Yao-Hua Tan,et al.  Robust group fused lasso for multisample copy number variation detection under uncertainty. , 2016, IET systems biology.