Discrete-Time Neural Network for Fast Solving Large Linear $L_{1}$ Estimation Problems and its Application to Image Restoration

There is growing interest in solving linear L1 estimation problems for sparsity of the solution and robustness against non-Gaussian noise. This paper proposes a discrete-time neural network which can calculate large linear L1 estimation problems fast. The proposed neural network has a fixed computational step length and is proved to be globally convergent to an optimal solution. Then, the proposed neural network is efficiently applied to image restoration. Numerical results show that the proposed neural network is not only efficient in solving degenerate problems resulting from the nonunique solutions of the linear L1 estimation problems but also needs much less computational time than the related algorithms in solving both linear L1 estimation and image restoration problems.

[1]  Stephen B. Furber,et al.  Biologically Inspired Means for Rank-Order Encoding Images: A Quantitative Analysis , 2010, IEEE Transactions on Neural Networks.

[2]  Aggelos K. Katsaggelos,et al.  Iterative regularized least-mean mixed-norm image restoration , 2002 .

[3]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[4]  A. Conn,et al.  Minimization Techniques for Piecewise Differentiable Functions: The l_1 Solution to an Overdetermined Linear System , 1978 .

[5]  Mohamed S. Kamel,et al.  Novel Cooperative Neural Fusion Algorithms for Image Restoration and Image Fusion , 2007, IEEE Transactions on Image Processing.

[6]  Michael K. Ng,et al.  A Fast l1-TV Algorithm for Image Restoration , 2009, SIAM J. Sci. Comput..

[7]  N. T. Hoai-Phuong,et al.  Optimization under Composite Monotonic Constraints and Constrained Optimization over the Efficient Set , 2006 .

[8]  Wu Li,et al.  The Linear l1 Estimator and the Huber M-Estimator , 1998, SIAM J. Optim..

[9]  Harald Haas,et al.  Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication , 2004, Science.

[10]  Abd-Krim Seghouane.  Model Selection Criteria for Image Restoration , 2009, IEEE Transactions on Neural Networks.

[11]  S. Haykin,et al.  Adaptive Filter Theory , 1986 .

[12]  Russell M. Mersereau,et al.  Blur identification by the method of generalized cross-validation , 1992, IEEE Trans. Image Process..

[13]  Richard J. Mammone,et al.  Image restoration by convex projections using adaptive constraints and the L1 norm , 1992, IEEE Trans. Signal Process..

[14]  Elwood T. Olsen,et al.  L1 and L∞ minimization via a variant of Karmarkar's algorithm , 1989, IEEE Trans. Acoust. Speech Signal Process..

[15]  Mark A. Lukas,et al.  An L 1 estimation algorithm with degeneracy and linear constraints , 2002 .

[16]  Danilo P. Mandic,et al.  Recurrent Neural Networks for Prediction: Learning Algorithms, Architectures and Stability , 2001 .

[17]  P. Tseng,et al.  Modified Projection-Type Methods for Monotone Variational Inequalities , 1996 .

[18]  Yide Ma,et al.  A New Automatic Parameter Setting Method of a Simplified PCNN for Image Segmentation , 2011, IEEE Transactions on Neural Networks.

[19]  D. Donoho.  For most large underdetermined systems of equations, the minimal 𝓁1‐norm near‐solution approximates the sparsest near‐solution , 2006 .

[20]  Mohamed S. Kamel,et al.  A Cooperative Recurrent Neural Network for Solving L1 Estimation Problems with General Linear Constraints , 2008, Neural Computation.

[21]  A. Dax.  A row relaxation method for large l1 problems , 1991 .

[22]  Ling Guan,et al.  A neural network adaptive filter for the removal of impulse noise in digital images , 1996, Neural Networks.

[23]  Stephen P. Boyd,et al.  Disciplined Convex Programming , 2006 .

[24]  M. R. Osborne,et al.  A new approach to variable selection in least squares problems , 2000 .

[25]  Youshen Xia,et al.  A Compact Cooperative Recurrent Neural Network for Computing General Constrained $L_1$ Norm Estimators , 2009, IEEE Transactions on Signal Processing.

[26]  Bor-Sen Chen,et al.  System parameter estimation with input/output noisy data and missing measurements , 2000, IEEE Trans. Signal Process..

[27]  Joonki Paik,et al.  Real-time iterative framework of regularized image restoration and its application to video enhancement , 2003, Real Time Imaging.

[28]  Nikolas P. Galatsanos,et al.  Methods for choosing the regularization parameter and estimating the noise variance in image restoration and their relation , 1992, IEEE Trans. Image Process..

[29]  Danilo P. Mandic,et al.  Recurrent Neural Networks for Prediction: Learning Algorithms, Architectures and Stability , 2001 .

[30]  Michael Elad,et al.  From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images , 2009, SIAM Rev..

[31]  D. M. Titterington,et al.  A Study of Methods of Choosing the Smoothing Parameter in Image Restoration by Regularization , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[32]  Yung-Fang Chen,et al.  Reduced-dimension blind space-time 2-D RAKE receivers for DS-CDMA communication systems , 2000, IEEE Trans. Signal Process..

[33]  Mila Nikolova,et al.  Efficient Minimization Methods of Mixed l2-l1 and l1-l1 Norms for Image Restoration , 2005, SIAM J. Sci. Comput..

[34]  Ling Guan,et al.  A network of networks processing model for image regularization , 1997, IEEE Trans. Neural Networks.

[35]  Abd-Krim Seghouane.  A Note on Image Restoration Using $C_{p}$ and MSE , 2008, IEEE Signal Processing Letters.

[36]  Deniz Erdogmus,et al.  An error-entropy minimization algorithm for supervised training of nonlinear adaptive systems , 2002, IEEE Trans. Signal Process..

[37]  James A. Cadzow.  Minimum l1, l2, and l∞ Norm Approximate Solutions to an Overdetermined System of Linear Equations , 2002, Digit. Signal Process..

[38]  R. Unbehauen,et al.  Neural networks for solving systems of linear equations. II. Minimax and least absolute value problems , 1992 .