Multi-Objective Sparse Reconstruction With Transfer Learning and Localized Regularization

Multi-objective sparse reconstruction methods have shown strong potential in sparse reconstruction. However, most methods are computationally expensive due to the requirement of excessive functional evaluations. Most of these methods adopt arbitrary regularization values for iterative thresholding-based local search, which hardly produces high-precision solutions stably. In this article, we propose a multi-objective sparse reconstruction scheme with novel techniques of transfer learning and localized regularization. Firstly, we design a knowledge transfer operator to reuse the search experience from previously solved homogeneous or heterogeneous sparse reconstruction problems, which can significantly accelerate the convergence and improve the reconstruction quality. Secondly, we develop a localized regularization strategy for iterative thresholding-based local search, which uses systematically designed independent regularization values according to decomposed subproblems. The strategy can lead to improved reconstruction accuracy. Therefore, our proposed scheme is more computationally efficient and accurate, compared to existing multi-objective sparse reconstruction methods. This is validated by extensive experiments on simulated signals and benchmark problems.

[1]  Lie Wang,et al.  Orthogonal Matching Pursuit for Sparse Signal Recovery With Noise , 2011, IEEE Transactions on Information Theory.

[2]  Rayan Saab,et al.  Algorithm 890: Sparco: A Testing Framework for Sparse Reconstruction , 2009, TOMS.

[3]  M. Peruggia Experiments with Mixtures: Designs, Models, and the Analysis of Mixture Data , 2003 .

[4]  Bernhard Schölkopf,et al.  A Kernel Method for the Two-Sample-Problem , 2006, NIPS.

[5]  Qingfu Zhang,et al.  A Preference-Based Multiobjective Evolutionary Approach for Sparse Optimization , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[6]  Guang Li,et al.  A Knee Point Driven Particle Swarm Optimization Algorithm for Sparse Reconstruction , 2017, SEAL.

[7]  Yew-Soon Ong,et al.  Curbing Negative Influences Online for Seamless Transfer Evolutionary Optimization , 2019, IEEE Transactions on Cybernetics.

[8]  Min Bai,et al.  Obtaining free USArray data by multi-dimensional seismic reconstruction , 2019, Nature Communications.

[9]  Lei Zhou,et al.  Evolutionary Multitasking via Explicit Autoencoding , 2019, IEEE Transactions on Cybernetics.

[10]  Hisao Ishibuchi,et al.  Performance of Decomposition-Based Many-Objective Algorithms Strongly Depends on Pareto Front Shapes , 2017, IEEE Transactions on Evolutionary Computation.

[11]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[12]  Chi-Keong Goh,et al.  Multiproblem Surrogates: Transfer Evolutionary Multiobjective Optimization of Computationally Expensive Problems , 2019, IEEE Transactions on Evolutionary Computation.

[13]  J. Cornell Experiments with Mixtures: Designs, Models and the Analysis of Mixture Data , 1982 .

[14]  Fang Liu,et al.  Nonconvex Compressed Sensing by Nature-Inspired Optimization Algorithms , 2015, IEEE Transactions on Cybernetics.

[15]  Stephen J. Wright,et al.  Sparse Reconstruction by Separable Approximation , 2008, IEEE Transactions on Signal Processing.

[16]  Kilian Q. Weinberger,et al.  Marginalized Denoising Autoencoders for Domain Adaptation , 2012, ICML.

[17]  Liang Feng,et al.  Insights on Transfer Optimization: Because Experience is the Best Teacher , 2018, IEEE Transactions on Emerging Topics in Computational Intelligence.

[18]  Lei Wu,et al.  A Barzilai–Borwein-Like Iterative Half Thresholding Algorithm for the $$L_{1/2}$$L1/2 Regularized Problem , 2016, J. Sci. Comput..

[19]  Qi Zhao,et al.  Adaptive decomposition-based evolutionary approach for multiobjective sparse reconstruction , 2018, Inf. Sci..

[20]  J. Borwein,et al.  Two-Point Step Size Gradient Methods , 1988 .

[21]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[22]  Tolga Çukur,et al.  Signal Compensation and Compressed Sensing for Magnetization-Prepared MR Angiography , 2011, IEEE Transactions on Medical Imaging.

[23]  Pengfei Wei,et al.  Deep Nonlinear Feature Coding for Unsupervised Domain Adaptation , 2016, IJCAI.

[24]  John E. Dennis,et al.  Normal-Boundary Intersection: A New Method for Generating the Pareto Surface in Nonlinear Multicriteria Optimization Problems , 1998, SIAM J. Optim..

[25]  Qingfu Zhang,et al.  A multi-phase multiobjective approach based on decomposition for sparse reconstruction , 2016, 2016 IEEE Congress on Evolutionary Computation (CEC).

[26]  Kalyanmoy Deb,et al.  A combined genetic adaptive search (GeneAS) for engineering design , 1996 .

[27]  Jing J. Liang,et al.  A novel multiobjective optimization algorithm for sparse signal reconstruction , 2020, Signal Process..

[28]  Bing Xue,et al.  Cross-Domain Reuse of Extracted Knowledge in Genetic Programming for Image Classification , 2017, IEEE Transactions on Evolutionary Computation.

[29]  D. Donoho,et al.  Sparse MRI: The application of compressed sensing for rapid MR imaging , 2007, Magnetic resonance in medicine.

[30]  Ivor W. Tsang,et al.  Memetic Search With Interdomain Learning: A Realization Between CVRP and CARP , 2015, IEEE Transactions on Evolutionary Computation.

[31]  Sushil J. Louis,et al.  Learning with case-injected genetic algorithms , 2004, IEEE Transactions on Evolutionary Computation.

[32]  Yonina C. Eldar,et al.  The Projected GSURE for Automatic Parameter Tuning in Iterative Shrinkage Methods , 2010, Applied and Computational Harmonic Analysis.

[33]  Qingfu Zhang,et al.  A Two-Phase Evolutionary Approach for Compressive Sensing Reconstruction , 2017, IEEE Transactions on Cybernetics.

[34]  Kalyanmoy Deb,et al.  An Evolutionary Many-Objective Optimization Algorithm Using Reference-Point-Based Nondominated Sorting Approach, Part I: Solving Problems With Box Constraints , 2014, IEEE Transactions on Evolutionary Computation.

[35]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[36]  Qi Zhao,et al.  A hybrid evolutionary algorithm for multiobjective sparse reconstruction , 2017, Signal, Image and Video Processing.

[37]  Yangkang Chen,et al.  Hybrid rank-sparsity constraint model for simultaneous reconstruction and denoising of 3D seismic data , 2017 .

[38]  Pascal Vincent,et al.  Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion , 2010, J. Mach. Learn. Res..

[39]  Stephen J. Wright,et al.  Sparse reconstruction by separable approximation , 2009, IEEE Trans. Signal Process..

[40]  Xin Yao,et al.  An Evolutionary Multiobjective Approach to Sparse Reconstruction , 2014, IEEE Transactions on Evolutionary Computation.

[41]  M. Lustig,et al.  Improving non‐contrast‐enhanced steady‐state free precession angiography with compressed sensing , 2009, Magnetic resonance in medicine.

[42]  Gary G. Yen,et al.  Transfer Learning-Based Dynamic Multiobjective Optimization Algorithms , 2016, IEEE Transactions on Evolutionary Computation.

[43]  Eckart Zitzler,et al.  HypE: An Algorithm for Fast Hypervolume-Based Many-Objective Optimization , 2011, Evolutionary Computation.

[44]  Patrick L. Combettes,et al.  Signal Recovery by Proximal Forward-Backward Splitting , 2005, Multiscale Model. Simul..

[45]  Georgios C. Anagnostopoulos,et al.  A Simple Method for Solving the SVM Regularization Path for Semidefinite Kernels , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[46]  Xiaohua Jia,et al.  Transmission-Efficient Clustering Method for Wireless Sensor Networks Using Compressive Sensing , 2014, IEEE Transactions on Parallel and Distributed Systems.

[47]  Kalyanmoy Deb,et al.  Simulated Binary Crossover for Continuous Search Space , 1995, Complex Syst..

[48]  Ingo Mierswa,et al.  Information preserving multi-objective feature selection for unsupervised learning , 2006, GECCO.

[49]  Zongben Xu,et al.  $L_{1/2}$ Regularization: A Thresholding Representation Theory and a Fast Solver , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[50]  Tapabrata Ray,et al.  Adaptive Sorting-Based Evolutionary Algorithm for Many-Objective Optimization , 2019, IEEE Transactions on Evolutionary Computation.

[51]  Ivor W. Tsang,et al.  Domain Adaptation via Transfer Component Analysis , 2009, IEEE Transactions on Neural Networks.

[52]  Qingfu Zhang,et al.  MOEA/D: A Multiobjective Evolutionary Algorithm Based on Decomposition , 2007, IEEE Transactions on Evolutionary Computation.

[53]  A. Enis Çetin,et al.  Projection onto Epigraph Sets for Rapid Self-Tuning Compressed Sensing MRI , 2018, IEEE Transactions on Medical Imaging.

[54]  K. Deb,et al.  A Review of Nadir Point Estimation Procedures Using Evolutionary Approaches: A Tale of Dimensionality Reduction , 2008 .

[55]  Bernhard Sendhoff,et al.  A Reference Vector Guided Evolutionary Algorithm for Many-Objective Optimization , 2016, IEEE Transactions on Evolutionary Computation.

[56]  E.J. Candes,et al.  An Introduction To Compressive Sampling , 2008, IEEE Signal Processing Magazine.

[57]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[58]  Liang Feng,et al.  Autoencoding Evolutionary Search With Learning Across Heterogeneous Problems , 2017, IEEE Transactions on Evolutionary Computation.

[59]  Mike E. Davies,et al.  Iterative Hard Thresholding for Compressed Sensing , 2008, ArXiv.

[60]  Bangti Jin,et al.  Iterative Soft/Hard Thresholding With Homotopy Continuation for Sparse Recovery , 2017, IEEE Signal Processing Letters.