An Adaptive Markov Random Field for Structured Compressive Sensing

Exploiting intrinsic structures in sparse signals underpin the recent progress in compressive sensing (CS). The key for exploiting such structures is to achieve two desirable properties: generality (i.e., the ability to fit a wide range of signals with diverse structures) and adaptability (i.e., being adaptive to a specific signal). Most existing approaches, however, often only achieve one of these two properties. In this paper, we propose a novel adaptive Markov random field sparsity prior for CS, which not only is able to capture a broad range of sparsity structures, but also can adapt to each sparse signal through refining the parameters of the sparsity prior with respect to the compressed measurements. To maximize the adaptability, we also propose a new sparse signal estimation, where the sparse signals, support, noise, and signal parameter estimation are unified into a variational optimization problem, which can be effectively solved with an alternative minimization scheme. Extensive experiments on three real-world datasets demonstrate the effectiveness of the proposed method in recovery accuracy, noise tolerance, and runtime.

[1]  Terence Sim,et al.  The CMU Pose, Illumination, and Expression Database , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Hong Sun,et al.  Compressive sensing for cluster structured sparse signals: variational Bayes approach , 2016, IET Signal Process..

[3]  Wei Wei,et al.  Exploring Structured Sparsity by a Reweighted Laplace Prior for Hyperspectral Compressive Sensing , 2016, IEEE Transactions on Image Processing.

[4]  Lei Zhang,et al.  Exploiting Clustering Manifold Structure for Hyperspectral Imagery Super-Resolution , 2018, IEEE Transactions on Image Processing.

[5]  M. Yuan,et al.  Model selection and estimation in regression with grouped variables , 2006 .

[6]  Lu Wang,et al.  Sparse Representation-Based ISAR Imaging Using Markov Random Fields , 2015, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[7]  Volkan Cevher,et al.  Sparse Signal Recovery Using Markov Random Fields , 2008, NIPS.

[8]  Aleksandra Pizurica,et al.  Compressed sensing in MRI with a Markov random field prior for spatial clustering of subband coefficients , 2016, 2016 24th European Signal Processing Conference (EUSIPCO).

[9]  Hongbin Li,et al.  Pattern-Coupled Sparse Bayesian Learning for Recovery of Block-Sparse Signals , 2013, IEEE Transactions on Signal Processing.

[10]  Bhaskar D. Rao,et al.  An Empirical Bayesian Strategy for Solving the Simultaneous Sparse Approximation Problem , 2007, IEEE Transactions on Signal Processing.

[11]  S. Godsill,et al.  Bayesian variable selection and regularization for time–frequency surface estimation , 2004 .

[12]  Wei Wei,et al.  Cluster Sparsity Field: An Internal Hyperspectral Imagery Prior for Reconstruction , 2018, International Journal of Computer Vision.

[13]  Jon Atli Benediktsson,et al.  Adaptive Markov Random Fields for Joint Unmixing and Segmentation of Hyperspectral Images , 2013, IEEE Transactions on Image Processing.

[14]  Laurent Daudet,et al.  Soft Bayesian pursuit algorithm for sparse representations , 2011, 2011 IEEE Statistical Signal Processing Workshop (SSP).

[15]  Lawrence Carin,et al.  Tree-Structured Compressive Sensing With Variational Bayesian Analysis , 2010, IEEE Signal Processing Letters.

[16]  Alex Krizhevsky,et al.  Learning Multiple Layers of Features from Tiny Images , 2009 .

[17]  Peng Wang,et al.  Adaptive Importance Learning for Improving Lightweight Image Super-Resolution Network , 2018, International Journal of Computer Vision.

[18]  Yonina C. Eldar,et al.  Reduce and Boost: Recovering Arbitrary Sets of Jointly Sparse Vectors , 2008, IEEE Transactions on Signal Processing.

[19]  George B. Dantzig,et al.  Decomposition Principle for Linear Programs , 1960 .

[20]  Jie Ren,et al.  Context-Aware Sparse Decomposition for Image Denoising and Super-Resolution , 2013, IEEE Transactions on Image Processing.

[21]  Vladimir Kolmogorov,et al.  Convergent Tree-Reweighted Message Passing for Energy Minimization , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Joel A. Tropp,et al.  Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit , 2007, IEEE Transactions on Information Theory.

[23]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[24]  Robert D. Nowak,et al.  Wavelet-based statistical signal processing using hidden Markov models , 1998, IEEE Trans. Signal Process..

[25]  Bruno A. Olshausen,et al.  Learning Horizontal Connections in a Sparse Coding Model of Natural Images , 2007, NIPS.

[26]  J. Besag Statistical Analysis of Non-Lattice Data , 1975 .

[27]  Nikos Komodakis,et al.  Markov Random Field modeling, inference & learning in computer vision & image understanding: A survey , 2013, Comput. Vis. Image Underst..

[28]  D.G. Tzikas,et al.  The variational approximation for Bayesian inference , 2008, IEEE Signal Processing Magazine.

[29]  Emmanuel J. Candès,et al.  A Probabilistic and RIPless Theory of Compressed Sensing , 2010, IEEE Transactions on Information Theory.

[30]  Max Welling,et al.  Learning in Markov Random Fields An Empirical Study , 2005 .

[31]  Richard G. Baraniuk,et al.  Wavelet-domain compressive signal reconstruction using a Hidden Markov Tree model , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[32]  Todd K. Moon,et al.  AMP-B-SBL: An algorithm for clustered sparse signals using approximate message passing , 2016, 2016 IEEE 7th Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON).

[33]  Yonina C. Eldar,et al.  Exploiting Statistical Dependencies in Sparse Representations for Signal Recovery , 2010, IEEE Transactions on Signal Processing.

[34]  Philip Schniter,et al.  Compressive Imaging Using Approximate Message Passing and a Markov-Tree Prior , 2010, IEEE Transactions on Signal Processing.

[35]  Todd K. Moon,et al.  Hierarchical Bayesian approach for jointly-sparse solution of multiple-measurement vectors , 2014, 2014 48th Asilomar Conference on Signals, Systems and Computers.

[36]  Julien Mairal,et al.  Optimization with Sparsity-Inducing Penalties , 2011, Found. Trends Mach. Learn..

[37]  Yonina C. Eldar,et al.  Robust Recovery of Signals From a Structured Union of Subspaces , 2008, IEEE Transactions on Information Theory.

[38]  Bernie Mulgrew,et al.  SPARS'09 - Signal Processing with Adaptive Sparse Structured Representations (2009) , 2009 .

[39]  Piotr Indyk,et al.  A Nearly-Linear Time Framework for Graph-Structured Sparsity , 2015, ICML.

[40]  Junzhou Huang,et al.  Learning with structured sparsity , 2009, ICML '09.

[41]  Michael I. Jordan,et al.  Graphical Models, Exponential Families, and Variational Inference , 2008, Found. Trends Mach. Learn..

[42]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[43]  Minh N. Do,et al.  Tree-Based Orthogonal Matching Pursuit Algorithm for Signal Reconstruction , 2006, 2006 International Conference on Image Processing.

[44]  Wei Wei,et al.  Beyond Low Rank: A Data-Adaptive Tensor Completion Method , 2017, ArXiv.

[45]  J. Besag Spatial Interaction and the Statistical Analysis of Lattice Systems , 1974 .

[46]  Laurent Daudet,et al.  Boltzmann Machine and Mean-Field Approximation for Structured Sparse Decompositions , 2012, IEEE Transactions on Signal Processing.

[47]  Hong Sun,et al.  Bayesian compressive sensing for cluster structured sparse signals , 2012, Signal Process..

[48]  P. Zhao,et al.  The composite absolute penalties family for grouped and hierarchical variable selection , 2009, 0909.0411.

[49]  Nir Friedman,et al.  Probabilistic Graphical Models - Principles and Techniques , 2009 .

[50]  Jun Fang,et al.  Two-Dimensional Pattern-Coupled Sparse Bayesian Learning via Generalized Approximate Message Passing , 2015, IEEE Transactions on Image Processing.

[51]  José M. Bioucas-Dias,et al.  Collaborative sparse regression using spatially correlated supports - Application to hyperspectral unmixing , 2014, IEEE Transactions on Image Processing.

[52]  Ofer Meshi,et al.  An Alternating Direction Method for Dual MAP LP Relaxation , 2011, ECML/PKDD.

[53]  Bhaskar D. Rao,et al.  Latent Variable Bayesian Models for Promoting Sparsity , 2011, IEEE Transactions on Information Theory.

[54]  Hong Sun,et al.  Model based Bayesian compressive sensing via Local Beta Process , 2015, Signal Process..

[55]  Ramazan Ali Sadeghzadeh,et al.  Bayesian compressive sensing using wavelet based Markov random fields , 2017, Signal Process. Image Commun..

[56]  Bhaskar D. Rao,et al.  Recovery of block sparse signals using the framework of block sparse Bayesian learning , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[57]  Geoffrey E. Hinton Training Products of Experts by Minimizing Contrastive Divergence , 2002, Neural Computation.

[58]  Volkan Cevher,et al.  Low-Dimensional Models for Dimensionality Reduction and Signal Recovery: A Geometric Perspective , 2010, Proceedings of the IEEE.

[59]  B. Torrésani,et al.  Structured Sparsity: from Mixed Norms to Structured Shrinkage , 2009 .

[60]  Lawrence Carin,et al.  Exploiting Structure in Wavelet-Based Bayesian Compressive Sensing , 2009, IEEE Transactions on Signal Processing.

[61]  Feng Chen,et al.  Graph-Structured Sparse Optimization for Connected Subgraph Detection , 2016, 2016 IEEE 16th International Conference on Data Mining (ICDM).

[62]  Volkan Cevher,et al.  Sparse Signal Recovery and Acquisition with Graphical Models , 2010, IEEE Signal Processing Magazine.

[63]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[64]  Wei Wei,et al.  Pairwise Matching through Max-Weight Bipartite Belief Propagation , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[65]  Jean-Philippe Vert,et al.  Group lasso with overlap and graph lasso , 2009, ICML '09.

[66]  Nazanin Rahnavard,et al.  Model-Based Nonuniform Compressive Sampling and Recovery of Natural Images Utilizing a Wavelet-Domain Universal Hidden Markov Model , 2017, IEEE Transactions on Signal Processing.

[67]  Piotr Indyk,et al.  Approximation Algorithms for Model-Based Compressive Sensing , 2014, IEEE Transactions on Information Theory.

[68]  Volkan Cevher,et al.  Model-Based Compressive Sensing , 2008, IEEE Transactions on Information Theory.