Separable Dictionary Learning with Global Optimality and Applications to Diffusion MRI

Dictionary learning is a popular class of methods for modeling complex data by learning sparse representations directly from the data. For some large-scale applications, exploiting a known structure of the signal is often essential for reducing the complexity of algorithms and representations. One such method is tensor factorization by which a large multi-dimensional dataset can be explicitly factored or separated along each dimension of the data in order to break the representation up into smaller components. Learning dictionaries for tensor structured data is called tensor or separable dictionary learning. While there have been many recent works on separable dictionary learning, typical formulations involve solving a non-convex optimization problem and guaranteeing global optimality remains a challenge. In this work, we propose a framework that uses recent developments in matrix/tensor factorization to provide theoretical and numerical guarantees of the global optimality for the separable dictionary learning problem. We will demonstrate our algorithm on diffusion magnetic resonance imaging (dMRI) data, a medical imaging modality which measures water diffusion along multiple angular directions in every voxel of an MRI volume. For this application, state-of-the-art methods learn dictionaries for the angular domain of the signals without consideration for the spatial domain. In this work, we apply the proposed separable dictionary learning method to learn spatial and angular dMRI dictionaries jointly and show results on denoising phantom and real dMRI brain data.

[1]  Yigang Cen,et al.  Analytic separable dictionary learning based on oblique manifold , 2017, Neurocomputing.

[2]  Michael Elad,et al.  Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries , 2006, IEEE Transactions on Image Processing.

[3]  Joseph F. Murray,et al.  Dictionary Learning Algorithms for Sparse Representation , 2003, Neural Computation.

[4]  Syed Zubair,et al.  Tensor dictionary learning with sparse TUCKER decomposition , 2013, 2013 18th International Conference on Digital Signal Processing (DSP).

[5]  Julien Cohen-Adad,et al.  Accelerated diffusion spectrum imaging with compressed sensing using adaptive dictionaries , 2012, MICCAI.

[6]  Joos Vandewalle,et al.  A Multilinear Singular Value Decomposition , 2000, SIAM J. Matrix Anal. Appl..

[7]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[8]  René Vidal,et al.  Blood cell detection and counting in holographic lens-free imaging by convolutional sparse dictionary learning and coding , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).

[9]  Jean Ponce,et al.  Convex Sparse Matrix Factorizations , 2008, ArXiv.

[10]  Stephen P. Boyd,et al.  Proximal Algorithms , 2013, Found. Trends Optim..

[11]  Suyash P. Awate,et al.  Regularized Dictionary Learning with Robust Sparsity Fitting for Compressed Sensing Multishell HARDI , 2016, MICCAI 2016.

[12]  Dinggang Shen,et al.  Tensorial Spherical Polar Fourier Diffusion MRI with Optimal Dictionary Learning , 2015, MICCAI.

[13]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[14]  René Vidal,et al.  ($k, q$)-Compressed Sensing for dMRI with Joint Spatial-Angular Sparsity Prior , 2017, ArXiv.

[15]  V. Wedeen,et al.  Diffusion MRI of Complex Neural Architecture , 2003, Neuron.

[16]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[17]  Wenxing Ye,et al.  An over-complete dictionary based regularized reconstruction of a field of ensemble average propagators , 2012, 2012 9th IEEE International Symposium on Biomedical Imaging (ISBI).

[18]  D. Donoho,et al.  Sparse MRI: The application of compressed sensing for rapid MR imaging , 2007, Magnetic resonance in medicine.

[19]  Wenxing Ye,et al.  Dictionary Learning on the Manifold of Square Root Densities and Application to Reconstruction of Diffusion Propagator Fields , 2013, IPMI.

[20]  Yen-Wei Chen,et al.  K-CPD: Learning of overcomplete dictionaries for tensor sparse coding , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[21]  Maxime Descoteaux,et al.  Non Local Spatial and Angular Matching : Enabling higher spatial resolution diffusion MRI datasets through adaptive denoising , 2016, Medical Image Anal..

[22]  Stefanos Zafeiriou,et al.  Robust Kronecker-Decomposable Component Analysis for Low-Rank Modeling , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[23]  Florian Roemer,et al.  Tensor-based algorithms for learning multidimensional separable dictionaries , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[24]  Suyash P. Awate,et al.  Bayesian Dictionary Learning and Undersampled Multishell HARDI Reconstruction , 2017, IPMI.

[25]  Rachid Deriche,et al.  A computational diffusion MRI and parametric dictionary learning framework for modeling the diffusion signal and its features , 2013, Medical Image Anal..

[26]  René Vidal,et al.  Structured Low-Rank Matrix Factorization: Optimality, Algorithm, and Applications to Image Processing , 2014, ICML.

[27]  Yogesh Rathi,et al.  On approximation of orientation distributions by means of spherical ridgelets , 2008, ISBI.

[28]  Martin Kleinsteuber,et al.  Separable Dictionary Learning , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  René Vidal,et al.  Spatial-Angular Sparse Coding for HARDI , 2016, MICCAI.

[30]  Yogesh Rathi,et al.  Fast and Accurate Reconstruction of HARDI Data Using Compressed Sensing , 2010, MICCAI.

[31]  Maxime Descoteaux,et al.  Denoising and fast diffusion imaging with physically constrained sparse dictionary learning , 2014, Medical Image Anal..

[32]  Xiaoyan Sun,et al.  TenSR: Multi-dimensional Tensor Sparse Representation , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2009, Found. Comput. Math..

[34]  Lawrence Carin,et al.  Tensor-Dictionary Learning with Deep Kruskal-Factor Analysis , 2016, AISTATS.

[35]  Rachid Deriche,et al.  Regularized Spherical Polar Fourier Diffusion MRI with Optimal Dictionary Learning , 2013, MICCAI.

[36]  Christophe Lenglet,et al.  A nonparametric Riemannian framework for processing high angular resolution diffusion images and its applications to ODF-based morphometry , 2011, NeuroImage.

[37]  A. Bruckstein,et al.  K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation , 2005 .

[38]  René Vidal,et al.  Structured Low-Rank Matrix Factorization: Global Optimality, Algorithms, and Applications , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[39]  Yuemin Zhu,et al.  Sparse representation based MRI denoising with total variation , 2008, 2008 9th International Conference on Signal Processing.

[40]  René Vidal,et al.  Global Optimality in Tensor Factorization, Deep Learning, and Beyond , 2015, ArXiv.

[41]  G. Sapiro,et al.  Reconstruction of the orientation distribution function in single‐ and multiple‐shell q‐ball imaging within constant solid angle , 2010, Magnetic resonance in medicine.

[42]  Carl-Fredrik Westin,et al.  Probabilistic ODF Estimation from Reduced HARDI Data with Sparse Regularization , 2011, MICCAI.

[43]  Kjersti Engan,et al.  Method of optimal directions for frame design , 1999, 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258).

[44]  Christophe Lenglet,et al.  Estimation of white matter fiber parameters from compressed multiresolution diffusion MRI using sparse Bayesian learning , 2017, NeuroImage.

[45]  René Vidal,et al.  Joint spatial‐angular sparse coding for dMRI with separable dictionaries , 2018, Medical Image Anal..

[46]  Yogesh Rathi,et al.  Spatially Regularized Compressed Sensing for High Angular Resolution Diffusion Imaging , 2011, IEEE Transactions on Medical Imaging.

[47]  Anand D. Sarwate,et al.  STARK: Structured dictionary learning through rank-one tensor recovery , 2017, 2017 IEEE 7th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP).

[48]  Rachid Deriche,et al.  Parametric Dictionary Learning for Modeling EAP and ODF in Diffusion MRI , 2012, MICCAI.

[49]  Yonina C. Eldar,et al.  Compressed Sensing with Coherent and Redundant Dictionaries , 2010, ArXiv.

[50]  Yigang Cen,et al.  Improved separable dictionary learning , 2016, 2016 IEEE 13th International Conference on Signal Processing (ICSP).

[51]  Mariano Rivera,et al.  Sparse and Adaptive Diffusion Dictionary (SADD) for recovering intra-voxel white matter structure , 2015, Medical Image Anal..

[52]  Cássio Fraga Dantas,et al.  Learning Dictionaries as a Sum of Kronecker Products , 2017, IEEE Signal Processing Letters.

[53]  Gaël Varoquaux,et al.  Learning brain regions via large-scale online structured sparse dictionary learning , 2016, NIPS.

[54]  Guillermo Sapiro,et al.  Online Learning for Matrix Factorization and Sparse Coding , 2009, J. Mach. Learn. Res..

[55]  Vicente Grau,et al.  Prospective acceleration of diffusion tensor imaging with compressed sensing using adaptive dictionaries , 2015, Magnetic resonance in medicine.

[56]  Rachid Deriche,et al.  Deterministic and Probabilistic Tractography Based on Complex Fibre Orientation Distributions , 2009, IEEE Transactions on Medical Imaging.