Sparse methods for biomedical data

Following recent technological revolutions, the investigation of massive biomedical data with growing scale, diversity, and complexity has taken a center stage in modern data analysis. Although complex, the underlying representations of many biomedical data are often sparse. For example, for a certain disease such as leukemia, even though humans have tens of thousands of genes, only a few genes are relevant to the disease; a gene network is sparse since a regulatory pathway involves only a small number of genes; many biomedical signals are sparse or compressible in the sense that they have concise representations when expressed in a proper basis. Therefore, finding sparse representations is fundamentally important for scientific discovery. Sparse methods based on the '1 norm have attracted a great amount of research efforts in the past decade due to its sparsity-inducing property, convenient convexity, and strong theoretical guarantees. They have achieved great success in various applications such as biomarker selection, biological network construction, and magnetic resonance imaging. In this paper, we review state-of-the-art sparse methods and their applications to biomedical data.

[1]  Peter Kellman,et al.  Real‐time accelerated interactive MRI with adaptive TSENSE and UNFOLD , 2003, Magnetic resonance in medicine.

[2]  M. Wainwright,et al.  Joint support recovery under high-dimensional scaling: Benefits and perils of ℓ 1,∞ -regularization , 2008, NIPS 2008.

[3]  Tom Goldstein,et al.  The Split Bregman Method for L1-Regularized Problems , 2009, SIAM J. Imaging Sci..

[4]  Yurii Nesterov,et al.  Introductory Lectures on Convex Optimization - A Basic Course , 2014, Applied Optimization.

[5]  L. Ying,et al.  Accelerating SENSE using compressed sensing , 2009, Magnetic resonance in medicine.

[6]  A. C. Brau,et al.  ESPIRiT ( Efficient Eigenvector-Based L 1 SPIRiT ) for Compressed Sensing Parallel Imaging-Theoretical Interpretation and Improved Robustness for Overlapped FOV Prescription , 2010 .

[7]  Joel A. Tropp,et al.  Algorithms for simultaneous sparse approximation. Part I: Greedy pursuit , 2006, Signal Process..

[8]  Y. Nesterov Gradient methods for minimizing composite objective function , 2007 .

[9]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[10]  Jianqing Fan,et al.  A Selective Overview of Variable Selection in High Dimensional Feature Space. , 2009, Statistica Sinica.

[11]  M. Kowalski Sparse regression using mixed norms , 2009 .

[12]  Xiaotong Shen,et al.  Simultaneous Grouping Pursuit and Feature Selection Over an Undirected Graph , 2013, Journal of the American Statistical Association.

[13]  Le Song,et al.  Estimating time-varying networks , 2008, ISMB 2008.

[14]  Stephen P. Boyd,et al.  An Interior-Point Method for Large-Scale l1-Regularized Logistic Regression , 2007, J. Mach. Learn. Res..

[15]  Alexandre d'Aspremont,et al.  Model Selection Through Sparse Max Likelihood Estimation Model Selection Through Sparse Maximum Likelihood Estimation for Multivariate Gaussian or Binary Data , 2022 .

[16]  J. Friedman,et al.  New Insights and Faster Computations for the Graphical Lasso , 2011 .

[17]  Peter Kellman Parallel Imaging : The Basics , 2002 .

[18]  Wei Chu,et al.  Biomarker discovery in microarray gene expression data with Gaussian processes , 2005, Bioinform..

[19]  Jieping Ye,et al.  Multi-Task Feature Learning Via Efficient l2, 1-Norm Minimization , 2009, UAI.

[20]  Edgar Mueller,et al.  Dynamic cardiac MRI reconstruction with weighted redundant Haar wavelets , 2022 .

[21]  José M. Bioucas-Dias,et al.  An Augmented Lagrangian Approach to the Constrained Optimization Formulation of Imaging Inverse Problems , 2009, IEEE Transactions on Image Processing.

[22]  J. Mesirov,et al.  Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. , 1999, Science.

[23]  Massimiliano Pontil,et al.  Convex multi-task feature learning , 2008, Machine Learning.

[24]  Xiaotong Shen,et al.  Journal of the American Statistical Association Likelihood-based Selection and Sharp Parameter Estimation Likelihood-based Selection and Sharp Parameter Estimation , 2022 .

[25]  Francis R. Bach,et al.  Structured Variable Selection with Sparsity-Inducing Norms , 2009, J. Mach. Learn. Res..

[26]  H. Bondell,et al.  Simultaneous Regression Shrinkage, Variable Selection, and Supervised Clustering of Predictors with OSCAR , 2008, Biometrics.

[27]  S. Panchanathan,et al.  BEST: a novel computational approach for comparing gene expression patterns from early stages of Drosophila melanogaster development. , 2002, Genetics.

[28]  P. Brown,et al.  Drug target validation and identification of secondary drug target effects using DNA microarrays , 1998, Nature Medicine.

[29]  Robin M Heidemann,et al.  SMASH, SENSE, PILS, GRAPPA: How to Choose the Optimal Method , 2004, Topics in magnetic resonance imaging : TMRI.

[30]  R. Tibshirani,et al.  Sparse inverse covariance estimation with the graphical lasso. , 2008, Biostatistics.

[31]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[32]  Trevor J. Hastie,et al.  The Graphical Lasso: New Insights and Alternatives , 2011, Electronic journal of statistics.

[33]  E.J. Candes,et al.  An Introduction To Compressive Sampling , 2008, IEEE Signal Processing Magazine.

[34]  G. Obozinski Joint covariate selection for grouped classification , 2007 .

[35]  Tso-Jung Yen,et al.  Discussion on "Stability Selection" by Meinshausen and Buhlmann , 2010 .

[36]  Leslie Greengard,et al.  Accelerating the Nonuniform Fast Fourier Transform , 2004, SIAM Rev..

[37]  M. Lustig,et al.  SPIRiT: Iterative self‐consistent parallel imaging reconstruction from arbitrary k‐space , 2010, Magnetic resonance in medicine.

[38]  Cun-Hui Zhang,et al.  Confidence Intervals for Low-Dimensional Parameters With High-Dimensional Data , 2011 .

[39]  Richard G. Baraniuk,et al.  Compressive Sensing , 2008, Computer Vision, A Reference Guide.

[40]  Jun Liu,et al.  Regularized reconstruction using redundant Haar wavelets : A means to achieve high under-sampling factors in non-contrast-enhanced 4 D MRA , .

[41]  R. Tibshirani,et al.  A note on the group lasso and a sparse group lasso , 2010, 1001.0736.

[42]  Fan Chung,et al.  Spectral Graph Theory , 1996 .

[43]  R. Tibshirani,et al.  �-norm Support Vector Machines , 2003 .

[44]  Peter Bühlmann,et al.  Missing values: sparse inverse covariance estimation and an extension to sparse regression , 2009, Statistics and Computing.

[45]  Julien Mairal,et al.  Proximal Methods for Sparse Hierarchical Dictionary Learning , 2010, ICML.

[46]  Robin M Heidemann,et al.  Generalized autocalibrating partially parallel acquisitions (GRAPPA) , 2002, Magnetic resonance in medicine.

[47]  Eric P. Xing,et al.  Tree-Guided Group Lasso for Multi-Task Regression with Structured Sparsity , 2009, ICML.

[48]  H. Zou,et al.  Structured variable selection and estimation , 2009, 1011.0610.

[49]  T. Ideker,et al.  Network-based classification of breast cancer metastasis , 2007, Molecular systems biology.

[50]  R. Tibshirani,et al.  PATHWISE COORDINATE OPTIMIZATION , 2007, 0708.1485.

[51]  Jason Weston,et al.  Gene Selection for Cancer Classification using Support Vector Machines , 2002, Machine Learning.

[52]  Christof Baltes,et al.  k‐t BLAST reconstruction from non‐Cartesian k‐t space sampling , 2006, Magnetic resonance in medicine.

[53]  Scott E. Fraser,et al.  Imaging in Systems Biology , 2007, Cell.

[54]  Trevor Darrell,et al.  An efficient projection for l1, ∞ regularization , 2009, ICML '09.

[55]  Jun Liu,et al.  Efficient `1=`q Norm Regularization , 2010 .

[56]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .

[57]  Kaustubh Supekar,et al.  Sparse logistic regression for whole-brain classification of fMRI data , 2010, NeuroImage.

[58]  W. Manning,et al.  Simultaneous acquisition of spatial harmonics (SMASH): Fast imaging with radiofrequency coil arrays , 1997, Magnetic resonance in medicine.

[59]  T. Hromádka,et al.  Sparse Representation of Sounds in the Unanesthetized Auditory Cortex , 2008, PLoS biology.

[60]  Kedar Khare,et al.  Accelerated MR imaging using compressive sensing with no free parameters , 2012, Magnetic resonance in medicine.

[61]  Peter Buhlmann Statistical significance in high-dimensional linear models , 2012, 1202.1377.

[62]  W. McGinnis,et al.  From DNA to Diversity, Molecular Genetics and the Evolution of Animal Design, 2nd edition , 2005 .

[63]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[64]  M. Ashburner,et al.  Systematic determination of patterns of gene expression during Drosophila embryogenesis , 2002, Genome Biology.

[65]  Larry A. Wasserman,et al.  Stability Approach to Regularization Selection (StARS) for High Dimensional Graphical Models , 2010, NIPS.

[66]  Pablo Tamayo,et al.  Gene set enrichment analysis: A knowledge-based approach for interpreting genome-wide expression profiles , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[67]  Jieping Ye,et al.  Moreau-Yosida Regularization for Grouped Tree Structure Learning , 2010, NIPS.

[68]  N. Gostling,et al.  From DNA to Diversity: Molecular Genetics and the Evolution of Animal Design , 2002, Heredity.

[69]  Yu-Chung N. Cheng,et al.  Magnetic Resonance Imaging: Physical Principles and Sequence Design , 1999 .

[70]  Ji Zhu,et al.  Regularized Multivariate Regression for Identifying Master Predictors with Application to Integrative Genomics Study of Breast Cancer. , 2008, The annals of applied statistics.

[71]  Eric P. Xing,et al.  On Time Varying Undirected Graphs , 2011, AISTATS.

[72]  Jeffrey A. Fessler,et al.  Parallel MR Image Reconstruction Using Augmented Lagrangian Methods , 2011, IEEE Transactions on Medical Imaging.

[73]  M Usman,et al.  k‐t group sparse: A method for accelerating dynamic MRI , 2011, Magnetic resonance in medicine.

[74]  Emmanuel J. Candès,et al.  Quantitative Robust Uncertainty Principles and Optimally Sparse Decompositions , 2004, Found. Comput. Math..

[75]  M. Elad,et al.  An EigenVector Approach to AutoCalibrating Parallel MRI , Where SENSE Meets GRAPPA , 2010 .

[76]  R. Tibshirani,et al.  Sparse Principal Component Analysis , 2006 .

[77]  Jieping Ye,et al.  Feature grouping and selection over an undirected graph , 2012, KDD.

[78]  T. Hastie,et al.  Classification of gene microarrays by penalized logistic regression. , 2004, Biostatistics.

[79]  P. Bühlmann,et al.  The group lasso for logistic regression , 2008 .

[80]  Tong Zhang,et al.  Analysis of Multi-stage Convex Relaxation for Sparse Regularization , 2010, J. Mach. Learn. Res..

[81]  D. Donoho,et al.  Sparse MRI: The application of compressed sensing for rapid MR imaging , 2007, Magnetic resonance in medicine.

[82]  Junzhou Huang,et al.  Learning with structured sparsity , 2009, ICML '09.

[83]  R.G. Baraniuk,et al.  Compressive Sensing [Lecture Notes] , 2007, IEEE Signal Processing Magazine.

[84]  Wotao Yin,et al.  A Fast Hybrid Algorithm for Large-Scale l1-Regularized Logistic Regression , 2010, J. Mach. Learn. Res..

[85]  Hongzhe Li,et al.  In Response to Comment on "Network-constrained regularization and variable selection for analysis of genomic data" , 2008, Bioinform..

[86]  Patrick Danaher,et al.  The joint graphical lasso for inverse covariance estimation across multiple classes , 2011, Journal of the Royal Statistical Society. Series B, Statistical methodology.

[87]  Jing Li,et al.  Learning Brain Connectivity of Alzheimer's Disease from Neuroimaging Data , 2009, NIPS.

[88]  Grace Wahba,et al.  Detecting disease-causing genes by LASSO-Patternsearch algorithm , 2007, BMC proceedings.

[89]  Jieping Ye,et al.  Learning Sparse Representations for Fruit-Fly Gene Expression Pattern Image Annotation and Retrieval , 2012, BMC Bioinformatics.

[90]  F H Epstein,et al.  Adaptive sensitivity encoding incorporating temporal filtering (TSENSE) † , 2001, Magnetic resonance in medicine.

[91]  D. Larkman,et al.  Parallel magnetic resonance imaging , 2007, Physics in medicine and biology.

[92]  Trevor J. Hastie,et al.  Genome-wide association analysis by lasso penalized logistic regression , 2009, Bioinform..

[93]  Han Liu,et al.  Blockwise coordinate descent procedures for the multi-task lasso, with applications to neural semantic basis discovery , 2009, ICML '09.

[94]  Michael I. Jordan,et al.  A Direct Formulation for Sparse Pca Using Semidefinite Programming , 2004, NIPS 2004.

[95]  Jean-Philippe Vert,et al.  Group lasso with overlap and graph lasso , 2009, ICML '09.

[96]  Trevor J. Hastie,et al.  Exact Covariance Thresholding into Connected Components for Large-Scale Graphical Lasso , 2011, J. Mach. Learn. Res..

[97]  P. Zhao,et al.  The composite absolute penalties family for grouped and hierarchical variable selection , 2009, 0909.0411.

[98]  R. Tibshirani,et al.  Spatial smoothing and hot spot detection for CGH data using the fused lasso. , 2008, Biostatistics.

[99]  A. Ng Feature selection, L1 vs. L2 regularization, and rotational invariance , 2004, Twenty-first international conference on Machine learning - ICML '04.

[100]  Richard G. Baraniuk,et al.  Sparse Signal Detection from Incoherent Projections , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[101]  Robert D. Nowak,et al.  Compressive wireless sensing , 2006, 2006 5th International Conference on Information Processing in Sensor Networks.

[102]  Gregory Piatetsky-Shapiro,et al.  High-Dimensional Data Analysis: The Curses and Blessings of Dimensionality , 2000 .

[103]  N. Meinshausen,et al.  Stability selection , 2008, 0809.2932.

[104]  Robert Tibshirani,et al.  1-norm Support Vector Machines , 2003, NIPS.

[105]  Yunmei Chen,et al.  Computational Acceleration for MR Image Reconstruction in Partially Parallel Imaging , 2011, IEEE Transactions on Medical Imaging.

[106]  Leon Wenliang Zhong,et al.  Efficient Sparse Modeling With Automatic Feature Grouping , 2011, IEEE Transactions on Neural Networks and Learning Systems.

[107]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[108]  Hanchuan Peng,et al.  Bioimage informatics: a new area of engineering biology , 2008, Bioinform..

[109]  Jieping Ye,et al.  Efficient Sparse Group Feature Selection via Nonconvex Optimization , 2012, ICML.

[110]  E. Xing,et al.  Statistical Estimation of Correlated Genome Associations to a Quantitative Trait Network , 2009, PLoS genetics.

[111]  P. Boesiger,et al.  SENSE: Sensitivity encoding for fast MRI , 1999, Magnetic resonance in medicine.

[112]  Shuiwang Ji,et al.  SLEP: Sparse Learning with Efficient Projections , 2011 .

[113]  E. Levina,et al.  Joint estimation of multiple graphical models. , 2011, Biometrika.

[114]  M. Yuan,et al.  Model selection and estimation in regression with grouped variables , 2006 .

[115]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.

[116]  T. Hohage,et al.  Image reconstruction by regularized nonlinear inversion—Joint estimation of coil sensitivities and image content , 2008, Magnetic resonance in medicine.

[117]  Francis R. Bach,et al.  Consistency of the group Lasso and multiple kernel learning , 2007, J. Mach. Learn. Res..

[118]  Leslie Ying,et al.  Joint image reconstruction and sensitivity estimation in SENSE (JSENSE) , 2007, Magnetic resonance in medicine.

[119]  Arkadi Nemirovski,et al.  EFFICIENT METHODS IN CONVEX PROGRAMMING , 2007 .

[120]  Trevor Darrell,et al.  An efficient projection for l 1 , infinity regularization. , 2009, ICML 2009.

[121]  Po-Ling Loh,et al.  High-dimensional regression with noisy and missing data: Provable guarantees with non-convexity , 2011, NIPS.

[122]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[123]  Paul M. Thompson,et al.  Multi-source feature learning for joint analysis of incomplete multiple heterogeneous neuroimaging data , 2012, NeuroImage.

[124]  Jianmin Wang,et al.  Field‐of‐view limitations in parallel imaging , 2004, Magnetic resonance in medicine.

[125]  D.,et al.  Regression Models and Life-Tables , 2022 .

[126]  R. Tibshirani,et al.  Sparsity and smoothness via the fused lasso , 2005 .

[127]  Jieping Ye,et al.  Sparse learning and stability selection for predicting MCI to AD conversion using baseline ADNI data , 2012, BMC Neurology.