Sparse Multicategory Generalized Distance Weighted Discrimination in Ultra-High Dimensions

Distance weighted discrimination (DWD) is an appealing classification method that is capable of overcoming data piling problems in high-dimensional settings. Especially when various sparsity structures are assumed in these settings, variable selection in multicategory classification poses great challenges. In this paper, we propose a multicategory generalized DWD (MgDWD) method that maintains intrinsic variable group structures during selection using a sparse group lasso penalty. Theoretically, we derive minimizer uniqueness for the penalized MgDWD loss function and consistency properties for the proposed classifier. We further develop an efficient algorithm based on the proximal operator to solve the optimization problem. The performance of MgDWD is evaluated using finite sample simulations and miRNA data from an HIV study.

[1]  Runze Li,et al.  Variable selection for support vector machines in moderately high dimensions , 2016, Journal of the Royal Statistical Society. Series B, Statistical methodology.

[2]  Xihong Lin,et al.  Some considerations of classification for high dimension low-sample size data , 2013, Statistical methods in medical research.

[3]  Sham M. Kakade,et al.  A tail inequality for quadratic forms of subgaussian random vectors , 2011, ArXiv.

[4]  John H. L. Hansen,et al.  Speaker Recognition by Machines and Humans: A tutorial review , 2015, IEEE Signal Processing Magazine.

[5]  Xin Wang,et al.  Multiclass Probability Estimation With Support Vector Machines , 2019, Journal of Computational and Graphical Statistics.

[6]  Robert M. Haralick,et al.  Textural Features for Image Classification , 1973, IEEE Trans. Syst. Man Cybern..

[7]  Yanhua Wang,et al.  Regularized quantile regression under heterogeneous sparsity with application to quantitative genetic traits , 2015, Comput. Stat. Data Anal..

[8]  Linglong Kong,et al.  HIV-associated sensory polyneuropathy and neuronal injury are associated with miRNA-455-3p induction. , 2018, JCI insight.

[9]  Kim-Chuan Toh,et al.  Fast Algorithms for Large-Scale Generalized Distance Weighted Discrimination , 2016, 1604.05473.

[10]  Michael J. Todd,et al.  Multiclass Distance-Weighted Discrimination , 2013 .

[11]  Hao Helen Zhang,et al.  Hard or Soft Classification? Large-Margin Unified Machines , 2011, Journal of the American Statistical Association.

[12]  Hui Zou,et al.  Sparse Distance Weighted Discrimination , 2015, 1501.06066.

[13]  James Stephen Marron,et al.  Distance‐weighted discrimination , 2015 .

[14]  Hao Helen Zhang,et al.  Weighted Distance Weighted Discrimination and Its Asymptotic Properties , 2010, Journal of the American Statistical Association.

[15]  J. S. Marron,et al.  Distance-Weighted Discrimination , 2007 .

[16]  Kim-Chuan Toh,et al.  A Convergent 3-Block SemiProximal Alternating Direction Method of Multipliers for Conic Programming with 4-Type Constraints , 2014, SIAM J. Optim..

[17]  Boxiang Wang,et al.  A Multicategory Kernel Distance Weighted Discrimination Method for Multiclass Classification , 2019, Technometrics.

[18]  Noah Simon,et al.  A Sparse-Group Lasso , 2013 .

[19]  Xiaotong Shen,et al.  On L1-Norm Multiclass Support Vector Machines , 2007 .

[20]  Bo Peng,et al.  An Error Bound for L1-norm Support Vector Machine Coefficients in Ultra-high Dimension , 2016, J. Mach. Learn. Res..

[21]  H. Zou,et al.  Another look at distance‐weighted discrimination , 2018 .

[22]  Hanwen Huang,et al.  Large scale analysis of generalization error in learning using margin based classification methods , 2020, Journal of Statistical Mechanics: Theory and Experiment.

[23]  Li Zhang,et al.  Sparse wavelet estimation in quantile regression with multiple functional predictors , 2017, Comput. Stat. Data Anal..