Multiclass linear discriminant analysis with ultrahigh‐dimensional features

Within the framework of Fisher's discriminant analysis, we propose a multiclass classification method which embeds variable screening for ultrahigh-dimensional predictors. Leveraging interfeature correlations, we show that the proposed linear classifier recovers informative features with probability tending to one and can asymptotically achieve a zero misclassification rate. We evaluate the finite sample performance of the method via extensive simulations and use this method to classify posttransplantation rejection types based on patients' gene expressions.

[1]  Jianqing Fan,et al.  COVARIANCE ASSISTED SCREENING AND ESTIMATION. , 2014, Annals of statistics.

[2]  Jeffrey S. Morris,et al.  Sure independence screening for ultrahigh dimensional feature space Discussion , 2008 .

[3]  W. Xiao,et al.  HLA-G expression in the peripheral blood of live kidney transplant recipients. , 2013, Chinese medical journal.

[4]  Jason Weston,et al.  Multi-Class Support Vector Machines , 1998 .

[5]  Zhou Yu,et al.  On marginal sliced inverse regression for ultrahigh dimensional model-free feature selection , 2016 .

[6]  Trevor Hastie,et al.  Regularized linear discriminant analysis and its application in microarrays. , 2007, Biostatistics.

[7]  Chih-Jen Lin,et al.  Probability Estimates for Multi-class Classification by Pairwise Coupling , 2003, J. Mach. Learn. Res..

[8]  Simultaneous variable selection and class fusion for high-dimensional linear discriminant analysis. , 2010, Biostatistics.

[9]  R. Tibshirani,et al.  Penalized Discriminant Analysis , 1995 .

[10]  Yoshua Bengio,et al.  Pattern Recognition and Neural Networks , 1995 .

[11]  D. Leaf,et al.  BPI Fold-Containing Family A Member 2/Parotid Secretory Protein Is an Early Biomarker of AKI. , 2017, Journal of the American Society of Nephrology : JASN.

[12]  Ashutosh Kumar Singh,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2010 .

[13]  Jianqing Fan,et al.  High Dimensional Classification Using Features Annealed Independence Rules. , 2007, Annals of statistics.

[14]  P. Bickel,et al.  Covariance regularization by thresholding , 2009, 0901.3079.

[15]  H. Zou,et al.  A direct approach to sparse discriminant analysis in ultra-high dimensions , 2012 .

[16]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[17]  Trevor Hastie,et al.  Regularized Discriminant Analysis and Its Application in Microarrays , 2004 .

[18]  Chih-Jen Lin,et al.  A Comparison of Methods for Multi-class Support Vector Machines , 2015 .

[19]  Jianqing Fan,et al.  Sure independence screening for ultrahigh dimensional feature space , 2006, math/0612857.

[20]  S. Gowder.  Renal Membrane Transport Proteins and the Transporter Genes , 2014 .

[21]  David Rossell,et al.  Tractable Bayesian Variable Selection: Beyond Normality , 2016, Journal of the American Statistical Association.

[22]  Ivan Tyukin,et al.  Correction of AI systems by linear discriminants: Probabilistic foundations , 2018, Inf. Sci..

[23]  Yang Feng,et al.  A road to classification in high dimensional space: the regularized optimal affine discriminant , 2010, Journal of the Royal Statistical Society. Series B, Statistical methodology.

[24]  Chih-Jen Lin,et al.  A comparison of methods for multiclass support vector machines , 2002, IEEE Trans. Neural Networks.

[25]  J. Shao,et al.  Sparse linear discriminant analysis by thresholding for high dimensional data , 2011, 1105.3561.

[26]  R. Cortese,et al.  LFB1 and LFB3 homeoproteins are sequentially expressed during kidney development. , 1992, Development.

[27]  Gerhard Widmer,et al.  Deep Linear Discriminant Analysis , 2015, ICLR.

[28]  Lixing Zhu,et al.  Covariance-enhanced discriminant analysis. , 2015, Biometrika.

[29]  Runze Li,et al.  Ultrahigh-Dimensional Multiclass Linear Discriminant Analysis by Pairwise Sure Independence Screening , 2016, Journal of the American Statistical Association.

[30]  Jiashun Jin,et al.  Impossibility of successful classification when useful features are rare and weak , 2009, Proceedings of the National Academy of Sciences.

[31]  Michael I. Jordan,et al.  On Discriminative vs. Generative Classifiers: A comparison of logistic regression and naive Bayes , 2001, NIPS.

[32]  Yoram Singer,et al.  Reducing Multiclass to Binary: A Unifying Approach for Margin Classifiers , 2000, J. Mach. Learn. Res..

[33]  Sandra E. Safo,et al.  General sparse multi-class linear discriminant analysis , 2016, Comput. Stat. Data Anal..

[34]  T. Cai,et al.  A Direct Estimation Approach to Sparse Linear Discriminant Analysis , 2011, 1107.3442.

[35]  S. Leal,et al.  Methods for detecting associations with rare variants for common diseases: application to analysis of sequence data. , 2008, American journal of human genetics.

[36]  R. Tibshirani,et al.  Penalized classification using Fisher's linear discriminant , 2011, Journal of the Royal Statistical Society. Series B, Statistical methodology.

[37]  Wei Cai,et al.  Network linear discriminant analysis , 2018, Comput. Stat. Data Anal..

[38]  V. Johnson,et al.  Bayesian Model Selection in High-Dimensional Settings , 2012, Journal of the American Statistical Association.

[39]  Yi Yang,et al.  Multiclass Sparse Discriminant Analysis , 2015, 1504.05845.

[40]  S. Horvath,et al.  Kidney Transplant Rejection and Tissue Injury by Gene Profiling of Biopsies and Peripheral Blood Lymphocytes , 2004, American journal of transplantation : official journal of the American Society of Transplantation and the American Society of Transplant Surgeons.

[41]  Alexander J. Smola,et al.  Bundle Methods for Regularized Risk Minimization , 2010, J. Mach. Learn. Res..

[42]  Azriel Rosenfeld,et al.  Computer Vision , 1988, Adv. Comput..

[43]  Wenyi Wang,et al.  Bayesian variable selection for binary outcomes in high-dimensional genomic studies using non-local priors , 2016, Bioinform..

[44]  Martin T. Wells,et al.  Simultaneous Sparse Estimation of Canonical Vectors in the p ≫ N Setting , 2014, 1403.6095.

[45]  Jianqing Fan,et al.  High Dimensional Covariance Matrix Estimation in Approximate Factor Models , 2011, Annals of statistics.

[46]  R. Tibshirani,et al.  Discriminant Analysis by Gaussian Mixtures , 1996 .

[47]  W Y Zhang,et al.  Discussion on `Sure independence screening for ultra-high dimensional feature space' by Fan, J and Lv, J. , 2008 .

[48]  Valen E Johnson.  On Numerical Aspects of Bayesian Model Selection in High and Ultrahigh-dimensional Settings. , 2013, Bayesian analysis.

[49]  Nello Cristianini,et al.  Large Margin DAGs for Multiclass Classification , 1999, NIPS.