Regularized maximum correntropy machine

In this paper we investigate the usage of regularized correntropy framework for learning of classifiers from noisy labels. The class label predictors learned by minimizing transitional loss functions are sensitive to the noisy and outlying labels of training samples, because the transitional loss functions are equally applied to all the samples. To solve this problem, we propose to learn the class label predictors by maximizing the correntropy between the predicted labels and the true labels of the training samples, under the regularized Maximum Correntropy Criteria (MCC) framework. Moreover, we regularize the predictor parameter to control the complexity of the predictor. The learning problem is formulated by an objective function considering the parameter regularization and MCC simultaneously. By optimizing the objective function alternately, we develop a novel predictor learning algorithm. The experiments on two challenging pattern classification tasks show that it significantly outperforms the machines with transitional loss functions.

[1]  Zhiwu Lu,et al.  A Publishing Framework for Digitally Augmented Paper Documents: Towards Cross-Media Information Integration , 2006, PCM.

[2]  Shouhuai Xu,et al.  An evasion and counter-evasion study in malicious websites detection , 2014, 2014 IEEE Conference on Communications and Network Security.

[3]  Qi Hao,et al.  A Multi-Agent-Based Intelligent Sensor and Actuator Network Design for Smart House and Home Automation , 2013, J. Sens. Actuator Networks.

[4]  Zhiwu Lu,et al.  Unsupervised Image Categorization Using Constrained Entropy-Regularized Likelihood Learning with Pairwise Constraints , 2007, ISNN.

[5]  Cheng-Kok Koh,et al.  Passivity Enforcement for Descriptor Systems Via Matrix Pencil Perturbation , 2012, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[6]  Jim Jing-Yan Wang,et al.  Discriminative sparse coding on multi-manifolds , 2013, Knowl. Based Syst..

[7]  Paul Ampadu,et al.  Reliable Ultra-Low-Voltage Cache Design for Many-Core Systems , 2012, IEEE Transactions on Circuits and Systems II: Express Briefs.

[8]  Hong Man,et al.  DSPM: Dynamic Structure Preserving Map for action recognition , 2013, 2013 IEEE International Conference on Multimedia and Expo (ICME).

[9]  Zhiwu Lu,et al.  Action Recognition Based on Learnt Motion Semantic Vocabulary , 2010, PCM.

[10]  Shouhuai Xu,et al.  Trustworthy Information: Concepts and Mechanisms , 2010, WAIM.

[11]  Zhiwu Lu,et al.  Image categorization with spatial mismatch kernels , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Zhiwu Lu,et al.  Generalized Relevance Models for Automatic Image Annotation , 2009, PCM.

[13]  Yi Wang,et al.  SAGA: array storage as a DB with support for structural aggregations , 2014, SSDBM '14.

[14]  Shouhuai Xu,et al.  Cross-layer detection of malicious websites , 2013, CODASPY.

[15]  Chao Wang,et al.  ProClusEnsem: Predicting membrane protein types by fusing different modes of pseudo amino acid composition , 2012, Comput. Biol. Medicine.

[16]  Xindong Wu,et al.  Online learning with mobile sensor data for user recognition , 2014, SAC.

[17]  Zhiwu Lu,et al.  Spectral learning of latent semantics for action recognition , 2011, 2011 International Conference on Computer Vision.

[18]  Zhiwu Lu,et al.  A Regularized Minimum Cross-entropy Algorithm on Mixture of Experts for Curve Detection , 2005, 2005 International Conference on Neural Networks and Brain.

[19]  Jim Jing-Yan Wang,et al.  Sparse structure regularized ranking , 2014, Multimedia Tools and Applications.

[20]  Ran He,et al.  A Regularized Correntropy Framework for Robust Pattern Recognition , 2011, Neural Computation.

[21]  Jianqiu Zhang,et al.  Mathematical modeling and stability analysis of macrophage activation in left ventricular remodeling post-myocardial infarction , 2012, BMC Genomics.

[22]  Shouhuai Xu,et al.  A Characterization of Cybersecurity Posture from Network Telescope Data , 2014, INTRUST.

[23]  Zhiwu Lu,et al.  Latent semantic learning with structured sparse representation for human action recognition , 2011, Pattern Recognit..

[24]  Ran He,et al.  Maximum Correntropy Criterion for Robust Face Recognition , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Yi Wang,et al.  Maximum mutual information regularized classification , 2014, Eng. Appl. Artif. Intell..

[26]  Xin-Lin Huang,et al.  Non-informative hierarchical Bayesian inference for non-negative matrix factorization , 2015, Signal Process..

[27]  Zhiwu Lu,et al.  Latent Semantic Learning by Efficient Sparse Coding with Hypergraph Regularization , 2011, AAAI.

[28]  Zhiwu Lu,et al.  A Semi-supervised Learning Algorithm on Gaussian Mixture with Automatic Model Selection , 2007, Neural Processing Letters.

[29]  Jim Jing-Yan Wang,et al.  Beyond cross-domain learning: Multiple-domain nonnegative matrix factorization , 2014, Eng. Appl. Artif. Intell..

[30]  Qingquan Sun Neuro-Disorder Patient Monitoring via Gait Sensor Networks: Toward an Intelligent, Context-Oriented Signal Processing , 2012 .

[31]  Xindong Wu,et al.  Online Group Feature Selection from Feature Streams , 2013, AAAI.

[32]  Jim Jing-Yan Wang,et al.  Joint learning and weighting of visual vocabulary for bag-of-feature based tissue classification , 2013, Pattern Recognit..

[33]  Chi-Un Lei,et al.  On Vector Fitting methods in signal/power integrity applications , 2010 .

[34]  Jim Jing-Yan Wang,et al.  Multiple graph regularized nonnegative matrix factorization , 2013, Pattern Recognit..

[35]  Honggang Zhang,et al.  Region-based high-level semantics extraction with CEDD , 2010, 2010 2nd IEEE InternationalConference on Network Infrastructure and Digital Content.

[36]  Samuel T. Ariaratnam,et al.  Network Capacity Assessment of Combined Heat and Power-Based Distributed Generation in Urban Energy Infrastructures , 2013, IEEE Transactions on Smart Grid.

[37]  Zhiwu Lu,et al.  Learning descriptive visual representation for image classification and annotation , 2015, Pattern Recognit..

[38]  Zhiwu Lu,et al.  Heterogeneous Constraint Propagation with Constrained Sparse Representation , 2012, 2012 IEEE 12th International Conference on Data Mining.

[39]  Shouhuai Xu,et al.  Adaptive Epidemic Dynamics in Networks , 2013, ACM Trans. Auton. Adapt. Syst..

[40]  Jim Jing-Yan Wang,et al.  Feature selection and multi-kernel learning for sparse representation on a manifold , 2014, Neural Networks.

[41]  Zhiwu Lu,et al.  Context-based multi-label image annotation , 2009, CIVR '09.

[42]  Zhiwu Lu,et al.  Image annotation by semantic sparse recoding of visual content , 2012, ACM Multimedia.

[43]  Bao-Gang Hu,et al.  Robust feature extraction via information theoretic learning , 2009, ICML '09.

[44]  Honggang Zhang,et al.  Document Clustering Based On Max-Correntropy Non-Negative Matrix Factorization , 2014, ArXiv.

[45]  Ngai Wong,et al.  Frequency-domain transient analysis of multitime partial differential equation systems , 2011, 2011 IEEE/IFIP 19th International Conference on VLSI and System-on-Chip.

[46]  Zhen Qin,et al.  Documents clustering based on max-correntropy nonnegative matrix factorization , 2014, 2014 International Conference on Machine Learning and Cybernetics.

[47]  Peng Wu,et al.  Unsupervised Multi-Level Non-Negative Matrix Factorization Model: Binary Data Case , 2012, J. Information Security.

[48]  J. Wang,et al.  Semi-Supervised Transductive Hot Spot Predictor Working on Multiple Assumptions , 2014 .

[49]  Shouhuai Xu,et al.  Characterizing Honeypot-Captured Cyber Attacks: Statistical Framework and Case Study , 2013, IEEE Transactions on Information Forensics and Security.

[50]  Zhiwu Lu,et al.  Learning Descriptive Visual Representation by Semantic Regularized Matrix Factorization , 2013, IJCAI.

[51]  Zhiwu Lu,et al.  Spatial temporal pyramid matching using temporal sparse representation for human motion retrieval , 2014, The Visual Computer.

[52]  Zhiwu Lu,et al.  Gaussian mixture learning via robust competitive agglomeration , 2010, Pattern Recognit. Lett..

[53]  Ran He,et al.  Robust Semi-Supervised Learning Algorithm Based on Maximum Correntropy Criterion: Robust Semi-Supervised Learning Algorithm Based on Maximum Correntropy Criterion , 2012 .

[54]  Samuel T. Ariaratnam,et al.  Optimal Allocation of CHP-Based Distributed Generation on Urban Energy Distribution Networks , 2014, IEEE Transactions on Sustainable Energy.

[55]  Paul Ampadu,et al.  Variation-tolerant cache by two-layer error control codes , 2013, 2013 IEEE International Symposium on Defect and Fault Tolerance in VLSI and Nanotechnology Systems (DFTS).

[56]  Mark D. Smucker,et al.  Tolerance of Effectiveness Measures to Relevance Judging Errors , 2014, ECIR.

[57]  Zhiwu Lu,et al.  L1-graph construction using structured sparsity , 2013, Neurocomputing.

[58]  Yufang Jin,et al.  A conceptual cellular interaction model of left ventricular remodelling post-MI: dynamic network with exit-entry competition strategy , 2010, BMC Systems Biology.

[59]  Zhiwu Lu,et al.  Semantic Sparse Recoding of Visual Content for Image Applications , 2011, IEEE Transactions on Image Processing.

[60]  Tao Li,et al.  PreDNA: accurate prediction of DNA-binding sites in proteins by integrating sequence and geometric structure information , 2013, Bioinform..

[61]  Xiaolei Wang,et al.  Non-negative matrix factorization by maximizing correntropy for cancer clustering , 2013, BMC Bioinformatics.

[62]  Yi Wang,et al.  Supporting a Light-Weight Data Management Layer over HDF5 , 2013, 2013 13th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing.

[63]  Xin Gao,et al.  ProDis-ContSHC: learning protein dissimilarity measures and hierarchical context coherently for protein-protein comparison in protein database retrieval , 2012, BMC Bioinformatics.

[64]  Zhiwu Lu,et al.  Unsupervised learning of finite mixtures using entropy regularization and its application to image segmentation , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[65]  Jim Jing-Yan Wang,et al.  Supervised Transfer Sparse Coding , 2014, AAAI.

[66]  Xindong Wu,et al.  Group Feature Selection with Streaming Features , 2013, 2013 IEEE 13th International Conference on Data Mining.

[67]  Xin-Lin Huang,et al.  Primate-inspired adaptive routing in intermittently connected mobile communication systems , 2014, Wirel. Networks.

[68]  Xin Gao,et al.  Multiple graph regularized protein domain ranking , 2012, BMC Bioinformatics.

[69]  Honggang Zhang,et al.  Graph Regularized Non-negative Matrix Factorization By Maximizing Correntropy , 2014, J. Comput..

[70]  Zhiwu Lu An Iterative Entropy Regularized Likelihood Learning Algorithm for Cluster Analysis with the Number of Clusters Automatically Detected , 2005, 2005 International Conference on Neural Networks and Brain.

[71]  Shouhuai Xu,et al.  A Stochastic Model of Multivirus Dynamics , 2012, IEEE Transactions on Dependable and Secure Computing.

[72]  Xin Gao,et al.  Adaptive graph regularized Nonnegative Matrix Factorization via feature selection , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[73]  Zhiwu Lu,et al.  Exhaustive and Efficient Constraint Propagation: A Graph-Based Learning Approach and Its Applications , 2011, International Journal of Computer Vision.

[74]  Qi Hao,et al.  Context awareness emergence for distributed binary pyroelectric sensors , 2010, 2010 IEEE Conference on Multisensor Fusion and Integration.

[75]  Zhiwu Lu,et al.  Noise-robust semi-supervised learning via fast sparse coding , 2015, Pattern Recognit..

[76]  Fei Hu,et al.  A Fast Raptor Codes Decoding Strategy for Real-Time Communication Systems , 2013, Netw. Commun. Technol..

[77]  Honggang Zhang,et al.  Adaptive Learning of Region-based pLSA Model for Total Scene Annotation , 2013, ArXiv.

[78]  Zhiwu Lu,et al.  Spatial Markov Kernels for Image Categorization and Annotation , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[79]  Chunhai Fan,et al.  A graphene-based sensor array for high-precision and adaptive target identification with ensemble aptamers. , 2012, Journal of the American Chemical Society.

[80]  Shie Mannor,et al.  Robustness and Regularization of Support Vector Machines , 2008, J. Mach. Learn. Res..

[81]  Yi Wang,et al.  SciMATE: A Novel MapReduce-Like Framework for Multiple Scientific Data Formats , 2012, 2012 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (ccgrid 2012).

[82]  Zhiwu Lu,et al.  Unified Constraint Propagation on Multi-View Data , 2013, AAAI.

[83]  Zhiwu Lu Unsupervised Image Segmentation Using an Iterative Entropy Regularized Likelihood Learning Algorithm , 2006, ISNN.

[84]  Horace Ho-Shing Ip Contextual Kernel and Spectral Methods for Learning the Semantics of Images , 2011, IEEE Transactions on Image Processing.

[85]  Xindong Wu,et al.  K-Means Clustering with Bagging and MapReduce , 2011, 2011 44th Hawaii International Conference on System Sciences.