Beyond cross-domain learning: Multiple-domain nonnegative matrix factorization

Traditional cross-domain learning methods transfer learning from a source domain to a target domain. In this paper, we propose the multiple-domain learning problem for several equally treated domains. The multiple-domain learning problem assumes that samples from different domains have different distributions, but share the same feature and class label spaces. Each domain could be a target domain, while also be a source domain for other domains. A novel multiple-domain representation method is proposed for the multiple-domain learning problem. This method is based on nonnegative matrix factorization (NMF), and tries to learn a basis matrix and coding vectors for samples, so that the domain distribution mismatch among different domains will be reduced under an extended variation of the maximum mean discrepancy (MMD) criterion. The novel algorithm - multiple-domain NMF (MDNMF) - was evaluated on two challenging multiple-domain learning problems - multiple user spam email detection and multiple-domain glioma diagnosis. The effectiveness of the proposed algorithm is experimentally verified.

[1]  Sethuraman Panchanathan,et al.  A Two-Stage Weighting Framework for Multi-Source Domain Adaptation , 2011, NIPS.

[2]  Shiliang Sun,et al.  Multitask Multiclass Support Vector Machines , 2011, 2011 IEEE 11th International Conference on Data Mining Workshops.

[3]  Shiliang Sun,et al.  Multitask multiclass support vector machines: Model and experiments , 2013, Pattern Recognit..

[4]  Liang-Tien Chia,et al.  Laplacian Sparse Coding, Hypergraph Laplacian Sparse Coding, and Applications , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Xiaolei Wang,et al.  Non-negative matrix factorization by maximizing correntropy for cancer clustering , 2013, BMC Bioinformatics.

[6]  Xin Gao,et al.  Adaptive graph regularized Nonnegative Matrix Factorization via feature selection , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[7]  Jim Jing-Yan Wang,et al.  Multiple graph regularized nonnegative matrix factorization , 2013, Pattern Recognit..

[8]  Snehamoy Chatterjee,et al.  Genetic algorithms for feature selection of image analysis-based quality monitoring model: An application to an iron mine , 2011, Eng. Appl. Artif. Intell..

[9]  BruzzoneLorenzo,et al.  Domain Adaptation Problems , 2010 .

[10]  Yi Yao,et al.  Boosting for transfer learning with multiple sources , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[11]  Xiangyang Wang,et al.  Active SVM-based relevance feedback using multiple classifiers ensemble and features reweighting , 2013, Eng. Appl. Artif. Intell..

[12]  G. Hemantha Kumar,et al.  Multilingual OCR system for South Indian scripts and English documents: An approach based on Fourier transform and principal component analysis , 2008, Eng. Appl. Artif. Intell..

[13]  Shih-Fu Chang,et al.  Cross-domain learning methods for high-level visual concept classification , 2008, 2008 15th IEEE International Conference on Image Processing.

[14]  Dacheng Tao,et al.  Ensemble manifold regularization , 2009, CVPR.

[15]  Rong Yan,et al.  Cross-domain video concept detection using adaptive svms , 2007, ACM Multimedia.

[16]  Hans-Peter Kriegel,et al.  Integrating structured biological data by Kernel Maximum Mean Discrepancy , 2006, ISMB.

[17]  Thomas S. Huang,et al.  Graph Regularized Nonnegative Matrix Factorization for Data Representation. , 2011, IEEE transactions on pattern analysis and machine intelligence.

[18]  T. Golub,et al.  Gene expression-based classification of malignant gliomas correlates better with survival than histological classification. , 2003, Cancer research.

[19]  Liang-Tien Chia,et al.  Multi-layer group sparse coding — For concurrent image classification and annotation , 2011, CVPR 2011.

[20]  E. Domany,et al.  Stem cell-related "self-renewal" signature and high epidermal growth factor receptor expression associated with resistance to concomitant chemoradiotherapy in glioblastoma. , 2008, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[21]  Hal Daumé,et al.  Frustratingly Easy Domain Adaptation , 2007, ACL.

[22]  Jie Yang,et al.  Initialization enhancer for non-negative matrix factorization , 2007, Eng. Appl. Artif. Intell..

[23]  Shiliang Sun,et al.  Dynamical ensemble learning with model-friendly classifiers for domain adaptation , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[24]  Lorenzo Bruzzone,et al.  Domain Adaptation Problems: A DASVM Classification Technique and a Circular Validation Strategy , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  S. Horvath,et al.  Gene Expression Profiling of Gliomas Strongly Predicts Survival , 2004, Cancer Research.

[26]  Arie Ben-David,et al.  About the relationship between ROC curves and Cohen's kappa , 2008, Eng. Appl. Artif. Intell..

[27]  Jayant P. Menon,et al.  Neuronal and glioma-derived stem cell factor induces angiogenesis within the brain. , 2006, Cancer cell.

[28]  Davar Giveki,et al.  Automatic detection of erythemato-squamous diseases using PSO-SVM based on association rules , 2013, Eng. Appl. Artif. Intell..

[29]  Ivor W. Tsang,et al.  Domain Adaptation via Transfer Component Analysis , 2009, IEEE Transactions on Neural Networks.

[30]  Jiawei Han,et al.  Non-negative Matrix Factorization on Manifold , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[31]  Farshid Keynia.  A new feature selection algorithm and composite neural network for electricity price forecasting , 2012, Eng. Appl. Artif. Intell..

[32]  Ivor W. Tsang,et al.  Domain Transfer SVM for video concept detection , 2009, CVPR 2009.

[33]  ChatterjeeSnehamoy,et al.  Genetic algorithms for feature selection of image analysis-based quality monitoring model , 2011 .

[34]  Anke Meyer-Bäse,et al.  ICA, kernel methods and nonnegativity: New paradigms for dynamical component analysis of fMRI data , 2009, Eng. Appl. Artif. Intell..

[35]  Hui Xiong,et al.  Cross-Domain Learning from Multiple Sources: A Consensus Regularization Perspective , 2010, IEEE Transactions on Knowledge and Data Engineering.

[36]  Hujun Bao,et al.  Understanding the Power of Clause Learning , 2009, IJCAI.

[37]  Shiliang Sun,et al.  Cross-domain representation-learning framework with combination of class-separate and domain-merge objectives , 2012, CDKD '12.

[38]  Lei Wang,et al.  AdaBoost with SVM-based component classifiers , 2008, Eng. Appl. Artif. Intell..

[39]  Ying Zhang,et al.  Boosted Learning of Visual Word Weighting Factors for Bag-of-Features Based Medical Image Retrieval , 2011, 2011 Sixth International Conference on Image and Graphics.

[40]  Shih-Fu Chang,et al.  Semi-Supervised Hashing for Large-Scale Search , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Ivor W. Tsang,et al.  Domain Transfer Multiple Kernel Learning , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  Yi Yang,et al.  A Multimedia Retrieval Framework Based on Semi-Supervised Ranking and Relevance Feedback , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[43]  Shih-Fu Chang,et al.  Semi-supervised hashing for scalable image retrieval , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[44]  Xian-Sheng Hua,et al.  Ensemble Manifold Regularization , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  Farbod Razzazi,et al.  A clustering based feature selection method in spectro-temporal domain for speech recognition , 2012, Eng. Appl. Artif. Intell..

[46]  Krzysztof Fujarewicz,et al.  Using SVD and SVM methods for selection, classification, clustering and modeling of DNA microarray data , 2004, Eng. Appl. Artif. Intell..

[47]  Ivor W. Tsang,et al.  This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 1 Domain Adaptation from Multiple Sources: A Domain- , 2022 .