Kernelized Unified Domain Adaptation on Geometrical Manifolds

Abstract Primitive machine learning algorithms like the k-nearest Neighbor (k-NN) and Support Vector Machine (SVM) are a major challenge for expert and intelligent systems that recognize objects with large-scale variations in lighting conditions, backgrounds, color, size, etc. The variations may be due to the fact that the training and test data may come from related but different domains. Considerable effort has been put into the advancement of domain adaptation methods. However, most of the existing work only concentrates on considering only a few of the following goals or objectives: (i) subspace alignment; (ii) Minimization of distribution divergence by using the Maximum Mean Discrepancy (MMD) criterion; (iii) Preservation of source domain discrimination information; (iv) Preservation of original similarity of the data samples; (v) Maximization of target domain variance. Current approaches to preserve source domain discriminant information can easily mis-classify target domain samples which are distributed near the edge of the cluster. In order to overcome the limitations of existing domain adaptation methods, and expert and intelligent systems, we propose the Unified Domain Adaptation on Geometrical Manifolds (UDAGM) framework. UDAGM optimizes all the aforementioned objectives jointly as well as uses the Regularized Coplanar Discriminant Analysis (RCDA) method for better inter-class separability and intra-class compactness. In addition, we extend our proposed framework UDAGM to a kernelised version in order to deal with non-linear separable datasets. Extensive experimentation on two real-world problems datasets (PIE face recognition and Office-Caltech) has proven that the proposed frameworks outperform several approaches to domain adaptation.

[1]  Ming Shao,et al.  Generalized Transfer Subspace Learning Through Low-Rank Constraint , 2014, International Journal of Computer Vision.

[2]  Rama Chellappa,et al.  Visual Domain Adaptation: A survey of recent advances , 2015, IEEE Signal Processing Magazine.

[3]  Yun Fu,et al.  Robust Transfer Metric Learning for Image Classification , 2017, IEEE Transactions on Image Processing.

[4]  Xiang Li,et al.  Deep Learning-Based Machinery Fault Diagnostics With Domain Adaptation Across Sensors at Different Places , 2020, IEEE Transactions on Industrial Electronics.

[5]  Ha Young Kim,et al.  Improving financial trading decisions using deep Q-learning: Predicting the number of shares, action strategies, and transfer learning , 2019, Expert Syst. Appl..

[6]  Ivor W. Tsang,et al.  Domain Adaptation via Transfer Component Analysis , 2009, IEEE Transactions on Neural Networks.

[7]  Jimson Mathew,et al.  A Kernelized Unified Framework for Domain Adaptation , 2019, IEEE Access.

[8]  M. A. Hossain,et al.  Intelligent image-based colourimetric tests using machine learning framework for lateral flow assays , 2020, Expert Syst. Appl..

[9]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[10]  Keith Worden,et al.  On the application of domain adaptation in structural health monitoring , 2020, Mechanical Systems and Signal Processing.

[11]  Trevor Darrell,et al.  Semi-supervised Domain Adaptation with Instance Constraints , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Weiguo Fan,et al.  A new image classification method using CNN transfer learning and web data augmentation , 2018, Expert Syst. Appl..

[13]  Yurong Liu,et al.  A survey of deep neural network architectures and their applications , 2017, Neurocomputing.

[14]  Ezzeddine Zagrouba,et al.  Deep feature learning for soft tissue sarcoma classification in MR images via transfer learning , 2019, Expert Syst. Appl..

[15]  Muhammad Uzair,et al.  Blind Domain Adaptation With Augmented Extreme Learning Machine Features , 2017, IEEE Transactions on Cybernetics.

[16]  Nor Azah Yusof,et al.  An intelligent mobile-enabled expert system for tuberculosis disease diagnosis in real time , 2018, Expert Syst. Appl..

[17]  Dong Liu,et al.  Robust visual domain adaptation with low-rank reconstruction , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Philip S. Yu,et al.  Adaptation Regularization: A General Framework for Transfer Learning , 2014, IEEE Transactions on Knowledge and Data Engineering.

[19]  Ming Shao,et al.  Deep Low-Rank Coding for Transfer Learning , 2015, IJCAI.

[20]  Mehrtash Tafazzoli Harandi,et al.  Learning an Invariant Hilbert Space for Domain Adaptation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Xiaogang Wang,et al.  Human Reidentification with Transferred Metric Learning , 2012, ACCV.

[22]  Hui Xiong,et al.  A Unified Framework for Metric Transfer Learning , 2017, IEEE Transactions on Knowledge and Data Engineering.

[23]  Mengjie Zhang,et al.  Scatter Component Analysis: A Unified Framework for Domain Adaptation and Domain Generalization , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Ming Li,et al.  2D-LDA: A statistical linear discriminant analysis for image matrix , 2005, Pattern Recognit. Lett..

[25]  Dacheng Tao,et al.  Bregman Divergence-Based Regularization for Transfer Subspace Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[26]  Gregory Ditzler,et al.  Domain adaptation bounds for multiple expert systems under concept drift , 2014, 2014 International Joint Conference on Neural Networks (IJCNN).

[27]  Yuan Shi,et al.  Geodesic flow kernel for unsupervised domain adaptation , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Dit-Yan Yeung,et al.  Transfer Metric Learning with Semi-Supervised Extension , 2012, TIST.

[29]  Asif Ekbal,et al.  Borrow from rich cousin: transfer learning for emotion detection using cross lingual embedding , 2020, Expert Syst. Appl..

[30]  Philip S. Yu,et al.  Transfer Feature Learning with Joint Distribution Adaptation , 2013, 2013 IEEE International Conference on Computer Vision.

[31]  Philip S. Yu,et al.  Transfer Joint Matching for Unsupervised Domain Adaptation , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Kate Saenko,et al.  Return of Frustratingly Easy Domain Adaptation , 2015, AAAI.

[33]  Anselmo Cardoso de Paiva,et al.  Lung nodule classification using artificial crawlers, directional texture and support vector machine , 2017, Expert Syst. Appl..

[34]  Tinne Tuytelaars,et al.  Unsupervised Visual Domain Adaptation Using Subspace Alignment , 2013, 2013 IEEE International Conference on Computer Vision.

[35]  Juergen Gall,et al.  Open Set Domain Adaptation , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[36]  Dao-Qing Dai,et al.  Regularized coplanar discriminant analysis for dimensionality reduction , 2017, Pattern Recognit..

[37]  Qionghai Dai,et al.  Low-Rank Structure Learning via Nonconvex Heuristic Recovery , 2010, IEEE Transactions on Neural Networks and Learning Systems.

[38]  Alex ChiChung Kot,et al.  Domain Generalization with Adversarial Feature Learning , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[39]  Fanjiang Xu,et al.  Cross-Domain Metric Learning Based on Information Theory , 2014, AAAI.

[40]  Jing Zhang,et al.  Joint Geometrical and Statistical Alignment for Visual Domain Adaptation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Bernhard Schölkopf,et al.  A Kernel Two-Sample Test , 2012, J. Mach. Learn. Res..

[42]  Muhammad A. Rushdi,et al.  Classification of scaled texture patterns with transfer learning , 2019, Expert Syst. Appl..

[43]  Kahkashan Afrin,et al.  A data-driven framework to new product demand prediction: Integrating product differentiation and transfer learning approach , 2018, Expert Syst. Appl..

[44]  Tatsuya Harada,et al.  Asymmetric Tri-training for Unsupervised Domain Adaptation , 2017, ICML.