FIDOS: A generalized Fisher based feature extraction method for domain shift

Traditional pattern recognition techniques often assume that the data sets used for training and testing follow the same distribution. However, this assumption is usually not true for many real world problems as data from the same classes but different domains, e.g., data are collected under different conditions, may show different characteristics. We introduce FIDOS, a generalized FIsher based method for DOmain Shift problem, that aims at learning invariant features across domains in a supervised manner. Different from classical Fisher feature extraction, FIDOS aims to minimize not only the within-class scatter but also the difference in distributions between domains. Therefore, the subspace constructed by FIDOS reduces the drift in distributions among different domains and at the same time preserves the discriminants across classes. Another advantage of FIDOS over classical Fisher is that FIDOS extracts more features when multiple source domains are available in the training set; this is essential for a good classification especially when the number of classes is small. Experimental results on both artificial and real data and comparisons with other methods demonstrate the efficiency of our method in classifying objects under domain shift situations.

[1]  Shiliang Sun,et al.  Transferable Discriminative Dimensionality Reduction , 2011, 2011 IEEE 23rd International Conference on Tools with Artificial Intelligence.

[2]  Ad Aertsen,et al.  Review of the BCI Competition IV , 2012, Front. Neurosci..

[3]  Bram van Ginneken,et al.  Segmentation of the posterior ribs in chest radiographs using iterated contextual pixel classification , 2006, IEEE Transactions on Medical Imaging.

[4]  Robert P. W. Duin,et al.  Linear dimensionality reduction via a heteroscedastic extension of LDA: the Chernoff criterion , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Goo Jun,et al.  Spatially Adaptive Classification of Land Cover With Remote Sensing Data , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[6]  Marco Loog,et al.  Nearest neighbor-based importance weighting , 2012, 2012 IEEE International Workshop on Machine Learning for Signal Processing.

[7]  Rama Chellappa,et al.  Domain adaptation for object recognition: An unsupervised approach , 2011, 2011 International Conference on Computer Vision.

[8]  Ivor W. Tsang,et al.  Domain Adaptation via Transfer Component Analysis , 2009, IEEE Transactions on Neural Networks.

[9]  Bart M. ter Haar Romeny,et al.  Geometry-Driven Diffusion in Computer Vision , 1994, Computational Imaging and Vision.

[10]  Klaus-Robert Müller,et al.  Covariate Shift Adaptation by Importance Weighted Cross Validation , 2007, J. Mach. Learn. Res..

[11]  Bernhard Schölkopf,et al.  Correcting Sample Selection Bias by Unlabeled Data , 2006, NIPS.

[12]  Joydeep Ghosh,et al.  An Active Learning Approach to Hyperspectral Data Classification , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[13]  Aleix M. Martínez,et al.  Subclass discriminant analysis , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Neil D. Lawrence,et al.  Dataset Shift in Machine Learning , 2009 .

[15]  Keinosuke Fukunaga,et al.  Introduction to Statistical Pattern Recognition , 1972 .

[16]  J. Koenderink,et al.  Representation of local geometry in the visual system , 1987, Biological Cybernetics.

[17]  Ivor W. Tsang,et al.  Domain Transfer Multiple Kernel Learning , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Yishay Mansour,et al.  Domain Adaptation with Multiple Sources , 2008, NIPS.

[19]  LoogMarco,et al.  Linear Dimensionality Reduction via a Heteroscedastic Extension of LDA , 2004 .

[20]  Hui Xiong,et al.  Transfer learning from multiple source domains via consensus regularization , 2008, CIKM '08.

[21]  K. Müller,et al.  Finding stationary subspaces in multivariate time series. , 2009, Physical review letters.

[22]  K. Doi,et al.  Development of a digital image database for chest radiographs with and without a lung nodule: receiver operating characteristic analysis of radiologists' detection of pulmonary nodules. , 2000, AJR. American journal of roentgenology.

[23]  R. Fisher.  THE STATISTICAL UTILIZATION OF MULTIPLE MEASUREMENTS , 1938 .

[24]  Ivor W. Tsang,et al.  Visual Event Recognition in Videos by Learning from Web Data , 2012, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Keinosuke Fukunaga,et al.  Introduction to statistical pattern recognition (2nd ed.) , 1990 .

[26]  Steffen Bickel,et al.  Discriminative Learning Under Covariate Shift , 2009, J. Mach. Learn. Res..

[27]  Trevor Darrell,et al.  Adapting Visual Category Models to New Domains , 2010, ECCV.

[28]  Yoshinobu Kawahara,et al.  Separation of stationary and non-stationary sources with a generalized eigenvalue problem , 2012, Neural Networks.