Multi-task analysis discriminative dictionary learning for one-class learning

Abstract One-class classification is a generalization of supervised learning based on one class of examples. It attracts growing attention in machine learning and data mining. In this paper, we propose a novel approach called multi-task dictionary learning for one-class learning (MTD-OC), which incorporates analysis discriminative dictionary learning into one-class learning. The analysis discriminative dictionary learning makes sure that dictionaries responding to different tasks are independent and discriminating as much as possible. The analysis discriminative dictionary learning simultaneously minimize l 2 , 1 -norm constraint, analysis incoherence term and sparse code extraction term, which aim to promote analysis incoherence and improve coding efficiency and accuracy for classification. The one-class classifier on the target task is then constructed by learning transfer knowledge from multiple source tasks. Here, one-class classification improves the performance of analysis discriminative dictionary, while analysis discriminative dictionary improves the performance of one-class classification term. In MTD-OC, the optimization function is formulated to deal with one-class classifier and analysis discriminative dictionary learning based on one class of examples. Then, we propose an iterative framework to solve the optimization function, and obtain the predictive classifier for the target class. Extensive experiments have shown that MTD-OC can improve the accuracy of one-class classifier by learning analysis discriminative dictionary from each task to construct a transfer classifier.

[1]  Andrew Y. Ng,et al.  The Importance of Encoding Versus Training with Sparse Coding and Vector Quantization , 2011, ICML.

[2]  John Blitzer,et al.  Domain Adaptation with Structural Correspondence Learning , 2006, EMNLP.

[3]  Xuelong Li,et al.  Joint Embedding Learning and Sparse Regression: A Framework for Unsupervised Feature Selection , 2014, IEEE Transactions on Cybernetics.

[4]  Longbing Cao,et al.  A Similarity-Based Classification Framework for Multiple-Instance Learning , 2014, IEEE Transactions on Cybernetics.

[5]  Francisco Herrera,et al.  A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms , 2011, Swarm Evol. Comput..

[6]  Tao Zhang,et al.  Collaborative representation based face classification exploiting block weighted LBP and analysis dictionary learning , 2019, Pattern Recognit..

[7]  Hau-San Wong,et al.  Unsupervised Multi-task Learning with Hierarchical Data Structure , 2019, Pattern Recognit..

[8]  Kenneth E. Barner,et al.  Label consistent recursive least squares dictionary learning for image classification , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[9]  Robert P. W. Duin,et al.  Support Vector Data Description , 2004, Machine Learning.

[10]  Dorit Merhof,et al.  Transfer Learning for Breast Cancer Malignancy Classification based on Dynamic Contrast-Enhanced MR Images , 2018, Bildverarbeitung für die Medizin.

[11]  Yanjiang Wang,et al.  Class specific centralized dictionary learning based kernel collaborative representation for fine-grained image classification , 2016, 2016 IEEE 13th International Conference on Signal Processing (ICSP).

[12]  C. Bennila Thangammal,et al.  Visible and infrared image fusion using DTCWT and adaptive combined clustered dictionary , 2018, Infrared Physics & Technology.

[13]  Tom Heskes,et al.  Task Clustering and Gating for Bayesian Multitask Learning , 2003, J. Mach. Learn. Res..

[14]  Xin Zheng,et al.  A multi-task transfer learning method with dictionary learning , 2020, Knowl. Based Syst..

[15]  Lei Zhang,et al.  Sparse representation or collaborative representation: Which helps face recognition? , 2011, 2011 International Conference on Computer Vision.

[16]  Abd-Krim Seghouane,et al.  Consistent adaptive sequential dictionary learning , 2018, Signal Process..

[17]  Jacob Scharcanski,et al.  Dictionaries of deep features for land-use scene classification of very high spatial resolution images , 2019, Pattern Recognit..

[18]  Jian Yang,et al.  A Locality-Constrained and Label Embedding Dictionary Learning Algorithm for Image Classification , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[19]  Luc Van Gool,et al.  Temporal 3D ConvNets: New Architecture and Transfer Learning for Video Classification , 2017, ArXiv.

[20]  Qiang Liu,et al.  MST-GEN: An Efficient Parameter Selection Method for One-Class Extreme Learning Machine , 2017, IEEE Transactions on Cybernetics.

[21]  Yi Ma,et al.  Learning Category-Specific Dictionary and Shared Dictionary for Fine-Grained Image Categorization , 2014, IEEE Transactions on Image Processing.

[22]  Chi-Keong Goh,et al.  Co-evolutionary multi-task learning with predictive recurrence for multi-step chaotic time series prediction , 2017, Neurocomputing.

[23]  David Zhang,et al.  Sparse Representation Based Fisher Discrimination Dictionary Learning for Image Classification , 2014, International Journal of Computer Vision.

[24]  Kathrin Klamroth,et al.  Biconvex sets and optimization with biconvex functions: a survey and extensions , 2007, Math. Methods Oper. Res..

[25]  Pierre Beauseroy,et al.  Multi-task learning for one-class SVM with additional new features , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[26]  Jie Zhou,et al.  Multi-task clustering via domain adaptation , 2012, Pattern Recognit..

[27]  Xuefeng Chen,et al.  Sparse representation based on parametric impulsive dictionary design for bearing fault diagnosis , 2019, Mechanical Systems and Signal Processing.

[28]  Alain Trémeau,et al.  Multi-task, multi-domain learning: Application to semantic segmentation and pose regression , 2017, Neurocomputing.

[29]  Lei Zhang,et al.  Projective dictionary pair learning for pattern classification , 2014, NIPS.

[30]  Sundaram Suresh,et al.  Online Learning with Regularized Kernel for One-class Classification , 2017, ArXiv.

[31]  Chung-Lin Huang,et al.  Vehicle verification between two nonoverlapped views using sparse representation , 2018, Pattern Recognit..

[32]  Zhixin Yang,et al.  Multilayer one-class extreme learning machine , 2019, Neural Networks.

[33]  Asok Ray,et al.  Multimodal Task-Driven Dictionary Learning for Image Classification , 2015, IEEE Transactions on Image Processing.

[34]  Wenkai Li,et al.  A Positive and Unlabeled Learning Algorithm for One-Class Classification of Remote-Sensing Data , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[35]  Bo Liu,et al.  An efficient active learning method for multi-task learning , 2020, Knowl. Based Syst..

[36]  Ralf Peeters,et al.  Testing exchangeability for transfer decision , 2017, Pattern Recognit. Lett..

[37]  José Ragot,et al.  Multi-task learning with one-class SVM , 2014, Neurocomputing.

[38]  Shambhu J. Upadhyaya,et al.  Transfer learning in long-text keystroke dynamics , 2017, 2017 IEEE International Conference on Identity, Security and Behavior Analysis (ISBA).

[39]  Licheng Jiao,et al.  Discriminative Dictionary Learning With Two-Level Low Rank and Group Sparse Decomposition for Image Classification , 2017, IEEE Transactions on Cybernetics.

[40]  Adam Jatowt,et al.  Estimating a one-class naive Bayes text classifier , 2020, Intell. Data Anal..

[41]  Kun Zhao,et al.  Multi-task feature selection with sparse regularization to extract common and task-specific features , 2019, Neurocomputing.

[42]  Guillermo Sapiro,et al.  Classification and clustering via dictionary learning with structured incoherence and shared features , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[43]  Pedro M. Domingos,et al.  Deep transfer via second-order Markov logic , 2009, ICML '09.

[44]  Amparo Alonso-Betanzos,et al.  One-Class Convex Hull-Based Algorithm for Classification in Distributed Environments , 2020, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[45]  Wen Gao,et al.  Multi-Task Learning with Low Rank Attribute Embedding for Multi-Camera Person Re-Identification , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[46]  Lili Yin,et al.  Active learning based support vector data description method for robust novelty detection , 2018, Knowl. Based Syst..

[47]  Svetha Venkatesh,et al.  Multiple task transfer learning with small sample sizes , 2015, Knowledge and Information Systems.

[48]  Jiawei Han,et al.  PEBL: Web page classification without negative examples , 2004, IEEE Transactions on Knowledge and Data Engineering.

[49]  Xinge You,et al.  Robust face recognition via occlusion dictionary learning , 2014, Pattern Recognit..

[50]  Guozhu Dong,et al.  OCLEP+: One-class Anomaly and Intrusion Detection Using Minimal Length of Emerging Patterns , 2018, ArXiv.

[51]  Sheng Tang,et al.  Scalable logo recognition based on compact sparse dictionary for mobile devices , 2015, 2015 IEEE 17th International Workshop on Multimedia Signal Processing (MMSP).

[52]  Bianca Zadrozny,et al.  Learning and evaluating classifiers under sample selection bias , 2004, ICML.

[53]  Kristen Grauman,et al.  Learning with Whom to Share in Multi-task Feature Learning , 2011, ICML.

[54]  Zhong Ming,et al.  Transfer Learning for Heterogeneous One-Class Collaborative Filtering , 2016, IEEE Intelligent Systems.

[55]  Jürgen Beyerer,et al.  Gaussian Mixture Trees for One Class Classification in Automated Visual Inspection , 2017, ICIAR.

[56]  Lawrence Carin,et al.  Logistic regression with an auxiliary data source , 2005, ICML.

[57]  Terrence J. Sejnowski,et al.  Learning Overcomplete Representations , 2000, Neural Computation.

[58]  Daniel L. Rubin,et al.  Transfer learning on fused multiparametric MR images for classifying histopathological subtypes of rhabdomyosarcoma , 2017, Comput. Medical Imaging Graph..

[59]  Francisco Herrera,et al.  Incremental learning model inspired in Rehearsal for deep convolutional networks , 2020, Knowl. Based Syst..

[60]  Jan Lundberg,et al.  Detection and identification of windmill bearing faults using a one-class support vector machine (SVM) , 2019, Measurement.

[61]  Aruna Tiwari,et al.  Localized Multiple Kernel Learning for Anomaly Detection: One-class Classification , 2018, Knowl. Based Syst..

[62]  Tong Zhang,et al.  A High-Performance Semi-Supervised Learning Method for Text Chunking , 2005, ACL.

[63]  Lei Yang,et al.  Automatic transfer learning for short text mining , 2017, EURASIP J. Wirel. Commun. Netw..

[64]  Jiawei Han,et al.  Knowledge transfer via multiple model local structure mapping , 2008, KDD.

[65]  Anton Schwaighofer,et al.  Learning Gaussian Process Kernels via Hierarchical Bayes , 2004, NIPS.

[66]  Yasuhiro Oikawa,et al.  3D sound source localization based on coherence-adjusted monopole dictionary and modified convex clustering , 2018, Applied Acoustics.

[67]  Larry S. Davis,et al.  Submodular dictionary learning for sparse coding , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[68]  Li Chen,et al.  Neighborhood-enhanced transfer learning for one-class collaborative filtering , 2019, Neurocomputing.

[69]  Stefan Winkler,et al.  Category Specific Dictionary Learning for Attribute Specific Feature Selection , 2016, IEEE Transactions on Image Processing.

[70]  Ying Sun,et al.  An unsupervised monitoring procedure for detecting anomalies in photovoltaic systems using a one-class Support Vector Machine , 2019, Solar Energy.

[71]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[72]  Rama Chellappa,et al.  Sparse dictionary-based representation and recognition of action attributes , 2011, 2011 International Conference on Computer Vision.

[73]  Philip S. Yu,et al.  A Framework for Clustering Uncertain Data Streams , 2008, 2008 IEEE 24th International Conference on Data Engineering.

[74]  Jianping Fan,et al.  Learning inter-related visual dictionary for object recognition , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[75]  Jian Zhang,et al.  Face recognition based on dictionary learning with the locality constraints of atoms , 2016, 2016 9th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI).

[76]  Zhongliang Jing,et al.  Discriminative Structured Dictionary Learning on Grassmann Manifolds and Its Application on Image Restoration , 2018, IEEE Transactions on Cybernetics.

[77]  Massimiliano Pontil,et al.  Regularized multi--task learning , 2004, KDD.

[78]  Chi-Keong Goh,et al.  Co-evolutionary multi-task learning for dynamic time series prediction , 2017, Appl. Soft Comput..

[79]  Jeonghan Ko,et al.  Geometric one-class classifiers using hyper-rectangles for knowledge extraction , 2019, Expert Syst. Appl..

[80]  Huaijiang Sun,et al.  Jointly discriminative projection and dictionary learning for domain adaptive collaborative representation-based classification , 2019, Pattern Recognit..

[81]  Pierre Beauseroy,et al.  Transfer learning for one class SVM adaptation to limited data distribution change , 2017, Pattern Recognit. Lett..

[82]  Yan Zhang,et al.  Sparsity-based inverse halftoning via semi-coupled multi-dictionary learning and structural clustering , 2018, Eng. Appl. Artif. Intell..

[83]  Larry S. Davis,et al.  Label Consistent K-SVD: Learning a Discriminative Dictionary for Recognition , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[84]  Vishal M. Patel,et al.  One-Class Convolutional Neural Network , 2019, IEEE Signal Processing Letters.

[85]  Philip S. Yu,et al.  Text classification without negative examples revisit , 2006, IEEE Transactions on Knowledge and Data Engineering.

[86]  Baoxin Li,et al.  Discriminative K-SVD for dictionary learning in face recognition , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[87]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[88]  Qun Dai,et al.  A hybrid transfer learning algorithm incorporating TrSVM with GASEN , 2019, Pattern Recognit..

[89]  Qiang Yang,et al.  Transferring Localization Models over Time , 2008, AAAI.

[90]  Tommy W. S. Chow,et al.  Sparse Codes Auto-Extractor for Classification: A Joint Embedding and Dictionary Learning Framework for Representation , 2016, IEEE Transactions on Signal Processing.

[91]  Qiang Yang,et al.  Transferring Naive Bayes Classifiers for Text Classification , 2007, AAAI.

[92]  Qiang Yang,et al.  Boosting for transfer learning , 2007, ICML '07.

[93]  Bo Liu,et al.  Transfer learning-based one-class dictionary learning for recommendation data stream , 2021, Inf. Sci..

[94]  Edwin V. Bonilla,et al.  Multi-task Gaussian Process Prediction , 2007, NIPS.

[95]  Jianping Fan,et al.  HD-MTL: Hierarchical Deep Multi-Task Learning for Large-Scale Visual Recognition , 2017, IEEE Transactions on Image Processing.

[96]  Qiang Yang,et al.  Co-clustering based classification for out-of-domain documents , 2007, KDD '07.

[97]  Raymond J. Mooney,et al.  Mapping and Revising Markov Logic Networks for Transfer Learning , 2007, AAAI.

[98]  Svetha Venkatesh,et al.  A flexible transfer learning framework for Bayesian optimization with convergence guarantee , 2019, Expert Syst. Appl..

[99]  Aruna Tiwari,et al.  On the construction of extreme learning machine for online and offline one-class classification - An expanded toolbox , 2017, Neurocomputing.

[100]  Steffen Bickel,et al.  Discriminative learning for differing training and test distributions , 2007, ICML '07.

[101]  Shukui Bo,et al.  One-class classification based river detection in remote sensing image , 2017, 2017 10th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI).

[102]  Massimiliano Pontil,et al.  Convex multi-task feature learning , 2008, Machine Learning.

[103]  Ran Liu,et al.  An Ensemble of Classifiers Based on Positive and Unlabeled Data in One-Class Remote Sensing Classification , 2018, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[104]  Jungong Han,et al.  Class-specific synthesized dictionary model for Zero-Shot Learning , 2019, Neurocomputing.

[105]  Philip S. Yu,et al.  An improved categorization of classifier's sensitivity on sample selection bias , 2005, Fifth IEEE International Conference on Data Mining (ICDM'05).

[106]  Daphne Koller,et al.  Learning a meta-level prior for feature relevance from multiple related tasks , 2007, ICML '07.

[107]  Sridhar Mahadevan,et al.  Manifold alignment using Procrustes analysis , 2008, ICML '08.

[108]  Hong-Jie Xing,et al.  Robust one-class support vector machine with rescaled hinge loss function , 2018, Pattern Recognit..

[109]  Shuicheng Yan,et al.  Jointly Learning Structured Analysis Discriminative Dictionary and Analysis Multiclass Classifier , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[110]  Neil D. Lawrence,et al.  Learning to learn with the informative vector machine , 2004, ICML.

[111]  Bernhard Schölkopf,et al.  Correcting Sample Selection Bias by Unlabeled Data , 2006, NIPS.

[112]  Xiangyang Xue,et al.  Multiple Task Learning Using Iteratively Reweighted Least Square , 2013, IJCAI.

[113]  Han-Joon Kim,et al.  GAN-Based One-Class Classification for Personalized Image Retrieval , 2018, 2018 IEEE International Conference on Big Data and Smart Computing (BigComp).

[114]  Kate Saenko,et al.  Return of Frustratingly Easy Domain Adaptation , 2015, AAAI.