Unsupervised Multi-task Learning with Hierarchical Data Structure

Abstract Unsupervised multi-task learning exploits the shared knowledge to improve performances by learning related tasks simultaneously. In this paper, we propose an unsupervised multi-task learning method with hierarchical data structure. It strengthens similarities between instances in the same cluster, and increases diversities of instances by utilizing instances from related clusters. Firstly, we introduce Representative Dual Features (RepDFs) that possess representative capabilities in the feature space and the sample space for each cluster concurrently. Secondly, we explore hierarchical structural similarities between clusters in related tasks from the topological perspective: 1) feature basis matrix, which learns compact representations for features in the feature space; and 2) sample refined matrix, which preserves local structures in the sample space. Thirdly, we adopt RepDFs to measure correlations between clusters and incorporate hierarchical structural similarities to conduct knowledge transfer among tasks. Experimental results on real-world data sets demonstrate the effectiveness and superiority of the proposed method over existing multi-task clustering methods.

[1]  Nicu Sebe,et al.  Egocentric Daily Activity Recognition via Multitask Clustering , 2015, IEEE Transactions on Image Processing.

[2]  Hongtao Lu,et al.  Multi-task co-clustering via nonnegative matrix factorization , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[3]  Younès Bennani,et al.  Entropy based probabilistic collaborative clustering , 2017, Pattern Recognit..

[4]  Deborah Chasman,et al.  Multi-task consensus clustering of genome-wide transcriptomes from related biological conditions , 2016, Bioinform..

[5]  Quanquan Gu,et al.  Learning the Shared Subspace for Multi-task Clustering and Transductive Transfer Classification , 2009, 2009 Ninth IEEE International Conference on Data Mining.

[6]  Roman Filipovych,et al.  Semi-supervised cluster analysis of imaging data , 2011, NeuroImage.

[7]  Hujun Bao,et al.  Sparse concept coding for visual analysis , 2011, CVPR 2011.

[8]  Ming Shao,et al.  Incomplete Multisource Transfer Learning , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[9]  Wei Liu,et al.  Knowledge transfer for spectral clustering , 2018, Pattern Recognit..

[10]  John C. Wooley,et al.  Ultrafast clustering algorithms for metagenomic sequence analysis , 2012, Briefings Bioinform..

[11]  Xianchao Zhang,et al.  Multi-task clustering through instances transfer , 2017, Neurocomputing.

[12]  Rich Caruana,et al.  Multitask Learning , 1997, Machine Learning.

[13]  Wei-Shi Zheng,et al.  Multi-task mid-level feature learning for micro-expression recognition , 2017, Pattern Recognit..

[14]  Sushmita Roy,et al.  A multi-task graph-clustering approach for chromosome conformation capture data sets identifies conserved modules of chromosomal interactions , 2016, Genome Biology.

[15]  Jianping Fan,et al.  Hierarchical learning of multi-task sparse metrics for large-scale image classification , 2017, Pattern Recognit..

[16]  Bernhard Schölkopf,et al.  Correcting Sample Selection Bias by Unlabeled Data , 2006, NIPS.

[17]  Maoguo Gong,et al.  Fuzzy C-Means Clustering With Local Information and Kernel Metric for Image Segmentation , 2013, IEEE Transactions on Image Processing.

[18]  Jiawei Han,et al.  Learning a Kernel for Multi-Task Clustering , 2011, AAAI.

[19]  Sebastian Nowozin,et al.  Image Segmentation UsingHigher-Order Correlation Clustering , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Ling Shao,et al.  Transfer Learning for Visual Categorization: A Survey , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[21]  Yang Yang,et al.  Multitask Spectral Clustering by Exploring Intertask Correlation , 2015, IEEE Transactions on Cybernetics.

[22]  Jianwen Zhang,et al.  Multitask Bregman clustering , 2010, Neurocomputing.

[23]  Xiao-Lei Zhang,et al.  Convex Discriminative Multitask Clustering , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Chandan K. Reddy,et al.  Multi-Task Clustering using Constrained Symmetric Non-Negative Matrix Factorization , 2014, SDM.

[25]  Jie Zhou,et al.  Multi-task clustering via domain adaptation , 2012, Pattern Recognit..

[26]  Xianchao Zhang,et al.  Smart Multi-Task Bregman Clustering and Multi-Task Kernel Clustering , 2013, AAAI.

[27]  Yangdong Ye,et al.  Multi-task Clustering of Human Actions by Sharing Information , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Thach Huy Nguyen,et al.  A feature-free and parameter-light multi-task clustering framework , 2012, Knowledge and Information Systems.

[29]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[30]  Daniel R. Figueiredo,et al.  struc2vec: Learning Node Representations from Structural Identity , 2017, KDD.

[31]  Mohamed Nadif,et al.  Multi-manifold matrix decomposition for data co-clustering , 2017, Pattern Recognit..

[32]  Peyman Adibi,et al.  Multitask fuzzy Bregman co-clustering approach for clustering data with multisource features , 2017, Neurocomputing.

[33]  Qiuqi Ruan,et al.  Multi-task clustering ELM for VIS-NIR cross-modal feature learning , 2017, Multidimens. Syst. Signal Process..

[34]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[35]  Xianchao Zhang,et al.  Self-Adapted Multi-Task Clustering , 2016, IJCAI.

[36]  Tieniu Tan,et al.  Transformation invariant subspace clustering , 2016, Pattern Recognit..

[37]  Eamonn J. Keogh,et al.  Addressing Big Data Time Series: Mining Trillions of Time Series Subsequences Under Dynamic Time Warping , 2013, TKDD.