Lifelong multi-task multi-view learning using latent spaces

In this paper, we study the problem of MTMV learning in a lifelong learning framework. Lifelong machine learning, like human lifelong learning, learns multiple tasks over time. Lifelong multi-task multi-view (Lifelong MTMV) learning is a new data mining and machine learning problem where new tasks and/or new views may come in anytime during the learning process. Our goal is to efficiently learn a model for a new task or new view by selectively transferring knowledge learned from previous tasks or views. To this end, we propose a latent space lifelong MTMV (lslMTMV) learning method to exploit task relatedness and information from multiple views. In this new method, we map views to a shared latent space and then learn a decision function in the latent space. Our new method supports knowledge sharing among multiple views and knowledge transfer from existing tasks to a new learning task naturally. We have evaluated our method using 3 real-world data sets. The experimental study results demonstrate that the classification accuracy of our algorithm is close or superior to state-of-the-art offline MTMV learning algorithms while the time needed to train such models is orders of magnitude less.

[1]  Bing Liu,et al.  Topic Modeling using Topics from Many Domains, Lifelong Learning and Big Data , 2014, ICML.

[2]  Jingrui He,et al.  A Graphbased Framework for Multi-Task Multi-View Learning , 2011, ICML.

[3]  Trevor Darrell,et al.  Factorized Latent Spaces with Structured Sparsity , 2010, NIPS.

[4]  Bing Liu,et al.  Mining Aspect-Specific Opinion using a Holistic Lifelong Topic Model , 2016, WWW.

[5]  Kjersti Engan,et al.  Recursive Least Squares Dictionary Learning Algorithm , 2010, IEEE Transactions on Signal Processing.

[6]  D. Silver,et al.  Continued Practice and Consolidation of a Learning Task , 2005 .

[7]  Marc'Aurelio Ranzato,et al.  Building high-level features using large scale unsupervised learning , 2011, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[8]  Qiang Yang,et al.  Lifelong Machine Learning Systems: Beyond Learning Algorithms , 2013, AAAI Spring Symposium: Lifelong Machine Learning.

[9]  Hal Daumé,et al.  Learning Task Grouping and Overlap in Multi-task Learning , 2012, ICML.

[10]  Thomas C. Wiegers,et al.  The Comparative Toxicogenomics Database: update 2017 , 2016, Nucleic Acids Res..

[11]  Fuzhen Zhuang,et al.  Shared Structure Learning for Multiple Tasks with Multiple Views , 2013, ECML/PKDD.

[12]  Luming Zhang,et al.  Interest Inference via Structure-Constrained Multi-Source Multi-Task Learning , 2015, IJCAI.

[13]  Jintao Zhang,et al.  Inductive multi-task learning with multiple view data , 2012, KDD.

[14]  Eric Eaton,et al.  Online Multi-Task Learning for Policy Gradient Methods , 2014, ICML.

[15]  Daniel L. Silver,et al.  Consolidation Using Context-Sensitive Multiple Task Learning , 2011, Canadian Conference on AI.

[16]  Hui Xiong,et al.  Multi-task Multi-view Learning for Heterogeneous Tasks , 2014, CIKM.

[17]  Bing Liu,et al.  Lifelong Learning for Sentiment Classification , 2015, ACL.

[18]  Eric Eaton,et al.  ELLA: An Efficient Lifelong Learning Algorithm , 2013, ICML.

[19]  Yi Yang,et al.  Beyond Doctors: Future Health Prediction from Multimedia and Multimodal Observations , 2015, ACM Multimedia.

[20]  Eric Eaton,et al.  Active Task Selection for Lifelong Machine Learning , 2013, AAAI.

[21]  Jingrui He,et al.  Learning with dual heterogeneity: a nonparametric bayes model , 2014, KDD.