A multi-task-based classification framework for multi-instance distance metric learning

Abstract In traditional multiple-instance learning (MIL), the Euclidean distance is used to measure the distance of data. Different from traditional MIL, multi-instance distance metric learning (MIDM) is proposed by learning an appropriate distance metric for multi-instance data, which has been demonstrated to improve the MIL performance. However, most of the existing work considers MIDM as a single-task learning problem, and focuses on single-task MIMD. The multi-task MIMD has not been explicitly addressed. In real-world MIDM applications, the amount of labeled training data may be scarce. If we train a MIDM classifier by using only a scarce amount of labeled data, the performance of the learnt MIDM classifier may be limited. Instead of learning each task independently, learning these related tasks simultaneously can explicitly improve the classification performance. In this paper, we propose a novel multi-task-based classification framework for MIDM (MT-MIDM), which is capable of constructing a more accurate classifier on each MIDM task by learning multiple tasks simultaneously and incorporating the classification information shared among the tasks into boosting the classification accuracy. Extensive experiments have showed that the proposed MT-MIDM method outperforms the single-task MIDM methods.

[1]  Jiayu Zhou,et al.  Online Multi-Task Learning Framework for Ensemble Forecasting , 2017, IEEE Transactions on Knowledge and Data Engineering.

[2]  Kilian Q. Weinberger,et al.  Distance Metric Learning for Large Margin Nearest Neighbor Classification , 2005, NIPS.

[3]  Dacheng Tao,et al.  Multi-task proximal support vector machine , 2015, Pattern Recognit..

[4]  Yann LeCun,et al.  Learning a similarity metric discriminatively, with application to face verification , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[5]  Jason Weston,et al.  Support vector machines for multi-class pattern recognition , 1999, ESANN.

[6]  Qiang Yang,et al.  Co-clustering based classification for out-of-domain documents , 2007, KDD '07.

[7]  Rich Caruana,et al.  Multitask Learning , 1997, Machine Learning.

[8]  Horst Bischof,et al.  Large scale metric learning from equivalence constraints , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Xianglong Tang,et al.  Combining example selection with instance selection to speed up multiple-instance learning , 2014, Neurocomputing.

[10]  Ye Xu,et al.  Multi-instance Metric Learning , 2011, 2011 IEEE 11th International Conference on Data Mining.

[11]  Qingyao Wu,et al.  Multi-instance multi-label distance metric learning for genome-wide protein function prediction , 2016, Comput. Biol. Chem..

[12]  Min Yang,et al.  Robust object tracking via online multiple instance metric learning , 2013, 2013 IEEE International Conference on Multimedia and Expo Workshops (ICMEW).

[13]  Thomas G. Dietterich,et al.  Solving the Multiple Instance Problem with Axis-Parallel Rectangles , 1997, Artif. Intell..

[14]  Jianping Fan,et al.  Multi-taskmulti-labelmultiple instance learning , 2010, Journal of Zhejiang University SCIENCE C.

[15]  Haifeng Zhao,et al.  Multiple instance learning via distance metric optimization , 2013, 2013 IEEE International Conference on Image Processing.

[16]  Chih-Jen Lin,et al.  A comparison of methods for multiclass support vector machines , 2002, IEEE Trans. Neural Networks.

[17]  Zhi-Hua Zhou,et al.  On the relation between multi-instance learning and semi-supervised learning , 2007, ICML '07.

[18]  Shiliang Sun,et al.  Multitask centroid twin support vector machines , 2015, Neurocomputing.

[19]  Hui Cheng,et al.  Multi-instance learning based on representative instance and feature mapping , 2016, Neurocomputing.

[20]  Geoffrey E. Hinton,et al.  Neighbourhood Components Analysis , 2004, NIPS.

[21]  Sally A. Goldman,et al.  MISSL: multiple-instance semi-supervised learning , 2006, ICML.

[22]  Charles A. Micchelli,et al.  Learning Multiple Tasks with Kernel Methods , 2005, J. Mach. Learn. Res..

[23]  Thomas Hofmann,et al.  Support Vector Machines for Multiple-Instance Learning , 2002, NIPS.

[24]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[25]  Jiawei Han,et al.  Knowledge transfer via multiple model local structure mapping , 2008, KDD.

[26]  Mark Craven,et al.  Multiple-Instance Active Learning , 2007, NIPS.

[27]  Dacheng Tao,et al.  Person Re-Identification Over Camera Networks Using Multi-Task Distance Metric Learning , 2014, IEEE Transactions on Image Processing.

[28]  Moustafa Ghanem,et al.  A novel refinement approach for text categorization , 2005, CIKM '05.

[29]  N. V. Vinodchandran,et al.  SVM-based generalized multiple-instance learning via approximate box counting , 2004, ICML.

[30]  José Ragot,et al.  Multi-task learning with one-class SVM , 2014, Neurocomputing.

[31]  Yeong-Yuh Xu,et al.  Multiple-instance learning based decision neural networks for image retrieval and classification , 2016, Neurocomputing.

[32]  Chandan K. Reddy,et al.  Multi-Task Clustering using Constrained Symmetric Non-Negative Matrix Factorization , 2014, SDM.

[33]  Shuiwang Ji,et al.  Deep Convolutional Neural Networks for Multi-instance Multi-task Learning , 2015, 2015 IEEE International Conference on Data Mining.

[34]  Yixin Chen,et al.  MILES: Multiple-Instance Learning via Embedded Instance Selection , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Wei Luo,et al.  Toxicity Prediction in Cancer Using Multiple Instance Learning in a Multi-task Framework , 2016, PAKDD.

[36]  Min-Ling Zhang,et al.  MIMLRBF: RBF neural networks for multi-instance multi-label learning , 2009, Neurocomputing.

[37]  Massimiliano Pontil,et al.  Regularized multi--task learning , 2004, KDD.

[38]  Mark Craven,et al.  Supervised versus multiple instance learning: an empirical comparison , 2005, ICML.

[39]  David Zhang,et al.  A Kernel Classification Framework for Metric Learning , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[40]  Carmen Banea,et al.  Random-Walk Term Weighting for Improved Text Classification , 2006 .

[41]  Zhi-Hua Zhou,et al.  Learning a distance metric from multi-instance multi-label data , 2009, CVPR.

[42]  Cordelia Schmid,et al.  Multiple Instance Metric Learning from Automatically Labeled Bags of Faces , 2010, ECCV.

[43]  Kilian Q. Weinberger,et al.  Large Margin Multi-Task Metric Learning , 2010, NIPS.

[44]  Qiang Yang,et al.  Boosting for transfer learning , 2007, ICML '07.

[45]  Peng Li,et al.  Distance Metric Learning with Eigenvalue Optimization , 2012, J. Mach. Learn. Res..

[46]  Shiliang Sun,et al.  Multitask multiclass support vector machines: Model and experiments , 2013, Pattern Recognit..

[47]  Jianping Yin,et al.  MI-ELM: Highly efficient multi-instance learning based on hierarchical extreme learning machine , 2016, Neurocomputing.

[48]  Zheru Chi,et al.  Multi-instance multi-label image classification: A neural approach , 2013, Neurocomputing.

[49]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[50]  Jian-ping,et al.  Multi-task multi-label multiple instance learning , 2010 .

[51]  Wei Fan,et al.  Actively Transfer Domain Knowledge , 2008, ECML/PKDD.

[52]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[53]  Feiping Nie,et al.  Maximum Margin Multi-Instance Learning , 2011, NIPS.

[54]  Yoram Singer,et al.  Pegasos: primal estimated sub-gradient solver for SVM , 2011, Math. Program..