Multiple instance learning via margin maximization

In this paper, we consider the classification problem within the multiple instance learning (MIL) context. Training data is composed of labeled bags of instances. Despite the large number of margin maximization based classification methods, there are only a few methods that consider the margin for MIL problems in the literature. We first formulate a combinatorial margin maximization problem for multiple instance classification and prove that it is NP-hard. We present a way to apply the kernel trick in this formulation for classifying nonlinear multiple instance data. We also propose a branch and bound algorithm and present computational results on publicly available benchmark data sets. Our approach outperforms a leading commercial solver in terms of the best integer solution and optimality gap in the majority of image annotation and molecular activity prediction test cases.

[1]  Thomas G. Dietterich,et al.  Solving the Multiple Instance Problem with Axis-Parallel Rectangles , 1997, Artif. Intell..

[2]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[3]  Xiaojun Qi,et al.  Incorporating multiple SVMs for automatic image annotation , 2007, Pattern Recognit..

[4]  Paul A. Viola,et al.  Multiple Instance Boosting for Object Detection , 2005, NIPS.

[5]  Hsin-Chia Fu,et al.  Neural Network Based Image Retrieval with Multiple Instance Leaning Techniques , 2005, KES.

[6]  Thomas Hofmann,et al.  Support Vector Machines for Multiple-Instance Learning , 2002, NIPS.

[7]  Thomas Hofmann,et al.  Multiple instance learning with generalized support vector machines , 2002, AAAI/IAAI.

[8]  Qi Zhang,et al.  EM-DD: An Improved Multiple-Instance Learning Technique , 2001, NIPS.

[9]  Nello Cristianini,et al.  An introduction to Support Vector Machines , 2000 .

[10]  Jun Wang,et al.  Solving the Multiple-Instance Problem: A Lazy Learning Approach , 2000, ICML.

[11]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[12]  Murat Dundar,et al.  Bayesian multiple instance learning: automatic feature selection and inductive transfer , 2008, ICML '08.

[13]  David R. Musicant,et al.  Supervised Learning by Training on Aggregate Outputs , 2007, Seventh IEEE International Conference on Data Mining (ICDM 2007).

[14]  Mark Craven,et al.  Classifying Biomedical Articles by Making Localized Decisions , 2005, TREC.

[15]  Murat Dundar,et al.  Multiple Instance Learning for Computer Aided Diagnosis , 2006, NIPS.

[16]  Panos M. Pardalos,et al.  Selective support vector machines , 2011, J. Comb. Optim..

[17]  Yixin Chen,et al.  MILES: Multiple-Instance Learning via Embedded Instance Selection , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Edward W. Wild,et al.  Multiple Instance Classification via Successive Linear Programming , 2008 .

[19]  Gustavo Carneiro,et al.  Supervised Learning of Semantic Classes for Image Annotation and Retrieval , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Olvi L. Mangasarian,et al.  Nonlinear Programming , 1969 .

[21]  G. Nemhauser,et al.  Integer Programming , 2020 .

[22]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[23]  Thomas G. Dietterich,et al.  Compass: A shape-based machine learning tool for drug design , 1994, J. Comput. Aided Mol. Des..

[24]  Yixin Chen,et al.  Image Categorization by Learning and Reasoning with Regions , 2004, J. Mach. Learn. Res..

[25]  Joseph F. Murray,et al.  Machine Learning Methods for Predicting Failures in Hard Drives: A Multiple-Instance Application , 2005, J. Mach. Learn. Res..

[26]  Sally A. Goldman,et al.  Multiple-Instance Learning of Real-Valued Data , 2001, J. Mach. Learn. Res..