Instance-Based Classification Methods

Instance-based classification algorithms perform their main learning process at the instance level. They try to approximate a function that assigns class labels to instances. The instance classifier is combined with an underlying MI assumption, which links the class label of instances inside a bag with the bag class label. Many strategies have been devised to construct the instance classifier. We discuss the most prominent of them: wrapper methods (Sect. 4.2), maximum likelihood methods (Sect. 4.3), decision trees and rules methods (Sect. 4.4), maximum margin methods (Sect. 4.5), connectionist methods (Sect. 4.6), and evolutionary methods (Sect. 4.7). An experimental analysis on the performance of representative instance-based classifiers is presented in Sect. 4.8. Summarizing remarks are given in Sect. 4.9.

[1]  Zhi-Hua Zhou,et al.  Neural Networks for Multi-Instance Learning , 2002 .

[2]  David D. Jensen,et al.  Identifying Predictive Structures in Relational Data Using Multiple Instance Learning , 2003, ICML.

[3]  Jing Hua,et al.  Region-based Image Annotation using Asymmetrical Support Vector Machine-based Multiple-Instance Learning , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[4]  Tomás Lozano-Pérez,et al.  A Framework for Multiple-Instance Learning , 1997, NIPS.

[5]  Sebastián Ventura,et al.  Multi-objective approach based on grammar-guided genetic programming for solving multiple instance problems , 2012, Soft Comput..

[6]  Yann Chevaleyre,et al.  Solving Multiple-Instance and Multiple-Part Learning Problems with Decision Trees and Rule Sets. Application to the Mutagenesis Problem , 2001, Canadian Conference on AI.

[7]  J. Ross Quinlan,et al.  Induction of Decision Trees , 1986, Machine Learning.

[8]  Yoram Singer,et al.  Improved Boosting Algorithms Using Confidence-rated Predictions , 1998, COLT' 98.

[9]  Yuan Xie,et al.  Online multiple instance gradient feature selection for robust visual tracking , 2012, Pattern Recognit. Lett..

[10]  Zhi-Hua Zhou,et al.  Improve Multi-Instance Neural Networks through Feature Selection , 2004, Neural Processing Letters.

[11]  Horst Bischof,et al.  Real-Time Tracking via On-line Boosting , 2006, BMVC.

[12]  Zhi-Hua Zhou,et al.  Adapting RBF Neural Networks to Multi-Instance Learning , 2006, Neural Processing Letters.

[13]  Zhiquan Qi,et al.  Online multiple instance boosting for object detection , 2011, Neurocomputing.

[14]  Nikunj C. Oza,et al.  Online Ensemble Learning , 2000, AAAI/IAAI.

[15]  Donald E. Knuth,et al.  backus normal form vs. Backus Naur form , 1964, CACM.

[16]  Andrzej Jaszkiewicz,et al.  Genetic local search for multi-objective combinatorial optimization , 2022 .

[17]  Oded Maron,et al.  Learning from Ambiguity , 1998 .

[18]  Julie Wilson,et al.  Novel feature selection method for genetic programming using metabolomic 1H NMR data , 2006 .

[19]  Peter L. Bartlett,et al.  Boosting Algorithms as Gradient Descent , 1999, NIPS.

[20]  Alexandros Agapitos,et al.  GP made faster with semantic surrogate modelling , 2016, Inf. Sci..

[21]  Ashwin Srinivasan,et al.  Multi-instance tree learning , 2005, ICML.

[22]  George D. Smith,et al.  Evolutionary constructive induction , 2005, IEEE Transactions on Knowledge and Data Engineering.

[23]  Cheng Hua Li,et al.  A Novel Neural Network-Based Approach for Multiple Instance Learning , 2010, 2010 10th IEEE International Conference on Computer and Information Technology.

[24]  Hendrik Blockeel,et al.  Classifying Relational Data with Neural Networks , 2005, ILP.

[25]  Marco Laumanns,et al.  SPEA2: Improving the strength pareto evolutionary algorithm , 2001 .

[26]  Hsin-Chia Fu,et al.  An EM based multiple instance learning method for image classification , 2008, Expert Syst. Appl..

[27]  I. Jolliffe.  Principal Component Analysis , 2002 .

[28]  Thomas Hofmann,et al.  Support Vector Machines for Multiple-Instance Learning , 2002, NIPS.

[29]  Zhi-Hua Zhou,et al.  Multi-Instance Learning Based Web Mining , 2005, Applied Intelligence.

[30]  Thomas E. McKee,et al.  Bankruptcy theory development and classification via genetic programming , 2006, Eur. J. Oper. Res..

[31]  Horst Bischof,et al.  Inverse Multiple Instance Learning for Classifier Grids , 2010, 2010 20th International Conference on Pattern Recognition.

[32]  Sebastián Ventura,et al.  Predicting Student Grades in Learning Management Systems with Multiple Instance Genetic Programming. , 2009, EDM 2009.

[33]  Hendrik Blockeel,et al.  A Comparison between Neural Network Methods for Learning Aggregate Functions , 2008, Discovery Science.

[34]  Hsin-Chia Fu,et al.  Neural Network Based Image Retrieval with Multiple Instance Leaning Techniques , 2005, KES.

[35]  Philip S. Yu,et al.  Top 10 algorithms in data mining , 2007, Knowledge and Information Systems.

[36]  Paul A. Viola,et al.  Multiple Instance Boosting for Object Detection , 2005, NIPS.

[37]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[38]  Thomas Hofmann,et al.  Kernel Methods for Missing Variables , 2005, AISTATS.

[39]  Tzung-Pei Hong,et al.  Learning discriminant functions with fuzzy attributes for classification using genetic programming , 2002, Expert systems with applications.

[40]  Yann Chevaleyre,et al.  Learning Rules from Multiple Instance Data: Issues and Algorithms , 2001 .

[41]  Qi Zhang,et al.  EM-DD: An Improved Multiple-Instance Learning Technique , 2001, NIPS.

[42]  Qingling Li,et al.  Visual tracking based on multiple instance learning particle filter , 2011, 2011 IEEE International Conference on Mechatronics and Automation.

[43]  K. Rose,et al.  Deterministic annealing, constrained clustering, and optimization , 1991, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks.

[44]  Tao Xu,et al.  A multiple instance learning based framework for semantic image segmentation , 2010, Multimedia Tools and Applications.

[45]  Mark Craven,et al.  Supervised versus multiple instance learning: an empirical comparison , 2005, ICML.

[46]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[47]  Athanasios Tsakonas,et al.  A comparison of classification accuracy of four genetic programming-evolved intelligent structures , 2006, Inf. Sci..

[48]  Lijun Liu,et al.  An efficient parallel neural network-based multi-instance learning algorithm , 2012, The Journal of Supercomputing.

[49]  David E. Goldberg,et al.  Zen and the Art of Genetic Algorithms , 1989, ICGA.

[50]  Zhi-Hua Zhou,et al.  On the relation between multi-instance learning and semi-supervised learning , 2007, ICML '07.

[51]  Artur S. d'Avila Garcez,et al.  Multi-instance learning using recurrent neural networks , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[52]  Zhouyu Fu,et al.  Fast multiple instance learning via L1,2 logistic regression , 2008, 2008 19th International Conference on Pattern Recognition.

[53]  Razvan C. Bunescu,et al.  Multiple instance learning for sparse positive bags , 2007, ICML '07.

[54]  Xin Xu,et al.  Statistical Learning in Multiple Instance Problems , 2003 .

[55]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[56]  Ming-Hsuan Yang,et al.  Visual tracking with online Multiple Instance Learning , 2009, CVPR.

[57]  Elena Deza,et al.  Dictionary of distances , 2006 .

[58]  Eibe Frank,et al.  Beyond Trees: Adopting MITI to Learn Rules and Ensemble Classifiers for Multi-Instance Data , 2011, Australasian Conference on Artificial Intelligence.

[59]  Horst Bischof,et al.  MIForests: Multiple-Instance Learning with Randomized Trees , 2010, ECCV.

[60]  William W. Cohen.  Fast Effective Rule Induction , 1995, ICML.

[61]  J. Friedman.  Special Invited Paper-Additive logistic regression: A statistical view of boosting , 2000 .

[62]  Thomas Bräunl,et al.  Dynamic population variation in genetic programming , 2009, Inf. Sci..

[63]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[64]  James T. Kwok,et al.  A regularization framework for multiple-instance learning , 2006, ICML.

[65]  Lawrence K. Saul,et al.  A statistical model for robust integration of narrowband cues in speech , 2001, Comput. Speech Lang..

[66]  Abdelhamid Bouchachia.  Multiple Instance Learning with Radial Basis Function Neural Networks , 2004, ICONIP.

[67]  De Xu,et al.  Transductive Multi-Instance Multi-Label learning algorithm with application to automatic image annotation , 2010, Expert Syst. Appl..

[68]  Sally A. Goldman,et al.  Multiple-Instance Learning of Real-Valued Data , 2001, J. Mach. Learn. Res..

[69]  Chi-Huang Shih,et al.  Multiple-Instance Learning via Decision-Based Neural Networks , 2011 .

[70]  Jan Ramon,et al.  Multi instance neural networks , 2000, ICML 2000.

[71]  Xin Xu,et al.  Logistic Regression and Boosting for Labeled Bags of Instances , 2004, PAKDD.

[72]  Sebastián Ventura,et al.  Speeding up multiple instance learning classification rules on GPUs , 2015, Knowledge and Information Systems.

[73]  Joel H. Saltz,et al.  Efficient Multiple Instance Convolutional Neural Networks for Gigapixel Resolution Image Classification , 2015, ArXiv.

[74]  Enrique Herrera-Viedma,et al.  Multi-instance genetic programming for web index recommendation , 2009, Expert Syst. Appl..

[75]  Peter A. Whigham,et al.  Grammar-based Genetic Programming: a survey , 2010, Genetic Programming and Evolvable Machines.

[76]  Lin Dong,et al.  A Comparison of Multi-instance Learning Algorithms , 2006 .

[77]  Horst Bischof,et al.  On-line semi-supervised multiple-instance boosting , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[78]  Sebastián Ventura,et al.  G3P-MI: A genetic programming algorithm for multiple instance learning , 2010, Inf. Sci..

[79]  Zhang Min.  A Multi-Instance Regression Algorithm Based on Neural Network , 2003 .

[80]  Lalit M. Patnaik,et al.  Application of genetic programming for multicategory pattern classification , 2000, IEEE Trans. Evol. Comput..

[81]  Eibe Frank,et al.  Applying propositional learning algorithms to multi-instance data , 2003 .

[82]  Aapo Hyvärinen,et al.  The Fixed-Point Algorithm and Maximum Likelihood Estimation for Independent Component Analysis , 1999, Neural Processing Letters.

[83]  Yixin Chen,et al.  Image Categorization by Learning and Reasoning with Regions , 2004, J. Mach. Learn. Res..