G3P-MI: A genetic programming algorithm for multiple instance learning

This paper introduces a new Grammar-Guided Genetic Programming algorithm for resolving multi-instance learning problems. This algorithm, called G3P-MI, is evaluated and compared to other multi-instance classification techniques in different application domains. Computational experiments show that the G3P-MI often obtains consistently better results than other algorithms in terms of accuracy, sensitivity and specificity. Moreover, it makes the knowledge discovery process clearer and more comprehensible, by expressing information in the form of IF-THEN rules. Our results confirm that evolutionary algorithms are very appropriate for dealing with multi-instance learning problems.

[1]  Mengjie Zhang,et al.  Using Gaussian distribution to construct fitness functions in genetic programming for multiclass object classification , 2006, Pattern Recognit. Lett..

[2]  Qi Zhang,et al.  EM-DD: An Improved Multiple-Instance Learning Technique , 2001, NIPS.

[3]  Julie Wilson,et al.  Novel feature selection method for genetic programming using metabolomic 1H NMR data , 2006 .

[4]  Adam Tauman Kalai,et al.  A Note on Learning from Multiple-Instance Examples , 2004, Machine Learning.

[5]  John C. Platt,et al.  Fast training of support vector machines using sequential minimal optimization, advances in kernel methods , 1999 .

[6]  Jean-Gabriel Ganascia,et al.  Representation Changes for Efficient Learning in Structural Domains , 1996, ICML.

[7]  Mark Craven,et al.  Supervised versus multiple instance learning: an empirical comparison , 2005, ICML.

[8]  Vic Ciesielski,et al.  Texture classifiers generated by genetic programming , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[9]  Yoav Freund,et al.  Experiments with a New Boosting Algorithm , 1996, ICML.

[10]  Ashwin Srinivasan,et al.  Theories for Mutagenicity: A Study in First-Order and Feature-Based Induction , 1996, Artif. Intell..

[11]  De Xu,et al.  Transductive Multi-Instance Multi-Label learning algorithm with application to automatic image annotation , 2010, Expert Syst. Appl..

[12]  Yixin Chen,et al.  MILES: Multiple-Instance Learning via Embedded Instance Selection , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Sally A. Goldman,et al.  Multiple-Instance Learning of Real-Valued Data , 2001, J. Mach. Learn. Res..

[14]  Athanasios Tsakonas,et al.  A comparison of classification accuracy of four genetic programming-evolved intelligent structures , 2006, Inf. Sci..

[15]  Yann Chevaleyre,et al.  Solving Multiple-Instance and Multiple-Part Learning Problems with Decision Trees and Rule Sets. Application to the Mutagenesis Problem , 2001, Canadian Conference on AI.

[16]  David Page,et al.  Multiple Instance Regression , 2001, ICML.

[17]  Peter A. Flach,et al.  An extended transformation approach to inductive logic programming , 2001, ACM Trans. Comput. Log..

[18]  Peter A. Whigham,et al.  Grammatically-based Genetic Programming , 1995 .

[19]  Nikhil R. Pal,et al.  A novel approach to design classifiers using genetic programming , 2004, IEEE Transactions on Evolutionary Computation.

[20]  Stuart Harvey Rubin,et al.  A Human-Centered Multiple Instance Learning Framework for Semantic Video Retrieval , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[21]  Jun Wang,et al.  Solving the Multiple-Instance Problem: A Lazy Learning Approach , 2000, ICML.

[22]  Thomas Gärtner,et al.  Multi-Instance Kernels , 2002, ICML.

[23]  Peter Auer,et al.  On Learning From Multi-Instance Examples: Empirical Evaluation of a Theoretical Approach , 1997, ICML.

[24]  Zhi-Hua Zhou,et al.  Adapting RBF Neural Networks to Multi-Instance Learning , 2006, Neural Processing Letters.

[25]  Xin Xu,et al.  Statistical Learning in Multiple Instance Problems , 2003 .

[26]  Tomás Lozano-Pérez,et al.  A Framework for Multiple-Instance Learning , 1997, NIPS.

[27]  Trevor S. Wiens,et al.  Three way k-fold cross-validation of resource selection functions , 2008 .

[28]  Thomas Hofmann,et al.  Support Vector Machines for Multiple-Instance Learning , 2002, NIPS.

[29]  Jeffrey Horn,et al.  Handbook of evolutionary computation , 1997 .

[30]  Stan Matwin,et al.  Filtering Multi-Instance Problems to Reduce Dimensionality in Relational Learning , 2004, Journal of Intelligent Information Systems.

[31]  Nader Vadiee.  Fuzzy rule-based expert systems II , 1993 .

[32]  Bernhard Pfahringer,et al.  A Two-Level Learning Method for Generalized Multi-instance Problems , 2003, ECML.

[33]  Ron Kohavi,et al.  A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection , 1995, IJCAI.

[34]  Lalit M. Patnaik,et al.  Application of genetic programming for multicategory pattern classification , 2000, IEEE Trans. Evol. Comput..

[35]  Jun Zhang,et al.  On Generalized Multiple-instance Learning , 2005, Int. J. Comput. Intell. Appl..

[36]  Edward W. Wild,et al.  Multiple Instance Classification via Successive Linear Programming , 2008 .

[37]  Zhi-Hua Zhou,et al.  Improve Multi-Instance Neural Networks through Feature Selection , 2004, Neural Processing Letters.

[38]  Enrique Herrera-Viedma,et al.  Multi-instance genetic programming for web index recommendation , 2009, Expert Syst. Appl..

[39]  Thomas Gärtner,et al.  Kernels and Distances for Structured Data , 2004, Machine Learning.

[40]  Walter Böhm,et al.  Exact Uniform Initialization For Genetic Programming , 1996, FOGA.

[41]  Tzung-Pei Hong,et al.  Learning discriminant functions with fuzzy attributes for classification using genetic programming , 2002, Expert systems with applications.

[42]  Yann Chevaleyre,et al.  Solving multiple-instance and multiple-part learning problems with decision trees and decision rules . Application to the mutagenesis problem , 2000 .

[43]  Sean Luke,et al.  A survey and comparison of tree generation algorithms , 2001 .

[44]  Tomás Lozano-Pérez,et al.  Image database retrieval with multiple-instance learning techniques , 2000, Proceedings of 16th International Conference on Data Engineering (Cat. No.00CB37073).

[45]  Jan Ramon,et al.  Multi instance neural networks , 2000, ICML 2000.

[46]  B. C. Brookes,et al.  Information Sciences , 2020, Cognitive Skills You Need for the 21st Century.

[47]  Zhi-Hua Zhou,et al.  Ensembles of Multi-Instance Neural Networks , 2004, Intelligent Information Processing.

[48]  Xin Xu,et al.  Logistic Regression and Boosting for Labeled Bags of Instances , 2004, PAKDD.

[49]  Zhi-Hua Zhou,et al.  Multi-Instance Learning from Supervised View , 2006, Journal of Computer Science and Technology.

[50]  O. J. Dunn.  Multiple Comparisons among Means , 1961 .

[51]  Luc De Raedt,et al.  Attribute-Value Learning Versus Inductive Logic Programming: The Missing Links (Extended Abstract) , 1998, ILP.

[52]  Celia C. Bojarczuk,et al.  Genetic programming for knowledge discovery in chest-pain diagnosis. , 2000, IEEE engineering in medicine and biology magazine : the quarterly magazine of the Engineering in Medicine & Biology Society.

[53]  Aravind Srinivasan,et al.  Approximating hyper-rectangles: learning and pseudo-random sets , 1997, STOC '97.

[54]  Ryszard S. Michalski,et al.  Inductive inference of VL decision rules , 1977, SGAR.

[55]  Peter A. Whigham,et al.  Grammatical bias for evolutionary learning , 1996 .

[56]  Yann Chevaleyre,et al.  Learning Rules from Multiple Instance Data: Issues and Algorithms , 2001 .

[57]  Thomas G. Dietterich,et al.  Solving the Multiple Instance Problem with Axis-Parallel Rectangles , 1997, Artif. Intell..

[58]  Zhi-Hua Zhou,et al.  On the relation between multi-instance learning and semi-supervised learning , 2007, ICML '07.

[59]  Tao Mei,et al.  MILC2: A Multi-Layer Multi-Instance Learning Approach to Video Concept Detection , 2008, MMM.

[60]  Nachol Chaiyaratana,et al.  Thalassaemia classification by neural networks and genetic programming , 2007, Inf. Sci..

[61]  Peter Auer,et al.  A Boosting Approach to Multiple Instance Learning , 2004, ECML.

[62]  Hsin-Chia Fu,et al.  An EM based multiple instance learning method for image classification , 2008, Expert Syst. Appl..

[63]  Thomas E. McKee,et al.  Bankruptcy theory development and classification via genetic programming , 2006, Eur. J. Oper. Res..

[64]  N. V. Vinodchandran,et al.  SVM-based generalized multiple-instance learning via approximate box counting , 2004, ICML.

[65]  Zhi-Hua Zhou,et al.  Neural Networks for Multi-Instance Learning , 2002 .

[66]  Giancarlo Ruffo,et al.  Learning single and multiple instance decision tree for computer security applications , 2000 .

[67]  Andreas Geyer-Schulz,et al.  Fuzzy Rule-Based Expert Systems and Genetic Machine Learning , 1996 .

[68]  Thomas Bräunl,et al.  Dynamic population variation in genetic programming , 2009, Inf. Sci..

[69]  Zhi-Hua Zhou,et al.  Solving multi-instance problems with classifier ensemble based on constructive clustering , 2007, Knowledge and Information Systems.

[70]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[71]  Kumar Chellapilla,et al.  Evolving computer programs without subtree crossover , 1997, IEEE Trans. Evol. Comput..

[72]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[73]  Farshad Fotouhi,et al.  Region based image annotation through multiple-instance learning , 2005, MULTIMEDIA '05.

[74]  Ian Witten,et al.  Data Mining , 2000 .

[75]  Philip M. Long,et al.  PAC Learning Axis-aligned Rectangles with Respect to Product Distributions from Multiple-Instance Examples , 1996, COLT '96.

[76]  César Hervás-Martínez,et al.  JCLEC: a Java framework for evolutionary computation , 2007, Soft Comput..

[77]  Zhi-Hua Zhou,et al.  Multi-Instance Learning Based Web Mining , 2005, Applied Intelligence.

[78]  Yu-Mei Chai,et al.  A Multi-Instance Learning Algorithm Based on Normalized Radial Basis Function Network , 2007, ISNN.

[79]  Fei Wang,et al.  Interactive localized content based image retrieval with multiple-instance active learning , 2010, Pattern Recognit..

[80]  Arthur Tay,et al.  Mining multiple comprehensible classification rules using genetic programming , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[81]  Eibe Frank,et al.  Applying propositional learning algorithms to multi-instance data , 2003 .

[82]  Yixin Chen,et al.  Image Categorization by Learning and Reasoning with Regions , 2004, J. Mach. Learn. Res..