Binary Image Classification: A Genetic Programming Approach to the Problem of Limited Training Instances

In the computer vision and pattern recognition fields, image classification represents an important yet difficult task. It is a challenge to build effective computer models to replicate the remarkable ability of the human visual system, which relies on only one or a few instances to learn a completely new class or an object of a class. Recently we proposed two genetic programming (GP) methods, one-shot GP and compound-GP, that aim to evolve a program for the task of binary classification in images. The two methods are designed to use only one or a few instances per class to evolve the model. In this study, we investigate these two methods in terms of performance, robustness, and complexity of the evolved programs. We use ten data sets that vary in difficulty to evaluate these two methods. We also compare them with two other GP and six non-GP methods. The results show that one-shot GP and compound-GP outperform or achieve results comparable to competitor methods. Moreover, the features extracted by these two methods improve the performance of other classifiers with handcrafted features and those extracted by a recently developed GP-based method in most cases.

[1]  Bo Yang,et al.  A comparative study on local binary pattern (LBP) based face recognition: LBP histogram versus LBP image , 2013, Neurocomputing.

[2]  Mengjie Zhang,et al.  Classification Strategies for Image Classification in Genetic Programming , 2003 .

[3]  Victor Ciesielski,et al.  A Domain-Independent Window Approach to Multiclass Object Detection Using Genetic Programming , 2003, EURASIP J. Adv. Signal Process..

[4]  S. Salzberg,et al.  INSTANCE-BASED LEARNING : Nearest Neighbour with Generalisation , 1995 .

[5]  Kai Ming Ting,et al.  Maximizing Tree Diversity by Building Complete-Random Decision Trees , 2005, PAKDD.

[6]  Qihao Weng,et al.  A survey of image classification methods and techniques for improving classification performance , 2007 .

[7]  Chien-Chang Ho,et al.  ADHD classification by a texture analysis of anatomical brain MRI data , 2012, Front. Syst. Neurosci..

[8]  Wanqing Li,et al.  A novel shape-based non-redundant local binary pattern descriptor for object detection , 2013, Pattern Recognit..

[9]  Blaz Zupan,et al.  Spam Filtering Using Statistical Data Compression Models , 2006, J. Mach. Learn. Res..

[10]  Joachim Denzler,et al.  Learning with few examples for binary and multiclass classification using regularization of randomized trees , 2011, Pattern Recognit. Lett..

[11]  M. B. Nebel,et al.  Automated diagnoses of attention deficit hyperactive disorder using magnetic resonance imaging , 2012, Front. Syst. Neurosci..

[12]  Mark Johnston,et al.  Genetic Programming Evolved Filters from a Small Number of Instances for Multiclass Texture Classification , 2014, IVCNZ '14.

[13]  Ron Kohavi,et al.  Scaling Up the Accuracy of Naive-Bayes Classifiers: A Decision-Tree Hybrid , 1996, KDD.

[14]  Mengjie Zhang,et al.  Two-Tier genetic programming: towards raw pixel-based image classification , 2012, Expert Syst. Appl..

[15]  Victor Ciesielski,et al.  Genetic Programming for Multiple Class Object Detection , 1999, Australian Joint Conference on Artificial Intelligence.

[16]  Mark Johnston,et al.  Multi-Objective Genetic Programming for object detection , 2010, IEEE Congress on Evolutionary Computation.

[17]  Mark Johnston,et al.  Genetic programming for image classification with unbalanced data , 2009, 2009 24th International Conference Image and Vision Computing New Zealand.

[18]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[19]  Zhi-Hua Zhou,et al.  Face recognition from a single image per person: A survey , 2006, Pattern Recognit..

[20]  John G. Cleary,et al.  K*: An Instance-based Learner Using and Entropic Distance Measure , 1995, ICML.

[21]  Mark Johnston,et al.  Genetic Programming for Multiclass Texture Classification Using a Small Number of Instances , 2014, SEAL.

[22]  Gustavo Olague,et al.  Evolutionary learning of local descriptor operators for object recognition , 2009, GECCO.

[23]  A. Dobra,et al.  Probabilistic Characterization of Random Decision Trees , 2008 .

[24]  Mengjie Zhang,et al.  Extracting image features for classification by two-tier genetic programming , 2012, 2012 IEEE Congress on Evolutionary Computation.

[25]  Mark Johnston,et al.  Binary image classification using genetic programming based on local binary patterns , 2013, 2013 28th International Conference on Image and Vision Computing New Zealand (IVCNZ 2013).

[26]  Mark Johnston,et al.  A One-Shot Learning Approach to Image Classification Using Genetic Programming , 2013, Australasian Conference on Artificial Intelligence.

[27]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[28]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[29]  Paul A. Viola,et al.  Learning from one example through shared densities on transforms , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[30]  Ryan M. Rifkin,et al.  In Defense of One-Vs-All Classification , 2004, J. Mach. Learn. Res..

[31]  Matti Pietikäinen,et al.  Face Description with Local Binary Patterns: Application to Face Recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Tetsuo Ishikawa,et al.  Visual one-shot learning as an ‘anti-camouflage device’: a novel morphing paradigm , 2011, Cognitive Neurodynamics.

[33]  Joshua B. Tenenbaum,et al.  One-Shot Learning with a Hierarchical Nonparametric Bayesian Model , 2011, ICML Unsupervised and Transfer Learning.

[34]  Anil K. Jain,et al.  Small Sample Size Effects in Statistical Pattern Recognition: Recommendations for Practitioners , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[35]  Massimiliano Pontil,et al.  Face Detection in Still Gray Images , 2000 .

[36]  Pat Langley,et al.  Estimating Continuous Distributions in Bayesian Classifiers , 1995, UAI.

[37]  Hamed Kiani Galoogahi,et al.  Shape Classification Using Local and Global Features , 2010, 2010 Fourth Pacific-Rim Symposium on Image and Video Technology.

[38]  Richard Szeliski,et al.  Computer Vision - Algorithms and Applications , 2011, Texts in Computer Science.

[39]  J. L. Hodges,et al.  Discriminatory Analysis - Nonparametric Discrimination: Consistency Properties , 1989 .

[40]  Victor Ciesielski,et al.  Texture analysis by genetic programming , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[41]  Matti Pietikäinen,et al.  Performance evaluation of texture measures with classification based on Kullback discrimination of distributions , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[42]  John C. Platt,et al.  Fast training of support vector machines using sequential minimal optimization, advances in kernel methods , 1999 .

[43]  Zhenhua Guo,et al.  Rotation invariant texture classification using LBP variance (LBPV) with global matching , 2010, Pattern Recognit..

[44]  Mengjie Zhang,et al.  A Two Phase Genetic Programming Approach to Object Detection , 2004, KES.

[45]  Bruno Galerne,et al.  Random Phase Textures: Theory and Synthesis , 2011, IEEE Transactions on Image Processing.

[46]  Mark Johnston,et al.  A Variant Program Structure in Tree-Based Genetic Programming for Multiclass Object Classification , 2009 .

[47]  Anil K. Jain,et al.  39 Dimensionality and sample size considerations in pattern recognition practice , 1982, Classification, Pattern Recognition and Reduction of Dimensionality.

[48]  Matthew B. Blaschko,et al.  Combining Local and Global Image Features for Object Class Recognition , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[49]  Matti Pietikäinen,et al.  Local Binary Patterns for Still Images , 2011 .

[50]  Mengjie Zhang,et al.  A domain independent Genetic Programming approach to automatic feature extraction for image classification , 2011, 2011 IEEE Congress of Evolutionary Computation (CEC).

[51]  Mengjie Zhang,et al.  Genetic programming for improving image descriptors generated using the scale-invariant feature transform , 2012, IVCNZ '12.

[52]  Mengjie Zhang,et al.  Multiclass Object Classification Using Genetic Programming , 2004, EvoWorkshops.

[53]  Matti Pietikäinen,et al.  A comparative study of texture measures with classification based on featured distributions , 1996, Pattern Recognit..

[54]  Mark Johnston,et al.  Automatic Construction of Invariant Features Using Genetic Programming for Edge Detection , 2012, Australasian Conference on Artificial Intelligence.

[55]  Leonardo Trujillo,et al.  Evolutionary-computer-assisted design of image operators that detect interest points using genetic programming , 2011, Image Vis. Comput..

[56]  Pierre Geurts,et al.  Extremely randomized trees , 2006, Machine Learning.

[57]  Li WangDong-Chen He,et al.  Texture classification using texture spectrum , 1990, Pattern Recognit..

[58]  Mengjie Zhang,et al.  Multiclass object classification for computer vision using Linear Genetic Programming , 2009, 2009 24th International Conference Image and Vision Computing New Zealand.

[59]  F. Wilcoxon Individual Comparisons by Ranking Methods , 1945 .

[60]  Bao-Liang Lu,et al.  Feature Selection for Fast Image Classification with Support Vector Machines , 2004, ICONIP.

[61]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[62]  Yaochu Jin,et al.  Evolutionary Multiobjective Image Feature Extraction in the Presence of Noise , 2015, IEEE Transactions on Cybernetics.

[63]  Matti Pietikäinen,et al.  Gray Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2000, ECCV.

[64]  Yoav Freund,et al.  Experiments with a New Boosting Algorithm , 1996, ICML.

[65]  Miguel A. Otaduy,et al.  Solid-Texture Synthesis: A Survey , 2010, IEEE Computer Graphics and Applications.

[66]  Paul W. Fieguth,et al.  Extended local binary patterns for texture classification , 2012, Image Vis. Comput..

[67]  Gustaf Kylberg,et al.  Kylberg Texture Dataset v. 1.0 , 2011 .

[68]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[69]  Pietro Perona,et al.  One-shot learning of object categories , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[70]  Mark Johnston,et al.  Genetic Programming for Edge Detection Based on Accuracy of Each Training Image , 2011, Australasian Conference on Artificial Intelligence.