Bioimage Classification with Handcrafted and Learned Features

Bioimage classification is increasingly becoming more important in many biological studies including those that require accurate cell phenotype recognition, subcellular localization, and histopathological classification. In this paper, we present a new General Purpose (GenP) bioimage classification method that can be applied to a large range of classification problems. The GenP system we propose is an ensemble that combines multiple texture features (both handcrafted and learned descriptors) for superior and generalizable discriminative power. Our ensemble obtains a boosting of performance by combining local features, dense sampling features, and deep learning features. Each descriptor is used to train a different Support Vector Machine that is then combined by sum rule. We evaluate our method on a diverse set of bioimage classification tasks each represented by a benchmark database, including some of those available in the IICBU 2008 database. Each bioimage classification task represents a typical subcellular, cellular, and tissue level classification problem. Our evaluation on these datasets demonstrates that the proposed GenP bioimage ensemble obtains state-of-the-art performance without any ad-hoc dataset tuning of the parameters (thereby avoiding any risk of overfitting/overtraining). To reproduce the experiments reported in this paper, the MATLAB code of all the descriptors is available at https://github.com/LorisNanni and https://www.dropbox.com/s/bguw035yrqz0pwp/ElencoCode.docx?dl=0.

[1]  H. Leonhardt,et al.  A guide to super-resolution fluorescence microscopy , 2010, The Journal of cell biology.

[2]  Jarbas Joaci de Mesquita Sá Junior,et al.  Plant leaf identification using Gabor wavelets , 2009 .

[3]  Vishal Monga,et al.  Histopathological Image Classification Using Discriminative Feature-Oriented Dictionary Learning , 2015, IEEE Transactions on Medical Imaging.

[4]  Robert F. Murphy,et al.  Determining the subcellular location of new proteins from microscope images using local features , 2013, Bioinform..

[5]  Hanchuan Peng,et al.  BIOCAT: a pattern recognition platform for customizable biological image classification and annotation , 2013, BMC Bioinformatics.

[6]  Lei Zhang,et al.  Towards effective codebookless model for image classification , 2015, Pattern Recognit..

[7]  Nicolas Hervé,et al.  Statistical color texture descriptors for histological images analysis , 2011, 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[8]  Loris Nanni,et al.  A local approach based on a Local Binary Patterns variant texture descriptor for classifying pain states , 2010, Expert Syst. Appl..

[9]  Adrien Depeursinge,et al.  Automated classification of brain tumor type in whole-slide digital pathology images using local representative tiles , 2016, Medical Image Anal..

[10]  B. S. Manjunath,et al.  Biological imaging software tools , 2012, Nature Methods.

[11]  Loris Nanni,et al.  Ensemble of Local Phase Quantization Variants with Ternary Encoding , 2013, Local Binary Patterns.

[12]  Esa Rahtu,et al.  BSIF: Binarized statistical image features , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[13]  Qilong Wang,et al.  Local Log-Euclidean Covariance Matrix (L2ECM) for Image Representation and Its Applications , 2012, ECCV.

[14]  Ephraim Feig,et al.  Fast algorithms for the discrete cosine transform , 1992, IEEE Trans. Signal Process..

[15]  Andrea Vedaldi,et al.  Vlfeat: an open and portable library of computer vision algorithms , 2010, ACM Multimedia.

[16]  Emanuele Menegatti,et al.  A comparison of methods for extracting information from the co-occurrence matrix for subcellular classification , 2013, Expert Syst. Appl..

[17]  Junzhou Huang,et al.  Joint Kernel-Based Supervised Hashing for Scalable Histopathological Image Analysis , 2015, MICCAI.

[18]  Hong-Bin Shen,et al.  Many Local Pattern Texture Features: Which Is Better for Image-Based Multilabel Human Protein Subcellular Localization Classification? , 2014, TheScientificWorldJournal.

[19]  L. Nanni,et al.  Non-Binary Coding for Texture Descriptors in Sub-Cellular and Stem Cell Image Classification , 2013 .

[20]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[21]  Xiaoyang Tan,et al.  Enhanced Local Texture Feature Sets for Face Recognition Under Difficult Lighting Conditions , 2007, IEEE Transactions on Image Processing.

[22]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[23]  Alessandra Lumini,et al.  Subcellular localization using fluorescence imagery: Utilizing ensemble classification with diverse feature extraction strategies and data balancing , 2013, Appl. Soft Comput..

[24]  Anant Madabhushi,et al.  Explicit shape descriptors: Novel morphologic features for histopathology classification , 2013, Medical Image Anal..

[25]  Cordelia Schmid,et al.  Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[26]  Fan Yang,et al.  Image-based classification of protein subcellular location patterns in human reproductive tissue by ensemble learning global and local features , 2014, Neurocomputing.

[27]  Mikhail Teverovskiy,et al.  Multifeature Prostate Cancer Diagnosis and Gleason Grading of Histological Images , 2007, IEEE Transactions on Medical Imaging.

[28]  Péter Horváth,et al.  Enhanced CellClassifier: a multi-class classification tool for microscopy images , 2010, BMC Bioinformatics.

[29]  Fabio A. González,et al.  Combining Unsupervised Feature Learning and Riesz Wavelets for Histopathology Image Representation: Application to Identifying Anaplastic Medulloblastoma , 2015, MICCAI.

[30]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[31]  Shih-Fu Chang,et al.  Image Retrieval: Current Techniques, Promising Directions, and Open Issues , 1999, J. Vis. Commun. Image Represent..

[32]  Loris Nanni,et al.  How could a subcellular image, or a painting by Van Gogh, be similar to a great white shark or to a pizza? , 2017, Pattern Recognit. Lett..

[33]  D. Sagi,et al.  Gabor filters as texture discriminator , 1989, Biological Cybernetics.

[34]  Anonymous Authors Empowering Multiple Instance Histopathology Cancer Diagnosis by Cell Graphs , 2014 .

[35]  Andrea Vedaldi,et al.  MatConvNet: Convolutional Neural Networks for MATLAB , 2014, ACM Multimedia.

[36]  Wolfgang Huber,et al.  EBImage—an R package for image processing with applications to cellular phenotypes , 2010, Bioinform..

[37]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[38]  Francesco Bianconi,et al.  Multi-class texture analysis in colorectal cancer histology , 2016, Scientific Reports.

[39]  Anne E Carpenter,et al.  CP-CHARM: segmentation-free image classification made accessible , 2016, BMC Bioinformatics.

[40]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[41]  R. Murphy,et al.  Automated subcellular location determination and high-throughput microscopy. , 2007, Developmental cell.

[42]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[43]  Robert F. Murphy,et al.  A neural network classifier capable of recognizing the patterns of all major subcellular structures in fluorescence microscope images of HeLa cells , 2001, Bioinform..

[44]  Yoshua Bengio,et al.  How transferable are features in deep neural networks? , 2014, NIPS.

[45]  Nicholas A. Hamilton,et al.  Fast automated cell phenotype image classification , 2007, BMC Bioinformatics.

[46]  Bahram Parvin,et al.  Classification of Histology Sections via Multispectral Convolutional Sparse Coding , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[47]  Hanchuan Peng,et al.  Bioimage informatics: a new area of engineering biology , 2008, Bioinform..

[48]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[49]  Robert M. Haralick,et al.  Textural Features for Image Classification , 1973, IEEE Trans. Syst. Man Cybern..

[50]  Gabriela Csurka,et al.  Visual categorization with bags of keypoints , 2002, eccv 2004.

[51]  Hanqing Lu,et al.  Face detection using improved LBP under Bayesian framework , 2004, Third International Conference on Image and Graphics (ICIG'04).

[52]  Andrew Zisserman,et al.  Return of the Devil in the Details: Delving Deep into Convolutional Nets , 2014, BMVC.

[53]  Loris Nanni,et al.  A very high performing system to discriminate tissues in mammograms as benign and malignant , 2012, Expert Syst. Appl..

[54]  Marc'Aurelio Ranzato,et al.  Building high-level features using large scale unsupervised learning , 2011, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[55]  Tom Heskes,et al.  A comparative study of cell classifiers for image-based high-throughput screening , 2014, BMC Bioinformatics.

[56]  Lior Shamir,et al.  IICBU 2008: a proposed benchmark suite for biological image analysis , 2008, Medical & Biological Engineering & Computing.

[57]  Matthijs C. Dorst Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[58]  Anil K. Jain,et al.  Image classification for content-based indexing , 2001, IEEE Trans. Image Process..

[59]  Kazuhiro Fukui,et al.  HEp-2 cell classification using rotation invariant co-occurrence among local binary patterns , 2014, Pattern Recognit..

[60]  Heng Huang,et al.  Bioimage classification with subcategory discriminant transform of high dimensional visual descriptors , 2016, BMC Bioinformatics.

[61]  Malay Kumar Kundu,et al.  Pap smear image classification using convolutional neural network , 2016, ICVGIP '16.

[62]  Bernd Fischer,et al.  CellCognition: time-resolved phenotype annotation in high-throughput live cell imaging , 2010, Nature Methods.

[63]  Yoshua Bengio,et al.  Why Does Unsupervised Pre-training Help Deep Learning? , 2010, AISTATS.

[64]  Ludmila I. Kuncheva,et al.  Measures of Diversity in Classifier Ensembles and Their Relationship with the Ensemble Accuracy , 2003, Machine Learning.

[65]  Ghassan Hamarneh,et al.  Automatic Diagnosis of Ovarian Carcinomas via Sparse Multiresolution Tissue Representation , 2015, MICCAI.

[66]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[67]  Hai Su,et al.  Robust Cell Detection and Segmentation in Histopathological Images Using Sparse Reconstruction and Stacked Denoising Autoencoders , 2015, MICCAI.

[68]  Loris Nanni,et al.  Local binary patterns variants as texture descriptors for medical image analysis , 2010, Artif. Intell. Medicine.

[69]  Yen-Wei Chen,et al.  HEp-2 Cell Classification Using K-Support Spatial Pooling in Deep CNNs , 2016, LABELS/DLMIA@MICCAI.

[70]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[71]  Yu-Bin Yang,et al.  Visual feature coding for image classification integrating dictionary structure , 2015, Pattern Recognit..

[72]  Loris Nanni,et al.  Handcrafted vs. non-handcrafted features for computer vision classification , 2017, Pattern Recognit..

[73]  Shu-Ching Chen,et al.  Histology Image Classification Using Supervised Classification and Multimodal Fusion , 2010, 2010 IEEE International Symposium on Multimedia.

[74]  Zhenhua Guo,et al.  Rotation invariant texture classification using LBP variance (LBPV) with global matching , 2010, Pattern Recognit..

[75]  Zhuowen Tu,et al.  Weakly supervised histopathology cancer image segmentation and classification , 2014, Medical Image Anal..

[76]  Eli Shechtman,et al.  In defense of Nearest-Neighbor based image classification , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[77]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[78]  Zhenhua Guo,et al.  A Completed Modeling of Local Binary Pattern Operator for Texture Classification , 2010, IEEE Transactions on Image Processing.

[79]  Lior Shamir,et al.  Source Code for Biology and Medicine Open Access Wndchrm – an Open Source Utility for Biological Image Analysis , 2022 .

[80]  Heng Huang,et al.  Region-based progressive localization of cell nuclei in microscopic images with data adaptive modeling , 2013, BMC Bioinformatics.

[81]  Mei-Ling Shyu,et al.  Biological Image Temporal Stage Classification via Multi-layer Model Collaboration , 2013, 2013 IEEE International Symposium on Multimedia.

[82]  Frédéric Jurie,et al.  Sampling Strategies for Bag-of-Features Image Classification , 2006, ECCV.

[83]  Kai Huang,et al.  Automated classification of subcellular patterns in multicell images without segmentation into single cells , 2004, 2004 2nd IEEE International Symposium on Biomedical Imaging: Nano to Macro (IEEE Cat No. 04EX821).

[84]  Ville Ojansivu,et al.  Blur Insensitive Texture Classification Using Local Phase Quantization , 2008, ICISP.

[85]  Jiwen Lu,et al.  PCANet: A Simple Deep Learning Baseline for Image Classification? , 2014, IEEE Transactions on Image Processing.

[86]  Xinge You,et al.  An adaptive hybrid pattern for noise-robust texture analysis , 2015, Pattern Recognit..

[87]  Bailing Zhang,et al.  Phenotype Recognition with Combined Features and Random Subspace Classifier Ensemble , 2011, BMC Bioinformatics.

[88]  Thomas Mensink,et al.  Image Classification with the Fisher Vector: Theory and Practice , 2013, International Journal of Computer Vision.

[89]  Paci Michelangelo,et al.  Review on Texture Descriptors for Image Classification , 2016 .

[90]  Rita Cucchiara,et al.  GOLD: Gaussians of Local Descriptors for image representation , 2015, Comput. Vis. Image Underst..

[91]  Runsheng Wang,et al.  Local multiple patterns based multiresolution gray-scale and rotation invariant texture classification , 2012, Inf. Sci..