Experimenting Liver Fibrosis Diagnostic by Two Photon Excitation Microscopy and Bag-of-Features Image Classification

The accurate staging of liver fibrosis is of paramount importance to determine the state of disease progression, therapy responses, and to optimize disease treatment strategies. Non-linear optical microscopy techniques such as two-photon excitation fluorescence (TPEF) and second harmonic generation (SHG) can image the endogenous signals of tissue structures and can be used for fibrosis assessment on non-stained tissue samples. While image analysis of collagen in SHG images was consistently addressed until now, cellular and tissue information included in TPEF images, such as inflammatory and hepatic cell damage, equally important as collagen deposition imaged by SHG, remain poorly exploited to date. We address this situation by experimenting liver fibrosis quantification and scoring using a combined approach based on TPEF liver surface imaging on a Thioacetamide-induced rat model and a gradient based Bag-of-Features (BoF) image classification strategy. We report the assessed performance results and discuss the influence of specific BoF parameters to the performance of the fibrosis scoring framework.

[1]  S. Xie,et al.  Label-Free Detection of Breast Masses Using Multiphoton Microscopy , 2013, PloS one.

[2]  Dinu Coltuc,et al.  Automated compensation of light attenuation in confocal microscopy by exact histogram specification , 2010, Microscopy research and technique.

[3]  Shuangmu Zhuo,et al.  Establishing diagnostic features for identifying the mucosa and submucosa of normal and cancerous gastric tissues by multiphoton microscopy. , 2011, Gastrointestinal endoscopy.

[4]  Fabio A. González,et al.  Histopathology Image Classification Using Bag of Features and Kernel Functions , 2009, AIME.

[5]  Ioannis Pratikakis,et al.  Bag of spatio-visual words for context inference in scene classification , 2013, Pattern Recognit..

[6]  Cordelia Schmid,et al.  Scale & Affine Invariant Interest Point Detectors , 2004, International Journal of Computer Vision.

[7]  Pietro Perona,et al.  A Bayesian hierarchical model for learning natural scene categories , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[8]  Shuangmu Zhuo,et al.  Preclinical study of using multiphoton microscopy to diagnose liver cancer and differentiate benign and malignant liver lesions. , 2012, Journal of biomedical optics.

[9]  B R Masters,et al.  Two-photon excitation fluorescence microscopy. , 2000, Annual review of biomedical engineering.

[10]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[11]  A Diaspro,et al.  Two-photon excitation of fluorescence for three-dimensional optical imaging of biological structures. , 2000, Journal of photochemistry and photobiology. B, Biology.

[12]  Tolga Can,et al.  Detection and tracking of sea-surface targets in infrared and visual band videos using the bag-of-features technique with scale-invariant feature transform. , 2011, Applied optics.

[13]  Sei-ichiro Kamata,et al.  Hilbert Scan Based Bag-of-Features for Image Retrieval , 2011, IEICE Trans. Inf. Syst..

[14]  Fei Yang,et al.  Visual tracking via bag of features , 2012 .

[15]  Gabriela Csurka,et al.  Visual categorization with bags of keypoints , 2002, eccv 2004.

[16]  Ina Pavlova,et al.  In vivo imaging of unstained tissues using a compact and flexible multiphoton microendoscope. , 2012, Journal of biomedical optics.

[17]  Christophe Odin,et al.  Fibrillar collagen scoring by second harmonic microscopy: a new tool in the assessment of liver fibrosis. , 2010, Journal of hepatology.

[18]  Nicholas Ayache,et al.  A smart atlas for endomicroscopy using automated video retrieval , 2011, Medical Image Anal..

[19]  Andrea Vedaldi,et al.  Vlfeat: an open and portable library of computer vision algorithms , 2010, ACM Multimedia.

[20]  Paul Campagnola,et al.  Second harmonic generation imaging microscopy: applications to diseases diagnostics. , 2011, Analytical chemistry.

[21]  K. Fujita.  [Two-photon laser scanning fluorescence microscopy]. , 2007, Tanpakushitsu kakusan koso. Protein, nucleic acid, enzyme.

[22]  Gertjan J. Burghouts,et al.  Performance evaluation of local colour invariants , 2009, Comput. Vis. Image Underst..

[23]  David Dagan Feng,et al.  Content-Based Image Retrieval in P2P Networks with Bag-of-Features , 2012, 2012 IEEE International Conference on Multimedia and Expo Workshops.

[24]  Mosalam Ebrahimi,et al.  SUSurE: Speeded Up Surround Extrema feature detector and descriptor for realtime applications , 2009, 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[25]  Bruce A. Draper,et al.  Introduction to the Bag of Features Paradigm for Image Classification and Retrieval , 2011, ArXiv.

[26]  G. Zouridakis,et al.  Malignant melanoma detection by Bag-of-Features classification , 2008, 2008 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[27]  Kurt Konolige,et al.  CenSurE: Center Surround Extremas for Realtime Feature Detection and Matching , 2008, ECCV.

[28]  Frédéric Jurie,et al.  Sampling Strategies for Bag-of-Features Image Classification , 2006, ECCV.

[29]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[30]  Qianjin Feng,et al.  Content-Based Retrieval of Focal Liver Lesions Using Bag-of-Visual-Words Representations of Single- and Multiphase Contrast-Enhanced CT Images , 2012, Journal of Digital Imaging.

[31]  Thomas Brox,et al.  Fast Rotation Invariant 3D Feature Computation Utilizing Efficient Local Neighborhood Operators , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Zhe L. Lin,et al.  A Local Bag-of-Features Model for Large-Scale Object Retrieval , 2010, ECCV.

[33]  Hanry Yu,et al.  Toward surface quantification of liver fibrosis progression. , 2010, Journal of biomedical optics.

[34]  M. Inaba.  Application of weighted Voronoi diagrams and randomization to variance-based k-clustering , 1994, SoCG 1994.

[35]  Radu Hristu,et al.  Influence of confocal scanning laser microscopy specific acquisition parameters on the detection and matching of speeded-up robust features. , 2011, Ultramicroscopy.

[36]  Radu Hristu,et al.  On the suitability of SIFT technique to deal with image modifications specific to confocal scanning laser microscopy. , 2010, Microscopy and microanalysis : the official journal of Microscopy Society of America, Microbeam Analysis Society, Microscopical Society of Canada.

[37]  Ina Pavlova,et al.  Compact and flexible raster scanning multiphoton endoscope capable of imaging unstained tissue , 2011, Proceedings of the National Academy of Sciences.

[38]  W. Denk,et al.  Two-photon laser scanning fluorescence microscopy. , 1990, Science.

[39]  Hans Burkhardt,et al.  A Bag of Features Approach for 3D Shape Retrieval , 2009, ISVC.

[40]  Gertjan J. Burghouts,et al.  Material-specific adaptation of color invariant features , 2009, Pattern Recognit. Lett..

[41]  Pierre Bedossa,et al.  Harmony in liver fibrosis... , 2010, Journal of hepatology.

[42]  Cordelia Schmid,et al.  Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[43]  อนิรุธ สืบสิงห์,et al.  Data Mining Practical Machine Learning Tools and Techniques , 2014 .

[44]  Chen-Yuan Dong,et al.  Optical biopsy of liver fibrosis by use of multiphoton microscopy. , 2004, Optics letters.

[45]  R. Sukthankar,et al.  PCA-SIFT: a more distinctive representation for local image descriptors , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[46]  Ioannis Pratikakis,et al.  3D object retrieval via range image queries in a bag-of-visual-words context , 2013, The Visual Computer.

[47]  Mark Goadrich,et al.  The relationship between Precision-Recall and ROC curves , 2006, ICML.

[48]  Kazufumi Kaneda,et al.  Computer-Aided Colorectal Tumor Classification in NBI Endoscopy Using CNN Features , 2016, ArXiv.

[49]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[50]  David G. Stork,et al.  Pattern Classification , 1973 .

[51]  Massimo Pinzani,et al.  Technology Insight: noninvasive assessment of liver fibrosis by biochemical scores and elastography , 2008, Nature Clinical Practice Gastroenterology &Hepatology.

[52]  Cordelia Schmid,et al.  A performance evaluation of local descriptors , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[53]  Barbara Caputo,et al.  Cue Integration for Medical Image Annotation , 2007, CLEF.

[54]  X. Forns,et al.  Noninvasive assessment of liver fibrosis , 2011, Hepatology.

[55]  J. MacQueen.  Some methods for classification and analysis of multivariate observations , 1967 .

[56]  Kartikeya Murari,et al.  A compact fiber-optic SHG scanning endomicroscope and its application to visualize cervical remodeling during pregnancy , 2012, Proceedings of the National Academy of Sciences.

[57]  Fabrice Carrat,et al.  Liver biopsy: the best, not the gold standard. , 2009, Journal of hepatology.

[58]  Afzal Godil,et al.  Non-rigid 3D shape retrieval using Multidimensional Scaling and Bag-of-Features , 2010, 2010 IEEE International Conference on Image Processing.

[59]  Chris Xu,et al.  Special Section Guest Editorial: Multiphoton Microscopy: Technical Innovations, Biological Applications, and Clinical Diagnostics , 2013, Journal of biomedical optics.

[60]  D. Jeulin,et al.  Imaging and 3D morphological analysis of collagen fibrils , 2012, Journal of microscopy.

[61]  Zhen Li,et al.  An efficient approach for scene categorization based on discriminative codebook learning in bag-of-words framework , 2013, Image Vis. Comput..

[62]  Mary Inaba,et al.  Applications of weighted Voronoi diagrams and randomization to variance-based k-clustering: (extended abstract) , 1994, SCG '94.

[63]  Chen-Yuan Dong,et al.  Second harmonic generation microscopy: principles and applications to disease diagnosis , 2011 .

[64]  Fabio A. González,et al.  Visual pattern mining in histology image collections using bag of features , 2011, Artif. Intell. Medicine.

[65]  Xingde Li,et al.  Scanning fiber-optic nonlinear endomicroscopy with miniature aspherical compound lens and multimode fiber collector. , 2009, Optics letters.

[66]  Mohammad Reza Zare,et al.  Automatic classification of medical X-ray images using a bag of visual words , 2013, IET Comput. Vis..

[67]  Frédéric Jurie,et al.  Creating efficient codebooks for visual recognition , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[68]  Cordelia Schmid,et al.  A Comparison of Affine Region Detectors , 2005, International Journal of Computer Vision.

[69]  Hanry Yu,et al.  Fibro-C-Index: comprehensive, morphology-based quantification of liver fibrosis using second harmonic generation and two-photon microscopy. , 2009, Journal of biomedical optics.

[70]  D. Brenner,et al.  Erratum: Liver fibrosis (Journal of Clinical Investigation (2005) 115 (209-218) DOI:10.1172/JCI200524282) , 2005 .

[71]  S. Friedman,et al.  Liver fibrosis -- from bench to bedside. , 2003, Journal of hepatology.

[72]  W. Webb,et al.  Multiphoton microscopy and microspectroscopy for diagnostics of inflammatory and neoplastic lung. , 2012, Journal of biomedical optics.