Combination of Image and Location Information for Snake Species Identification using Object Detection and EfficientNets

Snake species identification based on images is important to quickly treat patients suffering from snake bites using the correct antivenom. The SnakeCLEF 2020 challenge, which is part of the LifeCLEF research platform, is focused on this task and provides snake images and associated location information. This paper describes the participation of the FHDO Biomedical Computer Science Group (BCSG) in this challenge. The implemented machine learning workflow uses Mask Region-based Convolutional Neural Network (Mask R-CNN) for object detection, various image pre-processing steps, EfficientNets for classification as well as different methods to fuse image and location information. The best model submitted before the challenge deadline achieved a macro-averaging F1-score of 0.404. After the expiration of this deadline, the results could be improved up to a macro-averaging F1-score of 0.594.

[1]  D. Ruppert,et al.  Efficient Estimations from a Slowly Convergent Robbins-Monro Process , 1988 .

[2]  R. Badlishah Ahmad,et al.  Image Classification for Snake Species Using Machine Learning Techniques , 2016 .

[3]  Quoc V. Le,et al.  Self-Training With Noisy Student Improves ImageNet Classification , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[5]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[6]  Indika Perera,et al.  Snake Image Classification using Siamese Networks , 2019, Proceedings of the 2019 3rd International Conference on Graphics and Signal Processing.

[7]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[8]  Irina Matijosaitiene,et al.  Revealing the Unknown: Real-Time Recognition of Galápagos Snake Species Using Deep Learning , 2020, Animals : an open access journal from MDPI.

[9]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[10]  Christoph M. Friedrich,et al.  Variations on Branding with Text Occurrence for Optimized Body Parts Classification , 2019, 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[11]  Yann LeCun,et al.  Signature Verification Using A "Siamese" Time Delay Neural Network , 1993, Int. J. Pattern Recognit. Artif. Intell..

[12]  Isabelle Bolon,et al.  Identifying the snake: First scoping review on practices of communities and healthcare providers confronted with snakebite across the world , 2020, PloS one.

[13]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Lukás Picek,et al.  Overview of the SnakeCLEF 2020: AutomaticSnake Species Identification Challenge , 2020, CLEF.

[15]  Stefan Kahl,et al.  Overview of LifeCLEF 2020: A System-Oriented Evaluation of Automated Species Identification and Species Distribution Prediction , 2020, CLEF.

[16]  Alex Pappachen James,et al.  Discriminative histogram taxonomy features for snake species identification , 2014, Human-centric Computing and Information Sciences.

[17]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[18]  Fred L. Drake,et al.  Python 3 Reference Manual , 2009 .

[19]  Quoc V. Le,et al.  Searching for Activation Functions , 2018, arXiv.

[20]  Sergio Guadarrama,et al.  Speed/Accuracy Trade-Offs for Modern Convolutional Object Detectors , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Suyanto Suyanto,et al.  Image-Based Classification of Snake Species Using Convolutional Neural Network , 2019, 2019 International Seminar on Research of Information Technology and Intelligent Systems (ISRITI).

[22]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[23]  Sven Koitka,et al.  Optimized Convolutional Neural Network Ensembles for Medical Subfigure Classification , 2017, CLEF.

[24]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Quoc V. Le,et al.  EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks , 2019, ICML.