Hierarchical classification with reject option for live fish recognition

A live fish recognition system is needed in application scenarios where manual annotation is too expensive, i.e. too many underwater videos. We present a novel balance-enforced optimized tree with reject option (BEOTR) for live fish recognition. It recognizes the top 15 common species of fish and detects new species in an unrestricted natural environment recorded by underwater cameras. The three main contributions of the paper are: (1) a novel hierarchical classification method suited for greatly unbalanced classes, (2) a novel classification-rejection method to clear up decisions and reject unknown classes, (3) an application of the classification method to free swimming fish. This system assists ecological surveillance research, e.g. fish population statistics in the open sea. BEOTR is automatically constructed based on inter-class similarities. Afterwards, trajectory voting is used to eliminate accumulated errors during hierarchical classification and, therefore, achieves better performance. We apply a Gaussian mixture model and Bayes rule as a reject option after the hierarchical classification to evaluate the posterior probability of being a certain species to filter less confident decisions. The proposed BEOTR-based hierarchical classification method achieves significant improvements compared to state-of-the-art techniques on a live fish image dataset of 24,150 manually labelled images from South Taiwan Sea.

[1]  G. Karypis,et al.  Criterion functions for document clustering , 2005 .

[2]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[3]  Raimondo Schettini,et al.  Underwater Image Processing: State of the Art of Restoration and Image Enhancement Methods , 2010, EURASIP J. Adv. Signal Process..

[4]  Nelson H. C. Yung,et al.  Curvature scale space corner detector with adaptive threshold and dynamic region of support , 2004, ICPR 2004.

[5]  Qi Zhang,et al.  EM-DD: An Improved Multiple-Instance Learning Technique , 2001, NIPS.

[6]  Robert M. Haralick,et al.  Textural Features for Image Classification , 1973, IEEE Trans. Syst. Man Cybern..

[7]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Alastair R. Allen,et al.  Fish species recognition by shape analysis of images , 1990, Pattern Recognit..

[9]  S. Sathiya Keerthi,et al.  Which Is the Best Multiclass SVM Method? An Empirical Study , 2005, Multiple Classifier Systems.

[10]  Fei-Fei Li,et al.  What Does Classifying More Than 10, 000 Image Categories Tell Us? , 2010, ECCV.

[11]  Robert B. Fisher,et al.  Understanding fish behavior during typhoon events in real-life underwater environments , 2012, Multimedia Tools and Applications.

[12]  Leen-Kiat Soh,et al.  Texture analysis of SAR sea ice imagery using gray level co-occurrence matrices , 1999, IEEE Trans. Geosci. Remote. Sens..

[13]  Rong Yan,et al.  Multiple instance learning for labeling faces in broadcasting news video , 2005, MULTIMEDIA '05.

[14]  David Casasent,et al.  A support vector hierarchical method for multi-class classification and rejection , 2009, 2009 International Joint Conference on Neural Networks.

[15]  Andrew Blake,et al.  "GrabCut" , 2004, ACM Trans. Graph..

[16]  John Platt,et al.  Probabilistic Outputs for Support vector Machines and Comparisons to Regularized Likelihood Methods , 1999 .

[17]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[18]  A. D. Gordon.  A Review of Hierarchical Classification , 1987 .

[19]  Anil K. Jain,et al.  Unsupervised Learning of Finite Mixture Models , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Robert B. Fisher,et al.  GMM improves the reject option in hierarchical classification for fish recognition , 2014, IEEE Winter Conference on Applications of Computer Vision.

[21]  Shaogang Gong,et al.  Modelling facial colour and identity with Gaussian mixtures , 1998, Pattern Recognit..

[22]  Andrew Zisserman,et al.  Representing shape with a spatial pyramid kernel , 2007, CIVR '07.

[23]  D.R. Edgington,et al.  Detecting, Tracking and Classifying Animals in Underwater Video , 2005, OCEANS 2006.

[24]  S. Chib.  Marginal Likelihood from the Gibbs Output , 1995 .

[25]  Robert B. Fisher,et al.  Underwater Live Fish Recognition Using a Balance-Guaranteed Optimized Tree , 2012, ACCV.

[26]  Phoenix X. Huang,et al.  Balance-guaranteed optimized tree with reject option for live fish recognition , 2014 .

[27]  Alex A. Freitas,et al.  A survey of hierarchical classification across different application domains , 2010, Data Mining and Knowledge Discovery.

[28]  J. Flusser,et al.  Moments and Moment Invariants in Pattern Recognition , 2009 .

[29]  Concetto Spampinato,et al.  MTAP special issue on methods and tools for ground truth collection in multimedia applications , 2014, Multimedia Tools and Applications.

[30]  Thomas M. Breuel,et al.  Classification using a hierarchical Bayesian approach , 2002, Object recognition supported by user interaction for service robots.

[31]  Christof Koch,et al.  Detection and tracking of objects in underwater video , 2004, CVPR 2004.

[32]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[33]  M. Okamoto,et al.  Fundamental study to estimate fish biomass around coral reef using 3-dimensional underwater video system , 2000, OCEANS 2000 MTS/IEEE Conference and Exhibition. Conference Proceedings (Cat. No.00CH37158).

[34]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .

[35]  J. A. Marchant,et al.  Fish sizing and monitoring using a stereo image analysis system applied to fish farming , 1995 .

[36]  Trevor Darrell,et al.  Face Recognition from Long-Term Observations , 2002, ECCV.

[37]  N. J. C. Strachan,et al.  Recognition of fish species by colour and shape , 1993, Image Vis. Comput..

[38]  Farzin Mokhtarian,et al.  Robust Image Corner Detection Through Curvature Scale Space , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[39]  Jan Flusser,et al.  Affine Moment Invariants , 2009 .

[40]  Bo Zhang,et al.  Learning Vocabulary-Based Hashing with AdaBoost , 2010, MMM.

[41]  B. K. Liew,et al.  Automated Fish Counting Using Image Processing , 2009, 2009 International Conference on Computational Intelligence and Software Engineering.

[42]  Tomás Lozano-Pérez,et al.  A Framework for Multiple-Instance Learning , 1997, NIPS.

[43]  Robert B. Fisher,et al.  Automatic fish classification for underwater species behavior understanding , 2010, ARTEMIS '10.

[44]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[45]  Tomer Hertz,et al.  Computing Gaussian Mixture Models with EM Using Equivalence Constraints , 2003, NIPS.

[46]  Dah-Jye Lee,et al.  Contour matching for a fish recognition and migration-monitoring system , 2004, SPIE Optics East.

[47]  Lawrence M. Dill,et al.  FOOD AVAILABILITY AND TIGER SHARK PREDATION RISK INFLUENCE BOTTLENOSE DOLPHIN HABITAT USE , 2002 .

[48]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[49]  Boaz Zion,et al.  In-vivo fish sorting by computer vision , 2000 .

[50]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[51]  Robert B. Fisher,et al.  Supporting ground-truth annotation of image datasets using clustering , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[52]  Norval J. C. Strachan,et al.  Length measurement of fish by computer vision , 1993 .

[53]  Kun Duan,et al.  Discovering localized attributes for fine-grained recognition , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[54]  Terry P. Hughes,et al.  RECRUITMENT AND THE LOCAL DYNAMICS OF OPEN MARINE POPULATIONS , 1996 .

[55]  Andrew Rova,et al.  One Fish, Two Fish, Butterfish, Trumpeter: Recognizing Fish in Underwater Video , 2007, MVA.

[56]  David Mouillot,et al.  Amphidromous fish school migration revealed by combining fixed sonar monitoring (horizontal beaming) with fishing data , 2006 .

[57]  J. Rissanen.  Stochastic Complexity and Modeling , 1986 .

[58]  Rasmus Larsen,et al.  Shape and Texture Based Classification of Fish Species , 2009, SCIA.

[59]  Pedro Larrañaga,et al.  A review of feature selection techniques in bioinformatics , 2007, Bioinform..

[60]  Robert B. Fisher,et al.  A flexible system for automated composition of intelligent video analysis , 2011, 2011 7th International Symposium on Image and Signal Processing and Analysis (ISPA).