Video Based Live Tracking of Fishes in Tanks

We explore video tracking and classification in the context of real time marine wildlife observation. Among other applications it can help biologists by automating the process of gathering data, which is often done manually. In this paper we present a system to tackle the challenge of tracking and classifying fish in real time. We apply Background Subtraction techniques to detect the fish, followed by Feature Matching methods to track their movements over time. To deal with the shortcomings of tracking by detection we use a Kalman Filter to predict fish positions and a local search recovery method to re-identify fish tracks that are temporarily lost due to occlusions or lack of contrast. The species of tracked fish is recognized through Image Classification methods, using environment dependent features. We developed and tested our system using a custom built dataset, with several labeled image sequences of the fish tanks in the Oceanario de Lisboa. The impact of the proposed tracking methods are quantified and discussed. The proposed system is able to track and classify fish in real time in two scenarios, main tank and coral reef, reflecting different challenges.

[1]  Saiful Islam,et al.  Mahalanobis Distance , 2009, Encyclopedia of Biometrics.

[2]  Marc Chaumont,et al.  Coral Reef Fish Detection and Recognition in Underwater Videos by Supervised Machine Learning: Comparison Between Deep Learning and HOG+SVM Methods , 2016, ACIVS.

[3]  S. Johnsen,et al.  Real-Time Object Tracking and Classification Using a Static Camera , 2009 .

[4]  Simone Palazzo,et al.  A video processing and data retrieval framework for fish population monitoring , 2013, MAED '13.

[5]  SpampinatoConcetto,et al.  A rule-based event detection system for real-life underwater domain , 2014 .

[6]  James W. Davis,et al.  Real-time recognition of activity using temporal templates , 1996, Proceedings Third IEEE Workshop on Applications of Computer Vision. WACV'96.

[7]  Simone Palazzo,et al.  Quantitative performance analysis of object detection algorithms on underwater video footage , 2012, MAED '12.

[8]  Simone Palazzo,et al.  An innovative web-based collaborative platform for video annotation , 2014, Multimedia Tools and Applications.

[9]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[10]  Phoenix X. Huang,et al.  Hierarchical Classification for Live Fish Recognition , 2012 .

[11]  Mohammed Bennamoun,et al.  Real time surveillance for low resolution and limited data scenarios: An image set classification approach , 2018, Inf. Sci..

[12]  Nigel J. B. McFarlane,et al.  Segmentation and tracking of piglets in images , 1995, Machine Vision and Applications.

[13]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[14]  Hanan Samet,et al.  A general approach to connected-component labeling for arbitrary image representations , 1992, JACM.

[15]  Emma Beauxis-Aussalet,et al.  A case study of trust issues in scientific video collections , 2013, MAED '13.

[16]  Hironobu Fujiyoshi,et al.  Moving target classification and tracking from real-time video , 1998, Proceedings Fourth IEEE Workshop on Applications of Computer Vision. WACV'98 (Cat. No.98EX201).

[17]  Johan A. K. Suykens,et al.  Least Squares Support Vector Machine Classifiers , 1999, Neural Processing Letters.

[18]  Andrew Harvey,et al.  Forecasting, Structural Time Series Models and the Kalman Filter , 1990 .

[19]  W. Eric L. Grimson,et al.  Adaptive background mixture models for real-time tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[20]  Robert B. Fisher,et al.  Approximate nearest neighbor search to support manual image annotation of large domain-specific datasets , 2013, VIGTA@ICVS.

[21]  Concetto Spampinato,et al.  Generation of Ground Truth for Object Detection While Playing an Online Game: Productive Gaming or Recreational Working? , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[22]  Simone Palazzo,et al.  A rule-based event detection system for real-life underwater domain , 2013, Machine Vision and Applications.

[23]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[24]  Cheng-Lin Yang,et al.  Real-Time Data Streaming Architecture and Intelligent Workflow Management for Processing Massive Ecological Videos , 2013, 2013 International Conference on Social Computing.

[25]  Z. Zivkovic.  Improved adaptive Gaussian mixture model for background subtraction , 2004, ICPR 2004.

[26]  Stuart J. Russell,et al.  Image Segmentation in Video Sequences: A Probabilistic Approach , 1997, UAI.

[27]  Robert B. Fisher,et al.  Detecting, Tracking and Counting Fish in Low Quality Unconstrained Underwater Videos , 2008, VISAPP.

[28]  Daniela Giordano,et al.  A crowdsourcing approach to support video annotation , 2013, VIGTA@ICVS.

[29]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[30]  Neil A. Thacker,et al.  The Bhattacharyya metric as an absolute similarity measure for frequency coded data , 1998, Kybernetika.