Understanding fish behavior during typhoon events in real-life underwater environments

The study of fish populations in their own natural environment is a task that has usually been tackled in invasive ways which inevitably influenced the behavior of the fish under observation. Recent projects involving the installation of permanent underwater cameras (e.g. the Fish4Knowledge (F4K) project, for the observation of Taiwan’s coral reefs) allow to gather huge quantities of video data, without interfering with the observed environment, but at the same time require the development of automatic processing tools, since manual analysis would be impractical for such amounts of videos. Event detection is one of the most interesting aspects from the biologists’ point of view, since it allows the analysis of fish activity during particular events, such as typhoons. In order to achieve this goal, in this paper we present an automatic video analysis approach for fish behavior understanding during typhoon events. The first step of the proposed system, therefore, involves the detection of “typhoon” events and it is based on video texture analysis and on classification by means of Support Vector Machines (SVM). As part of our behavior understanding efforts, trajectory extraction and clustering have been performed to study the differences in behavior when disruptive events happen. The integration of event detection with fish behavior understanding surpasses the idea of simply detecting events by low-level features analysis, as it supports the full semantic comprehension of interesting events.

[1]  F. Porikli,et al.  Change Detection by Frequency Decomposition: Wave-Back , 2005 .

[2]  Fatih Murat Porikli,et al.  Achieving real-time object detection and tracking under extreme conditions , 2006, Journal of Real-Time Image Processing.

[3]  Antonio Albiol,et al.  Automatic video annotation and event detection for video surveillance , 2009, ICDP.

[4]  Tobias Bjerregaard,et al.  A survey of research and practices of Network-on-chip , 2006, CSUR.

[5]  Michael G. Strintzis,et al.  Estimation and representation of accumulated motion characteristics for semantic event detection , 2008, 2008 15th IEEE International Conference on Image Processing.

[6]  J. P. Chen,et al.  Damage to the Reefs of Siangjiao Bay Marine Protected Area of Kenting National Park, Southern Taiwan during Typhoon Morakot , 2011 .

[7]  Véronique Malaisé,et al.  Abstracting and reasoning over ship trajectories and web data with the Simple Event Model (SEM) , 2010, Multimedia Tools and Applications.

[8]  Concetto Spampinato,et al.  Integrating Location Tracking, Traffic Monitoring and Semantics in a Layered ITS Architecture , 2011 .

[9]  Rasmus Larsen,et al.  Shape and Texture Based Classification of Fish Species , 2009, SCIA.

[10]  Steffen Staab,et al.  F--a model of events based on the foundational ontology dolce+DnS ultralight , 2009, K-CAP '09.

[11]  David S. Doermann,et al.  Tools and techniques for video performance evaluation , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[12]  Catherine A. Sugar,et al.  Finding the Number of Clusters in a Dataset , 2003 .

[13]  Danelle E. Cline,et al.  An Automated Visual Event Detection System for Cabled Observatory Video , 2007, OCEANS 2007.

[14]  Robert B. Fisher,et al.  Automatic fish classification for underwater species behavior understanding , 2010, ARTEMIS '10.

[15]  Fabrizio Granelli,et al.  Quality Evaluation and Nonuniform Compression of Geometrically Distorted Images Using the Quadtree Distortion Map , 2004, EURASIP J. Adv. Signal Process..

[16]  Kentaro Toyama,et al.  Wallflower: principles and practice of background maintenance , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[17]  W. Förstner,et al.  A Metric for Covariance Matrices , 2003 .

[18]  Duc Phu Chau,et al.  Online evaluation of tracking algorithm performance , 2009, ICDP.

[19]  M. Arshad,et al.  Underwater crowd flow detection using Lagrangian dynamics , 2009 .

[20]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[21]  Shiguo Lian,et al.  Special issue on multimedia analysis and security , 2010, Multimedia Tools and Applications.

[22]  Takakazu Ishimatsu,et al.  A Morphological Approach to Fish Discrimination , 1998, MVA.

[23]  Guillaume-Alexandre Bilodeau,et al.  A Multiscale Region-Based Motion Detection and Background Subtraction Algorithm , 2010, Sensors.

[24]  Concetto Spampinato,et al.  Variational Method for Image Denoising by Distributed Genetic Algorithms on GRID Environment , 2006, 15th IEEE International Workshops on Enabling Technologies: Infrastructure for Collaborative Enterprises (WETICE'06).

[25]  Alberto Del Bimbo,et al.  Event detection and recognition for semantic annotation of video , 2010, Multimedia Tools and Applications.

[26]  Robert B. Fisher,et al.  Detecting, Tracking and Counting Fish in Low Quality Unconstrained Underwater Videos , 2008, VISAPP.

[27]  W. Eric L. Grimson,et al.  Adaptive background mixture models for real-time tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[28]  R. A. Salam,et al.  Underwater Image Enhancement Using an Integrated Colour Model , 2007 .

[29]  Jung Uk Cho,et al.  Field Programmable Gate Array (FPGA) Based Fish Detection Using Haar Classifiers , 2009 .

[30]  Shireen Elhabian,et al.  Moving Object Detection in Spatial Domain using Background Removal Techniques - State-of-Art , 2008 .

[31]  Chandrika Kamath,et al.  Robust Background Subtraction with Foreground Validation for Urban Traffic Video , 2005, EURASIP J. Adv. Signal Process..

[32]  Robert B. Fisher,et al.  Parametric Trajectory Representations for Behaviour Classification , 2009, BMVC.

[33]  L. Fang,et al.  Habitat and Fish Fauna Structure in a Subtropical Mountain Stream in Taiwan before and after a Catastrophic Typhoon , 2002, Environmental Biology of Fishes.

[34]  Eric Prud'hommeaux,et al.  Interpreting relational databases in the RDF domain , 2011, K-CAP '11.

[35]  Chao Li,et al.  Real-time Detection of Abnormal Vehicle Events with Multi-Feature over Highway Surveillance Video , 2008, 2008 11th International IEEE Conference on Intelligent Transportation Systems.

[36]  Rama Chellappa,et al.  Object Detection, Tracking and Recognition for Multiple Smart Cameras , 2008, Proceedings of the IEEE.

[37]  N. Lazarevic-McManus,et al.  Performance evaluation in visual surveillance using the F-measure , 2006, VSSN '06.

[38]  Fatih Murat Porikli,et al.  Covariance Tracking using Model Update Based on Lie Algebra , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[39]  Fatih Murat Porikli,et al.  Region Covariance: A Fast Descriptor for Detection and Classification , 2006, ECCV.

[40]  Wei Li,et al.  A Rule-Based Sports Video Event Detection Method , 2009, 2009 International Conference on Computational Intelligence and Software Engineering.

[41]  Christof Koch,et al.  Detection and tracking of objects in underwater video , 2004, CVPR 2004.

[42]  Marti A. Hearst.  Trends & Controversies: Support Vector Machines , 1998, IEEE Intell. Syst..

[43]  Raphaël Troncy,et al.  LODE: Linking Open Descriptions of Events , 2009, ASWC.

[44]  Michael G. Strintzis,et al.  Knowledge-assisted semantic video object detection , 2005, IEEE Transactions on Circuits and Systems for Video Technology.

[45]  Atsushi Nanami,et al.  The Structures and Dynamics of Fish Communities in an Okinawan Coral Reef: Effects of Coral-based Habitat Structures at Sites with Rocky and Sandy Sea Bottoms , 2002, Environmental Biology of Fishes.

[46]  Mohan S. Kankanhalli,et al.  Modeling, detecting, and processing events in multimedia , 2010, ACM Multimedia.

[47]  A. Murat Tekalp,et al.  Metrics for performance evaluation of video object segmentation and tracking without ground-truth , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[48]  Concetto Spampinato,et al.  Adaptive Background Modeling Integrated With Luminosity Sensors and Occlusion Processing for Reliable Vehicle Detection , 2011, IEEE Transactions on Intelligent Transportation Systems.

[49]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[50]  David Mouillot,et al.  Amphidromous fish school migration revealed by combining fixed sonar monitoring (horizontal beaming) with fishing data , 2006 .

[51]  Christof Koch,et al.  Automated event detection in underwater video , 2003, Oceans 2003. Celebrating the Past ... Teaming Toward the Future (IEEE Cat. No.03CH37492).

[52]  Yiannis Kompatsiaris,et al.  High-level event detection in video exploiting discriminant concepts , 2011, 2011 9th International Workshop on Content-Based Multimedia Indexing (CBMI).

[53]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[54]  Irene Y. H. Gu,et al.  Joint Feature Correspondences and Appearance Similarity for Robust Visual Object Tracking , 2010, IEEE Transactions on Information Forensics and Security.

[55]  Hassan Foroosh,et al.  Euclidean path modeling for video surveillance , 2008, Image Vis. Comput..

[56]  Dirk B. Walther,et al.  Automated Video Analysis for Oceanographic Research , 2003 .

[57]  Fiona F. Evans.  Detecting fish in underwater video using the EM algorithm , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[58]  Dan Schonfeld,et al.  Fast object tracking using adaptive block matching , 2005, IEEE Transactions on Multimedia.

[59]  Mario Fernando Montenegro Campos,et al.  Particle Filter-Based Predictive Tracking for Robust Fish Counting , 2005, XVIII Brazilian Symposium on Computer Graphics and Image Processing (SIBGRAPI'05).

[60]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[61]  Gadi Katzir,et al.  Coral Reef Restoration (Bolinao, Philippines) in the Face of Frequent Natural Catastrophes , 2010 .

[62]  Jun Zhou,et al.  Autonomous fish tracking by ROV using Monocular Camera , 2006, The 3rd Canadian Conference on Computer and Robot Vision (CRV'06).

[63]  Larry S. Davis,et al.  Efficient Kernel Density Estimation Using the Fast Gauss Transform with Applications to Color Modeling and Tracking , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[64]  Nando de Freitas,et al.  Sequential Monte Carlo Methods in Practice , 2001, Statistics for Engineering and Information Science.

[65]  Hang Nguyen,et al.  Robust VLC sequence decoding exploiting additional video stream properties with reduced complexity , 2004, 2004 IEEE International Conference on Multimedia and Expo (ICME) (IEEE Cat. No.04TH8763).

[66]  Dan Schonfeld,et al.  An online motion-based particle filter for head tracking applications , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[67]  Fatih Murat Porikli,et al.  Multiplicative Background-Foreground Estimation Under Uncontrolled Illumination using Intrinsic Images , 2005, 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION'05) - Volume 1.

[68]  Raimondo Schettini,et al.  Underwater Image Processing: State of the Art of Restoration and Image Enhancement Methods , 2010, EURASIP J. Adv. Signal Process..

[69]  Concetto Spampinato,et al.  Soft-Computing Agents Processing Webcam Images to Optimize Metropolitan Traffic Systems , 2004, ICCVG.