Detection of salient events in large datasets of underwater video

The aim of this work is to perform the automatic detection of events of interest, in this case defined as animal motion, in deep-sea videos and then to use the detected events as the basis for creating video abstracts. Video is collected by seafloor cameras connected to a cabled observatory network which provides power to the lights and sensors and enables two-way communication with the cameras. Continuous power and connectivity on the network permit high volumes of data to be collected. Such video data is of importance for marine biologists who are able to remotely observe species in the deep-sea environment through scheduled recordings of the video data. It is extremely time consuming for researchers looking for particular events of interest to manually search in the video database, and therefore, our study focuses on automatic detection of these events. Our approach is based on the Laptev spatio-temporal interest points detection method [1]. The output of the analysis is a summary video clip that contains all detected salient events with their associated start and end frames. We report experimental results on video abstraction using a database of videos from the NEPTUNE Canada cabled observatory.

[1]  J. Kildow,et al.  The importance of estimating the contribution of the oceans to national economies , 2010 .

[2]  John R. Kender,et al.  Rule-based semantic summarization of instructional videos , 2002, Proceedings. International Conference on Image Processing.

[3]  E. A. Widdera,et al.  Using red light for in situ observations of deep-sea fishes , 2005 .

[4]  Yueting Zhuang,et al.  Adaptive key frame extraction using unsupervised clustering , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[5]  B. Bett,et al.  Climate, carbon cycling, and deep-ocean ecosystems , 2009, Proceedings of the National Academy of Sciences.

[6]  Yael Pritch,et al.  Clustered Synopsis of Surveillance Video , 2009, 2009 Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance.

[7]  Chong-Wah Ngo,et al.  Integrating color and spatial features for content-based video retrieval , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[8]  Chong-Wah Ngo,et al.  Automatic video summarization by graph modeling , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[9]  Wayne H. Wolf,et al.  Key frame selection by motion analysis , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.

[10]  Lie Lu,et al.  Highlight sound effects detection in audio stream , 2003, 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698).

[11]  Ying Li,et al.  An Overview of Video Abstraction Techniques , 2001 .

[12]  Paolo Menesatti,et al.  Multi-parametric study of behavioural modulation in demersal decapods at the VENUS cabled observatory in Saanich Inlet, British Columbia, Canada , 2011 .

[13]  Tianming Liu,et al.  A novel video key-frame-extraction algorithm based on perceived motion energy model , 2003, IEEE Trans. Circuits Syst. Video Technol..

[14]  Avideh Zakhor,et al.  Applications of Video-Content Analysis and Retrieval , 2002, IEEE Multim..

[15]  Ivan Laptev,et al.  On Space-Time Interest Points , 2005, International Journal of Computer Vision.

[16]  D. Bailey,et al.  Temporal change in deep-sea benthic ecosystems: a review of the evidence from recent time-series studies. , 2010, Advances in marine biology.

[17]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[18]  Yvon Le Maho,et al.  Reliability of flipper-banded penguins as indicators of climate change , 2011, Nature.

[19]  Regunathan Radhakrishnan,et al.  Generation of sports highlights using motion activity in combination with a common audio feature extraction framework , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[20]  Noboru Babaguchi,et al.  Personalized abstraction of broadcasted American football video by highlight selection , 2004, IEEE Transactions on Multimedia.

[21]  F. Dirfaux.  Key frame selection to represent a video , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[22]  Xin Liu,et al.  Video summarization with minimal visual content redundancies , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[23]  Regunathan Radhakrishnan,et al.  Motion activity-based extraction of key-frames from video shots , 2002, Proceedings. International Conference on Image Processing.

[24]  Sheng Tang,et al.  An Innovative Model of Tempo and Its Application in Action Scene Detection for Movie Analysis , 2008, 2008 IEEE Workshop on Applications of Computer Vision.

[25]  Yael Pritch,et al.  Making a Long Video Short: Dynamic Video Synopsis , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[26]  Raimondo Schettini,et al.  Underwater Image Processing: State of the Art of Restoration and Image Enhancement Methods , 2010, EURASIP J. Adv. Signal Process..

[27]  Yael Pritch,et al.  Webcam Synopsis: Peeking Around the World , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[28]  Ba Tu Truong,et al.  Video abstraction: A systematic review and classification , 2007, TOMCCAP.

[29]  James M. Rehg,et al.  Statistical Color Models with Application to Skin Detection , 2004, International Journal of Computer Vision.

[30]  Alberto Del Bimbo,et al.  Semantic annotation of soccer videos: automatic highlights identification , 2003, Comput. Vis. Image Underst..

[31]  Shih-Fu Chang,et al.  Condensing computable scenes using visual complexity and film syntax analysis , 2001, IEEE International Conference on Multimedia and Expo, 2001. ICME 2001..

[32]  Francesca Antonucci,et al.  Automated Image Analysis for the Detection of Benthic Crustaceans and Bacterial Mat Coverage Using the VENUS Undersea Cabled Network , 2011, Sensors.

[33]  Daniel Mihai Toma,et al.  The New Seafloor Observatory (OBSEA) for Remote and Long-Term Coastal Ecosystem Monitoring , 2011, Sensors.

[34]  Kiyoharu Aizawa,et al.  Summarizing wearable video , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).