Learning deep-sea substrate types with visual topic models

We propose and evaluate a method for learning deep-sea substrate types using video recorded with a remotely operated vehicle (ROV). The goal of this work is to create a labelled spatial map of substrate types from ROV video in order to support biological and geological domain research. The output of our method describes the mixtures of geological features such as sediment and types of lava flow in images taken at a set of points chosen from an ROV dive. The main contribution of this work is the assembly of a pipeline combining several unique approaches which is able to robustly generate substrate type mixtures under the varying lighting and perspective conditions of deep-sea ROV dive videos. The pipeline comprises three main components: sampling, in which a trained classifier and spatial sampling is used to select relevant frames from the dataset; feature extraction, in which the improved local binary pattern descriptor (ILBP) is used to generate a Bag of Words (BoW) representation of the dataset; and topic modelling in which a variant of Latent Dirichlet Allocation (LDA), is used to infer the mixture of substrate types represented by each BoW. Our method significantly outperforms techniques relying on keypoint based features rather than texture based features, and k-means rather than LDA, demonstrating that our proposed pipeline accurately learns and identifies visible substrate types.

[1]  Thomas Kuhn,et al.  Seabed Classification Using a Bag-of-Prototypes Feature Representation , 2014, 2014 ICPR Workshop on Computer Vision for Analysis of Underwater Imagery.

[2]  Gregory Dudek,et al.  Modeling curiosity in a mobile robot for long-term autonomous exploration and monitoring , 2015, Autonomous Robots.

[3]  O. Pizarro,et al.  Towards image-based marine habitat classification , 2008, OCEANS 2008.

[4]  Harold W. Kuhn,et al.  The Hungarian method for the assignment problem , 1955, 50 Years of Integer Programming.

[5]  Stefan B. Williams,et al.  Autonomous underwater vehicle–assisted surveying of drowned reefs on the shelf edge of the Great Barrier Reef, Australia , 2010, J. Field Robotics.

[6]  David M. Blei,et al.  Probabilistic topic models , 2012, Commun. ACM.

[7]  Fei-Fei Li,et al.  Image Segmentation with Topic Random Field , 2010, ECCV.

[8]  Hanumant Singh,et al.  Anomaly detection in unstructured environments using Bayesian nonparametric scene modeling , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[9]  Shengcai Liao,et al.  Learning Multi-scale Block Local Binary Patterns for Face Recognition , 2007, ICB.

[10]  Chong Wang,et al.  Simultaneous image classification and annotation , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Stefan B. Williams,et al.  Topic-based habitat classification using visual data , 2009, OCEANS 2009-EUROPE.

[12]  Hervé Glotin,et al.  Efficient Bag of Scenes Analysis for Image Categorization , 2013, ICPRAM.

[13]  P. Lawton,et al.  Using object‐based image analysis to determine seafloor fine‐scale features and complexity , 2015 .

[14]  Gregory Dudek,et al.  Gibbs Sampling Strategies for Semantic Perception of Streaming Video Data , 2015, ArXiv.

[15]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[16]  Hervé Glotin,et al.  Bayesian Non-parametric Parsimonious Gaussian Mixture for Clustering , 2014, 2014 22nd International Conference on Pattern Recognition.