Assessment of a Smartphone-Based Camera System for Coastal Image Segmentation and Sargassum monitoring

Coastal video monitoring has proven to be a valuable ground-based technique to investigate ocean processes. Presently, there is a growing need for automatic, technically efficient, and inexpensive solutions for image processing. Moreover, beach and coastal water quality problems are becoming significant and need attention. This study employs a methodological approach to exploit low-cost smartphone-based images for coastal image classification. The objective of this paper is to present a methodology useful for supervised classification for image semantic segmentation and its application for the development of an automatic warning system for Sargassum algae detection and monitoring. A pixel-wise convolutional neural network (CNN) has demonstrated optimal performance in the classification of natural images by using abstracted deep features. Conventional CNNs demand a great deal of resources in terms of processing time and disk space. Therefore, CNN classification with superpixels has recently become a field of interest. In this work, a CNN-based deep learning framework is proposed that combines sticky-edge adhesive superpixels. The results indicate that a cheap camera-based video monitoring system is a suitable data source for coastal image classification, with optimal accuracy in the range between 75% and 96%. Furthermore, an application of the method for an ongoing case study related to Sargassum monitoring in the French Antilles proved to be very effective for developing a warning system, aiming at evaluating floating algae and algae that had washed ashore, supporting municipalities in beach management.

[1]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[2]  Miquel Palmer,et al.  Using stereoscopic video cameras to evaluate seagrass meadows nursery function in the Mediterranean , 2017 .

[3]  C. Lawrence Zitnick,et al.  Fast Edge Detection Using Structured Forests , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Leonardo Damiani,et al.  A new video monitoring system in support of Coastal Zone Management at Apulia Region, Italy , 2017 .

[5]  Xiao Ma,et al.  Superpixel segmentation: A benchmark , 2017, Signal Process. Image Commun..

[6]  Daniel Buscombe,et al.  Landscape Classification with Deep Neural Networks , 2018, Geosciences.

[7]  Ian L Turner,et al.  A video-based technique for mapping intertidal beach bathymetry , 2003 .

[8]  Cordelia Schmid,et al.  Local Features and Kernels for Classification of Texture and Object Categories: A Comprehensive Study , 2006, 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'06).

[9]  R. Holman,et al.  The history and technical capabilities of Argus , 2007 .

[10]  Erika Young,et al.  Satellite images suggest a new Sargassum source region in 2011 , 2013 .

[11]  Luc Van Gool,et al.  SEEDS: Superpixels Extracted Via Energy-Driven Sampling , 2012, International Journal of Computer Vision.

[12]  Fabio Nelli,et al.  Deep Learning with TensorFlow , 2018 .

[13]  Leonardo Damiani,et al.  New algorithms for shoreline monitoring from coastal video systems , 2017, Earth Science Informatics.

[14]  Dongping Ming,et al.  Superpixel based land cover classification of VHR satellite image combining multi-scale CNN and scale parameter estimation , 2019, Earth Science Informatics.

[15]  C. Louime,et al.  Sargassum Invasion of Coastal Environments: A Growing Concern , 2017 .

[16]  Jairo Espinosa,et al.  Virtual BUOY: A video-based approach for measuring near-shore wave peak period , 2019, Comput. Geosci..

[17]  Kristen D. Splinter,et al.  Evaluation of Opportunistic Shoreline Monitoring Capability Utilizing Existing “Surfcam” Infrastructure , 2016, Journal of Coastal Research.

[18]  Elisabeth A. Addink,et al.  Object-oriented extraction of beach morphology from video images , 2006 .

[19]  B. Castelle,et al.  Sea Level at the Coast from Video-Sensed Waves: Comparison to Tidal Gauges and Satellite Altimetry , 2019, Journal of Atmospheric and Oceanic Technology.

[20]  U. Andriolo Nearshore Wave Transformation Domains from Video Imagery , 2019, Journal of Marine Science and Engineering.

[21]  Dominique Pelletier,et al.  A general framework for indicator design and use with application to the assessment of coastal water quality and marine protected area management , 2011 .

[22]  Matthew B. Blaschko,et al.  Learning Fully-Connected CRFs for Blood Vessel Segmentation in Retinal Images , 2014, MICCAI.

[23]  Rafael Almar,et al.  Wave runup video motion detection using the Radon Transform , 2017 .

[24]  Fedor Baart,et al.  An Automated Method for Semantic Classification of Regions in Coastal Images , 2015 .

[25]  Chuanmin Hu,et al.  Mapping and quantifying Sargassum distribution and coverage in the Central West Atlantic using MODIS observations , 2016 .

[26]  Chuanmin Hu,et al.  A simple, fast, and reliable method to predict Sargassum washing ashore in the Lesser Antilles , 2017 .

[27]  Daniel Conley,et al.  Video-based nearshore bathymetry estimation in macro-tidal environments , 2016 .

[28]  Chuanmin Hu A novel ocean color index to detect floating algae in the global oceans , 2009 .

[29]  Raimundo Ibaceta,et al.  Assessing the Performance of a Low-Cost Method for Video-Monitoring the Water Surface and Bed Level in the Swash Zone of Natural Beaches , 2018, Remote. Sens..

[30]  Rui Taborda,et al.  Operational Use of Surfcam Online Streaming Images for Coastal Morphodynamic Studies , 2019, Remote. Sens..

[31]  Min Wang,et al.  Very high resolution remote sensing image classification with SEEDS-CNN and scale effect analysis for superpixel CNN classification , 2018, International Journal of Remote Sensing.

[32]  Jonathan T. Barron,et al.  Semantic Image Segmentation with Task-Specific Edge Detection Using CNNs and a Discriminatively Trained Domain Transform , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Kilian Vos,et al.  Shoreline change mapping using crowd-sourced smartphone images , 2019, Coastal Engineering.

[34]  Pascal Fua,et al.  SLIC Superpixels Compared to State-of-the-Art Superpixel Methods , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Richard P. Stumpf,et al.  Applications of Satellite Ocean Color Sensors for Monitoring and Predicting Harmful Algal Blooms , 2001 .

[36]  Naif Alajlan,et al.  Using convolutional features and a sparse autoencoder for land-use scene classification , 2016 .

[37]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Chuanmin Hu,et al.  Predicting Sargassum blooms in the Caribbean Sea from MODIS observations , 2017 .

[39]  Uwe Stilla,et al.  Classification With an Edge: Improving Semantic Image Segmentation with Boundary Detection , 2016, ISPRS Journal of Photogrammetry and Remote Sensing.