An instance segmentation framework for in-situ plankton taxa assessment

In this paper, we propose a deep learning instance segmentation framework for particle extraction of microscopic images that aims at calculating planktonic species distribution and concentration in-situ. The framework comprises three essential functional tasks on in-situ time-series images collected from an autonomous underwater vehicle: 1) manual labeling of the captured images, 2) object localization, segmentation, and identification, and 3) class distribution and planktonic organisms concentration calculation. Our proposed framework is based on the mask R-CNN architecture provided by the Detectron2 library developed by Facebook Artificial Intelligence Research (FAIR) for instance segmentation. Due to its modular design, we compare the performance of different networks by alternating the backbone sub-network in order to choose the most suitable architecture for the task of instance and semantic segmentation. We compile a custom annotated dataset from planktonic time-series images and train the different models over this dataset to perform the instance semantic segmentation. Evaluation results of the proposed framework, utilizing the best performing deep learning architecture along with the new annotated dataset, show better performance in terms of speed and accuracy of both in-situ segmentation and classification compared to traditional segmentation methods. In addition, we observe a significant improvement in the object classification quality when we train the model over our newly annotated dataset instead of training it over the dataset generated from the traditional methods. The inferred data from our novel instance segmentation framework, which provides the particle class distribution and concentration, can then be used to assist in constructing a dynamic probability density map of planktonic communities dispersion and abundance.