ICENETv2: A Fine-Grained River Ice Semantic Segmentation Network Based on UAV Images

Accurate ice segmentation is one of the most crucial techniques for intelligent ice monitoring. Compared with ice segmentation, it can provide more information for ice situation analysis, change trend prediction, and so on. Therefore, the study of ice segmentation has important practical significance. In this study, we focused on fine-grained river ice segmentation using unmanned aerial vehicle (UAV) images. This has the following difficulties: (1) The scale of river ice varies greatly in different images and even in the same image; (2) the same kind of river ice differs greatly in color, shape, texture, size, and so on; and (3) the appearances of different kinds of river ice sometimes appear similar due to the complex formation and change procedure. Therefore, to perform this study, the NWPU_YRCC2 dataset was built, in which all UAV images were collected in the Ningxia–Inner Mongolia reach of the Yellow River. Then, a novel semantic segmentation method based on deep convolution neural network, named ICENETv2, is proposed. To achieve multiscale accurate prediction, we design a multilevel features fusion framework, in which multi-scale high-level semantic features and lower-level finer features are effectively fused. Additionally, a dual attention module is adopted to highlight distinguishable characteristics, and a learnable up-sampling strategy is further used to improve the segmentation accuracy of the details. Experiments show that ICENETv2 achieves the state-of-the-art on the NWPU_YRCC2 dataset. Finally, our ICENETv2 is also applied to solve a realistic problem, calculating drift ice cover density, which is one of the most important factors to predict the freeze-up data of the river. The results demonstrate that the performance of ICENETv2 meets the actual application demand.

[1]  Yanning Zhang,et al.  ICENET: A Semantic Segmentation Deep Network for River Ice by Fusing Positional and Channel-Wise Attentive Features , 2020, Remote. Sens..

[2]  Huicong Jia,et al.  Disaster Chain Analysis of Avalanche and Landslide and the River Blocking Dam of the Yarlung Zangbo River in Milin County of Tibet on 17 and 29 October 2018 , 2019, International journal of environmental research and public health.

[4]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  R. Graf.  Estimation of the Dependence of Ice Phenomena Trends on Air and Water Temperature in River , 2020, Water.

[6]  George Papandreou,et al.  Rethinking Atrous Convolution for Semantic Image Segmentation , 2017, ArXiv.

[7]  Colin D. Rennie,et al.  Automated monitoring of river ice processes using shore-based imagery , 2017 .

[8]  A. Mynett,et al.  Challenges in modelling river flow and ice regime on the Ningxia–Inner Mongolia reach of the Yellow River, China , 2014 .

[9]  Ian D. Reid,et al.  RefineNet: Multi-path Refinement Networks for High-Resolution Semantic Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Daniel Bourgault,et al.  Shore-based photogrammetry of river ice , 2008 .

[11]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Jinchang Ren,et al.  Effective SAR sea ice image segmentation and touch floe separation using a combined multi-stage approach , 2015, 2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[13]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[14]  Daniel Rueckert,et al.  Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  H. Kalke,et al.  Support vector machine learning applied to digital images of river ice conditions , 2018, Cold Regions Science and Technology.

[17]  Peter Romanov,et al.  Global Multisensor Automated satellite-based Snow and Ice Mapping System (GMASI) for cryosphere monitoring , 2017 .

[19]  Kun Yu,et al.  DenseASPP for Semantic Segmentation in Street Scenes , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[20]  Na Zhang,et al.  Detection of sea ice in sediment laden water using MODIS in the Bohai Sea: a CART decision tree method , 2015 .

[21]  R. Whaley,et al.  Identifying historic river ice breakup timing using MODIS and Google Earth Engine in support of operational flood monitoring in Northern Ontario , 2019, Remote Sensing of Environment.

[22]  K. Lindenschmidt,et al.  Integration of space-borne and air-borne data in monitoring river ice processes in the Slave River, Canada , 2016 .

[23]  N. Frolova,et al.  Dangerous ice phenomena on the lowland rivers of European Russia , 2017, Natural Hazards.

[24]  Gang Yu,et al.  BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentation , 2018, ECCV.

[25]  Yumei Wu,et al.  Discussion on sea ice segmentation of high resolution radar data , 2016, 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[26]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[27]  Xiaogang Wang,et al.  Pyramid Scene Parsing Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Abhineet Singh,et al.  River Ice Segmentation With Deep Learning , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[29]  Franz J. Meyer,et al.  Analyzing floating and bedfast lake ice regimes across Arctic Alaska using 25 years of space-borne SAR imagery , 2018 .

[30]  Jun Fu,et al.  Dual Attention Network for Scene Segmentation , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).