Level set based shape prior and deep learning for image segmentation

Deep convolutional neural network can effectively extract hidden patterns in images and learn realistic image priors from the training set. And fully convolutional networks (FCNs) have achieved state-of-the-art performance in the image segmentation. However, these methods have the disadvantages of noise, boundary roughness and no prior shape. Therefore, this study proposes a level set with the deep prior method for the image segmentation based on the priors learned by FCNs. The FCNs can learn high-level semantic patterns from the training set. Also, the output of the FCNs represents the high-level semantic information as a probability map and the global affine transformation can obtain the optimal affine transformation of the intrinsic prior shape. Moreover, the improved level set method integrates the information of the original image, the probability map and the corrected prior shape to achieve the image segmentation. Compared with the traditional level set method of simple scenes, the proposed method solves the disadvantage of FCNs by using the high-level semantic information to segment images of complex scenes. Finally, Portrait data set are used to verify the effectiveness of the proposed method. The experimental results show that the proposed method can obtain more accurate segmentation results than the traditional FCNs.

[1]  Luc Van Gool,et al.  The Pascal Visual Object Classes Challenge: A Retrospective , 2014, International Journal of Computer Vision.

[2]  Tony F. Chan,et al.  Active contours without edges , 2001, IEEE Trans. Image Process..

[3]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[4]  Marcel Worring,et al.  Watersnakes: Energy-Driven Watershed Segmentation , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Demetri Terzopoulos,et al.  Snakes: Active contour models , 2004, International Journal of Computer Vision.

[6]  Pascal Fua,et al.  SLIC Superpixels Compared to State-of-the-Art Superpixel Methods , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Yizhou Yu,et al.  Automatic object extraction from images using deep neural networks and the level-set method , 2018, IET Image Process..

[8]  Kazumi Odaka,et al.  Adaptive Normalization of Handwritten Characters Using Global/Local Affine Transformation , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Chunming Li,et al.  Distance Regularized Level Set Evolution and Its Application to Image Segmentation , 2010, IEEE Transactions on Image Processing.

[10]  Daniel P. Huttenlocher,et al.  Efficient Graph-Based Image Segmentation , 2004, International Journal of Computer Vision.

[11]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Sankar K. Pal,et al.  A review on image segmentation techniques , 1993, Pattern Recognit..

[13]  Asad Munir,et al.  Adaptive active contours based on variable kernel with constant initialisation , 2018, IET Image Process..

[14]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Xiaochun Cao,et al.  Survey of recent progress in semantic image segmentation with CNNs , 2017, Science China Information Sciences.

[16]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Anton van den Hengel,et al.  Wider or Deeper: Revisiting the ResNet Model for Visual Recognition , 2016, Pattern Recognit..

[18]  Baba C. Vemuri,et al.  Shape Modeling with Front Propagation: A Level Set Approach , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Feng Zhang,et al.  A probability model-based level set method for biomedical image segmentation , 2005 .

[20]  D. Hubel,et al.  Receptive fields, binocular interaction and functional architecture in the cat's visual cortex , 1962, The Journal of physiology.

[21]  Bülent Sankur,et al.  Survey over image thresholding techniques and quantitative performance evaluation , 2004, J. Electronic Imaging.

[22]  Chunming Li,et al.  Minimization of Region-Scalable Fitting Energy for Image Segmentation , 2008, IEEE Transactions on Image Processing.