Novel Content Aware Pixel Abstraction for Image Semantic Segmentation

Image semantic segmentation is one of the recently researched topics due to the rise in visual deep learning-based applications. These applications work on the meaningful segments of the visual scene created by the application base. The literature identifies the current research status and highlights the existing problems of image semantic segmentation algorithms. These problems include the handling of complex images. Complex images can be of form high/low pixel intensities or dense structure regions of the image. Existing state-of-the-art deep learning algorithms fail to segment complex images semantically. For semantic segmentation of complex images, deep learning algorithms are proposed to be accompanied by pixel abstraction algorithm. The pixel abstraction algorithm creates atomic segments of the visual scene called super-pixels.  Super-pixels generate feature vectors supporting the same regions. These feature vectors reduce the computational complexity to create semantic segments of the visual scene. The pixel abstraction algorithms lack functionality due to different aspects, one of which is the initial hand-crafted seed from the user to create super-pixels that do not work for all types of visual scenarios to create accurate semantic segments. The second aspect that limits pixel abstraction algorithms' functionality is the distance measure used for super-pixel (cluster) creation. The distance measures employed in existing algorithms do not capture content-aware information of visual scene; instead, end-up creating super-pixels based on Euclidean distance, which is based on straight line distance. Hence, the created pixels are distorted and irregular. For proving the flawed functionality of the existing super-pixel creation algorithm, detailed visual analysis is presented, uncovering the indicators for future research towards the development of a novel algorithm creating continuous and regular super-pixels. For creating content-aware super-pixels, the article describes an automatic super-pixel creation algorithm based on the idea of capturing image information in relevance to the content present in it. For example, we illustrate the proposed framework in detail as two modular approaches to improve the resulting super-pixels' quality. Firstly, to automate the entire process, the probability density function is proposed to initialize the cluster centers such that hand-crafted seed is not required from the user. Secondly, to retrieve fine-grained object boundaries, a novel distance measure with induced content-aware nature and complex image handling is proposed. The novel algorithm has the potential to tackle the problem of discontinuity and irregularity of retrieved segment boundaries. Keywords:  Super-Pixels, Simple Linear Iterative Clustering, Distance Measures.